Snowflake Dynamic Tables: TARGET_LAG, Refresh Warehouses, and When to Keep dbt
How Dynamic Tables actually refresh: TARGET_LAG, the warehouse that pays for it, streams vs DT, and why I still keep dbt for gold.
- snowflake
- pipelines
- sql
All stacks · Warehouses & analytics
Cloud warehousing, performance, and cost control.
How Dynamic Tables actually refresh: TARGET_LAG, the warehouse that pays for it, streams vs DT, and why I still keep dbt for gold.
What a micropartition is, how clustering and sargable filters prune, why wrapping columns and SELECT * blow the scan, and how to read partitions_scanned.
CREATE STREAM on a table, consume it in a Task MERGE, SHOW STREAMS, stale offsets, and the cases where Airflow still orchestrates dbt and everything outside Snowflake.
CREATE … CLONE shares micropartitions until someone writes. How storage grows, how to clone at a timestamp, how I use clones in CI, and why I never mutate gold through a clone.
How to use Cortex LLM functions inside pipelines — COMPLETE, CLASSIFY, EMBED — with cost controls, schema checks, retries, and a warehouse-native RAG shape.
A practical guide to Snowflake Time Travel and Fail-safe: retention, UNDROP, cloning at a timestamp, storage cost, and a restore runbook you can test.
When to use Snowflake-managed Iceberg vs native tables: external volumes, catalogs, deletion vectors, and the interoperability jobs Iceberg actually wins.
Warehouse sizing, auto-suspend, clustering, ACCOUNT_USAGE queries, and resource monitors — estimate the bill from usage before you change prod.
A practical comparison of Snowpipe file ingestion and Snowpipe Streaming, covering real latency ranges, the credit math per million rows, and how to pick without over-engineering.
How to move from per-column masking policies to tag-based masking and mapping-table row access policies, including the performance traps and the audit queries that prove coverage.
How Snowflake secure data sharing works across accounts, regions, and clouds, when to use a reader account, and the governance patterns that keep a share from leaking more than you intended.
A production comparison of Snowpark Python and plain SQL in Snowflake, covering lazy DataFrames, UDF serialization costs, and the four workloads where Python genuinely earns its place.
How Time Travel retention, fail-safe, and zero-copy clones bill against storage, plus the migration and recovery patterns that make the storage cost worth paying.
How to split functional roles from access roles, use future grants and managed access schemas, and build a Snowflake permission model that stays reviewable as teams change.
How to pick a Snowflake warehouse size from evidence instead of intuition, using spilling, queueing, and scaling tests to decide between a bigger warehouse and more clusters.
A ranked playbook for reducing Snowflake spend: auto-suspend defaults, right-sizing warehouses, resource monitors that actually fire, and the ACCOUNT_USAGE queries that name your top ten offenders.
How the search access path works, what it costs to build and maintain, which predicates it serves, and when clustering keys are the better buy.
Clustering depth, what automatic reclustering actually costs, and a decision framework for clustering keys vs Search Optimization vs doing nothing.
How I read the operator tree: TableScan bytes, partition pruning, local and remote spilling, and the exchange steps that quietly dominate runtime.
Click-ops Snowflake grants will not exist in the next account. Modules for warehouse plus IAM, secrets out of state, UI to explore, Terraform for anything that must last.
An honest head-to-head: SQL warehousing vs Spark lakehouse, Iceberg interoperability, Cortex vs Mosaic, and when you actually need both.
The pricing models decide the architecture: warehouse-hours versus on-demand scans, plus multi-cloud vs GCP-native gravity.
Table-format choice in 2026: deletion vectors, catalogs, and which engine you are willing to lock in.
Native vector search in Snowflake, BigQuery, and pgvector has quietly become good enough for most RAG workloads. Here are the thresholds, the real costs, and when a dedicated store still wins.
A production comparison of Redshift Serverless and Snowflake: how RPU-seconds differ from credits, where each one bends under concurrency, and what actually breaks in a migration.
Snowflake changed the warehouse contract: storage, compute, and services scale independently, you pay per second of compute, and features like zero-copy cloning and Time Travel come standard instead of as weekend projects. That flexibility cuts both ways — the same elasticity that makes a proof of concept effortless makes an unmonitored account expensive.
The tutorials here come from running Snowflake in production, not from paraphrasing the docs. They cover the parts that decide whether your platform is fast and affordable: how micro-partitions and pruning actually work, when clustering pays for itself, how Streams and Tasks compare with Dynamic Tables, what serverless features really cost, and how to design RBAC and masking so governance is not an afterthought.
Storage and compute are separated, so you can scale each independently and run many isolated workloads against the same data without contention. Per-second billing, zero-copy cloning, Time Travel, and secure data sharing are native — on legacy warehouses those are either impossible or major engineering projects.
Compute is billed in credits per second while a warehouse runs, with a 60-second minimum each time it resumes. Each size doubles the credit rate (XS = 1 credit/hour up to 6XL = 512). Storage is billed separately per compressed TB-month, and serverless features like Snowpipe and Tasks meter their own credits.
Idle warehouses with long auto-suspend settings, oversized warehouses running small queries, and unbounded serverless usage. Setting auto-suspend to 60 seconds, testing one size smaller, and adding resource monitors fixes the majority of runaway bills we see.
Learn the one your team runs; the concepts transfer. Snowflake is the stronger default when SQL is the primary interface and you want minimal operations. Databricks wins for Spark-heavy and ML-heavy work. Iceberg support on both means the storage layer is converging anyway.
Yes — the architecture, performance, and cost articles map directly to SnowPro Core objectives, and the practice question sets under /certs/ include SnowPro-style exams with explanations.
One email a week with our latest tutorials. No spam.
Newsletter signup is not live yet. Use the contact form if you want to be notified.