Snowflake Dynamic Tables: TARGET_LAG, Refresh Warehouses, and When to Keep dbt
How Dynamic Tables actually refresh: TARGET_LAG, the warehouse that pays for it, streams vs DT, and why I still keep dbt for gold.
- snowflake
- pipelines
- sql
Topic
23 posts ·view all posts
How Dynamic Tables actually refresh: TARGET_LAG, the warehouse that pays for it, streams vs DT, and why I still keep dbt for gold.
What a micropartition is, how clustering and sargable filters prune, why wrapping columns and SELECT * blow the scan, and how to read partitions_scanned.
CREATE STREAM on a table, consume it in a Task MERGE, SHOW STREAMS, stale offsets, and the cases where Airflow still orchestrates dbt and everything outside Snowflake.
CREATE … CLONE shares micropartitions until someone writes. How storage grows, how to clone at a timestamp, how I use clones in CI, and why I never mutate gold through a clone.
How to use Cortex LLM functions inside pipelines — COMPLETE, CLASSIFY, EMBED — with cost controls, schema checks, retries, and a warehouse-native RAG shape.
A practical guide to Snowflake Time Travel and Fail-safe: retention, UNDROP, cloning at a timestamp, storage cost, and a restore runbook you can test.
When to use Snowflake-managed Iceberg vs native tables: external volumes, catalogs, deletion vectors, and the interoperability jobs Iceberg actually wins.
An honest head-to-head: SQL warehousing vs Spark lakehouse, Iceberg interoperability, Cortex vs Mosaic, and when you actually need both.
The pricing models decide the architecture: warehouse-hours versus on-demand scans, plus multi-cloud vs GCP-native gravity.
Table-format choice in 2026: deletion vectors, catalogs, and which engine you are willing to lock in.
Warehouse sizing, auto-suspend, clustering, ACCOUNT_USAGE queries, and resource monitors — estimate the bill from usage before you change prod.
A practical comparison of Snowpipe file ingestion and Snowpipe Streaming, covering real latency ranges, the credit math per million rows, and how to pick without over-engineering.
How to move from per-column masking policies to tag-based masking and mapping-table row access policies, including the performance traps and the audit queries that prove coverage.
How Snowflake secure data sharing works across accounts, regions, and clouds, when to use a reader account, and the governance patterns that keep a share from leaking more than you intended.
A production comparison of Snowpark Python and plain SQL in Snowflake, covering lazy DataFrames, UDF serialization costs, and the four workloads where Python genuinely earns its place.
How Time Travel retention, fail-safe, and zero-copy clones bill against storage, plus the migration and recovery patterns that make the storage cost worth paying.
How to split functional roles from access roles, use future grants and managed access schemas, and build a Snowflake permission model that stays reviewable as teams change.
How to pick a Snowflake warehouse size from evidence instead of intuition, using spilling, queueing, and scaling tests to decide between a bigger warehouse and more clusters.
A ranked playbook for reducing Snowflake spend: auto-suspend defaults, right-sizing warehouses, resource monitors that actually fire, and the ACCOUNT_USAGE queries that name your top ten offenders.
How Cortex Analyst semantic models and Cortex Search hybrid retrieval fit together, what the YAML actually controls, and where the accuracy comes from.
How the search access path works, what it costs to build and maintain, which predicates it serves, and when clustering keys are the better buy.
Clustering depth, what automatic reclustering actually costs, and a decision framework for clustering keys vs Search Optimization vs doing nothing.
How I read the operator tree: TableScan bytes, partition pruning, local and remote spilling, and the exchange steps that quietly dominate runtime.