Topic
16 posts ·view all posts
Polars🟡IntermediateA decision guide rather than a benchmark: what the expression API and lazy engine actually buy you, which pandas jobs will not get faster, and the ecosystem gaps that still send work back to pandas.
Python🟡IntermediateWhere the memory actually goes in a pandas job, how much dtype selection and Arrow-backed strings really save, and when to stop loading and start streaming.
Python🟡IntermediateA production migration guide: the expression API, lazy frames and query optimization, null versus NaN semantics, and the pandas patterns with no polars equivalent.
Snowflake🟡IntermediateA production comparison of Snowpark Python and plain SQL in Snowflake, covering lazy DataFrames, UDF serialization costs, and the four workloads where Python genuinely earns its place.
SQL🟡IntermediateDISTINCT, GROUP BY, ROW_NUMBER, QUALIFY, MERGE, and stream-side dedup compared on cost and semantics, plus how to pick the survivor row deterministically.
Apache Spark🟡IntermediateHow Spark executor memory is actually divided, why most OOMs are one oversized partition rather than a small cluster, and the six failures I can now diagnose from the error text alone.
BigQuery🟡IntermediateWhat BigQuery reads before it reads your data: partition metadata, Capacitor block statistics, and the specific query shapes that silently turn a pruned scan into a full one.
Snowflake🟡IntermediateHow to pick a Snowflake warehouse size from evidence instead of intuition, using spilling, queueing, and scaling tests to decide between a bigger warehouse and more clusters.
Apache Spark🟡IntermediateOne task at 40 minutes while 399 finished in 20 seconds is skew. Here is how I find the offending key, and the four fixes ranked by how much they cost me.
SQL🟡IntermediateWhat actually happens to a CTE in Postgres, Snowflake, and BigQuery, when it blocks predicate pushdown, and when referencing it twice costs you two scans.
Apache Spark🟡IntermediateA shuffle is a disk write, a network fetch, and a sort you did not ask for. Here is what it costs, how to spot it in the Spark UI, and the query rewrites that delete it.
SQL🟡IntermediateHow window frames actually evaluate, why RANGE and ROWS give different answers on ties, and six window patterns that delete self-joins from your models.
Snowflake🟡IntermediateHow the search access path works, what it costs to build and maintain, which predicates it serves, and when clustering keys are the better buy.
Snowflake🟡IntermediateClustering depth, what automatic reclustering actually costs, and a decision framework for clustering keys vs Search Optimization vs doing nothing.
SQL🟡IntermediateFive SQL habits that scan more than they should: SELECT *, functions on filter columns, DISTINCT as a bug fix, OR-joins, and per-row UDF calls.
Snowflake🟡IntermediateHow I read the operator tree: TableScan bytes, partition pruning, local and remote spilling, and the exchange steps that quietly dominate runtime.
