DataLane

Resources

A path per stack, not a bookmark dump

Pick one stack. Finish the weeks. Then use the YouTube, course, and book list for that stack — official channels first, paid courses second. Practice withinterview questionsor acertification test.

SQL

2–3 weeks
  1. Window functions: dedup, top-N, lag, running totals, sessionize
  2. Practice every pattern in the SQL Playground
  3. Read Fundamentals of Data Engineering ch. on querying

Python

2 weeks
  1. Write a small extract + pandas transform you can re-run
  2. Add row-count and null checks before the load
  3. Only then wrap it in Airflow or a Task

Airflow

2 weeks
  1. Run the official Docker compose locally
  2. Build the weather ETL: extract → parquet → idempotent load
  3. Learn catchup, data intervals, and why XCom is not a data bus

dbt

2–3 weeks
  1. Init a project; staging 1:1 with sources, marts at the grain
  2. unique + not_null on every mart key; then dbt build
  3. Add slim CI (state:modified+) once you have a prod manifest

Snowflake

3–4 weeks
  1. Cost first: auto-suspend, warehouse split, ACCOUNT_USAGE
  2. Time Travel vs Fail-safe — practice UNDROP on a disposable table
  3. Then Cortex as a transform, not a chat demo

Databricks

3 weeks
  1. Medallion: bronze / silver / gold with Delta
  2. When Iceberg is the shared lake instead
  3. Compare warehouse vs lakehouse for your workload

BigQuery

1–2 weeks
  1. Partition + cluster before you write the dashboard
  2. Watch bytes scanned, not just runtime
  3. Compare scan billing vs Snowflake credits

Kafka

2–3 weeks
  1. Topics, partitions, consumer groups until you can draw them
  2. Then Kafka vs Kinesis: control vs less ops
  3. Only after that: exactly-once and compacted topics

PySpark

3 weeks
  1. Driver vs executors; a first job that reads Parquet
  2. Partitioning and shuffles — why countDistinct is expensive
  3. When DuckDB or SQL is enough instead

AWS

2 weeks
  1. Map the job: S3 land, Glue or Spark transform, Redshift or Athena serve
  2. Kinesis vs MSK only after you know the Kafka model
  3. Skip services that duplicate a warehouse you already pay for

Azure

2 weeks
  1. ADF for orchestration, Synapse vs Fabric for analytics
  2. Do not run the same pipeline in all three “because we can”
  3. Read the Fabric fit article before a migration pitch

DuckDB

1 week
  1. Local Parquet / CSV pipelines without a warehouse
  2. Use the Playground for window practice
  3. MotherDuck only when you need to share the same files

AI / RAG

3 weeks
  1. Treat RAG ingest as ETL: chunk, embed, version, ACL
  2. Compare vector stores only after chunking works
  3. Then LLMs as pipeline steps, not a chatbot

MLOps

2 weeks
  1. Feature tables as incrementally built marts
  2. Training jobs as DAGs with data tests
  3. Know where the DE stops and the ML engineer starts

YouTube channels that cover the whole stack

Start here if you do not know which vendor playlist to open. Then jump to the stack section below.

YouTube, courses, and books by stack

Official docs and channels first. Udemy links go to topic pages so you can pick a recently updated course — we do not rank a single paid instructor. Thestack cataloglists every technology, including ones without a dedicated resource block yet.

Snowflake

YouTube

Courses

Books

dbt

YouTube

  • dbt Labs

    Coalesce talks, Coalesce keynotes, and how the product actually changed.

  • dbt Learn

    The official fundamentals path, still the right first 4 hours.

Courses

Books

Airflow

YouTube

  • Apache Airflow

    Summit talks and release walkthroughs — watch Airflow 3 sessions first.

  • Astronomer

    Clear DAG design talks; treat product pitches as optional.

Courses

  • Udemy — Airflow

    Pick a course updated after Airflow 2.7+. Skip anything that still teaches BashOperator-only DAGs.

  • The Astronomer Academy

    Free modules on deferrable operators, datasets, and local dev.

Books

Databricks

YouTube

  • Databricks

    DAIS keynotes, Delta, Unity Catalog, and Mosaic/AI sessions.

  • Advancing Analytics

    Practitioner Spark/Databricks walkthroughs with real clusters.

Courses

Books

Apache Spark

YouTube

Courses

Books

Kafka

YouTube

  • Apache Kafka

    Kafka Summit talks on transactions, KRaft, and Connect.

  • Confluent

    Schema Registry, ksqlDB, and Flink — know what is Kafka vs Confluent.

Courses

Books

SQL

YouTube

Courses

Books

Python

YouTube

Courses

Books

AWS

YouTube

  • AWS Developers

    Glue, Redshift, Athena, and Kinesis sessions from re:Invent.

  • AWS Events

    Keynotes when a service actually GA'd, not just a blog title.

Courses

Books

Azure / Fabric

YouTube

  • Microsoft Azure

    Fabric, Synapse, and ADF sessions — watch Fabric ones from the last 12 months only.

  • Guy in a Cube

    Power BI + Fabric serving layer, which is where most Fabric cost shows up.

Courses

Books

GCP / BigQuery

YouTube

Courses

Books

DuckDB

YouTube

  • DuckDB

    Official talks on the vectorized engine and extensions.

Courses

Books

MLOps

YouTube

Courses

Books

AI / RAG

YouTube

Courses

Books

Data modeling

YouTube

Courses

Books

Iceberg & lakehouse

YouTube

Courses

Books

Dagster

YouTube

  • Dagster

    Assets, checks, and Dagster+ walkthroughs from the makers.

Courses

  • Dagster docs (official)

    Start with software-defined assets. Skip random Udemy clones of Airflow DAGs in Dagster syntax.

Books

Prefect

YouTube

  • Prefect

    Flows, deployments, and work pools from the product team.

Courses

Books

ClickHouse

YouTube

  • ClickHouse

    Official meetups on MergeTree, inserts, and query performance.

Courses

Books

Trino

YouTube

  • Trino

    Community talks on connectors, Iceberg, and cost-based optimization.

Courses

  • Trino docs (official)

    Catalogs and connectors first. Starburst University if you are on the commercial build.

Books

PostgreSQL

YouTube

Courses

Books

Airbyte

YouTube

  • Airbyte

    Connector and platform walkthroughs. Filter for recent major versions.

Courses

Books

Fivetran

YouTube

  • Fivetran

    MAR, destinations, and schema change talks from the vendor. Treat pricing as marketing until you model it.

Courses

Books

Debezium

YouTube

Courses

Books

Microsoft Fabric

YouTube

  • Microsoft Fabric

    Official and MVP sessions. Prefer 2025+ — the item model moved.

Courses

Books

Amazon Redshift

YouTube

Courses

Books

Terraform

YouTube

  • HashiCorp

    State, modules, and provider talks. Skip the 2019 “first resource” videos.

Courses

Books

Kubernetes

YouTube

Courses

Books

Polars

YouTube

  • Polars

    Lazy frames and expressions from the maintainers.

Courses

Books

Data quality

YouTube

Courses

  • GX docs (official)

    Start with a few expectations on grain. Skip enterprise tours until you have owners.

Books

BI & semantic layer

YouTube

Courses

Books

Books that stay useful on every stack

Fundamentals of Data Engineering (Reis & Housley)

The lifecycle book. Read this before another tool tutorial.

Designing Data-Intensive Applications (Kleppmann)

Storage, replication, streams — why warehouses behave the way they do.

The Data Warehouse Toolkit (Kimball)

Grain, facts, and SCDs. Still how gold tables should be designed.

Staff Engineer (Larson)

How senior data engineers actually grow influence. Not a Spark book.

Get new paths when a stack changes

Vendor pricing and APIs move. Subscribers hear when a path is rewritten.

Newsletter signup is not live yet. Use the contact form if you want to be notified.

↑↓ navigate openesc close