The AI-Assisted Data Engineer: A Practical Daily Workflow
How to actually use AI coding assistants for pipeline work — SQL generation, dbt refactoring, debugging, and documentation — plus the failure modes to watch for.
- ai
- productivity
- copilot
Topic
5 posts ·view all posts
How to actually use AI coding assistants for pipeline work — SQL generation, dbt refactoring, debugging, and documentation — plus the failure modes to watch for.
A four-layer testing setup for Airflow: parse checks, DAG structure tests, task unit tests with mocked hooks, and dag.test() end-to-end runs against local infrastructure.
The Airflow 3 changes that break real DAGs: the Task SDK boundary, logical_date becoming optional, removed context keys, DAG versioning, and a pre-upgrade checklist.
Which Jinja patterns are worth the indirection in a production dbt project, how to test macros, and the metaprogramming that quietly makes a project unmaintainable.
How I lay out staging, intermediate, and marts so a dbt project still makes sense after two years: naming rules, ref boundaries, and the folders that always rot.