Airflow CLI & Concepts cheat sheet
TaskFlow patterns, testing commands, backfills, and scheduling gotchas — one printable page.
TaskFlow API (modern DAGs)
@dag(schedule="@daily", start_date=..., catchup=False) def my_pipeline(): ...- Define a DAG as a decorated function; call it at module bottom.
@task def extract() -> dict: ...- Each task is a decorated function; return values pass via XCom automatically.
load(transform(extract()))- Dependencies from plain function calls.
@task(retries=3, retry_delay=timedelta(minutes=5))- Per-task retry policy — every production task should have one.
CLI: testing and debugging
airflow dags list-import-errors- First stop when a DAG is missing from the UI.
airflow dags test my_dag 2026-08-01- Run a whole DAG locally for a date, no scheduler needed.
airflow tasks test my_dag my_task 2026-08-01- Run one task without dependencies or retries.
python dags/my_dag.py- Cheapest smoke test: import errors surface immediately.
Backfills and state
airflow dags backfill my_dag -s 2026-07-01 -e 2026-07-31- Re-run a date range.
airflow tasks clear my_dag -s 2026-07-01 -e 2026-07-02- Reset task states so the scheduler re-runs them.
airflow dags pause / unpause my_dag- Stop/start scheduling without deleting anything.
Scheduling gotchas
schedule="@daily" + catchup=False- Without catchup=False, Airflow backfills every interval since start_date on first enable.
data_interval_start / data_interval_end- A run stamped Aug 1 processes Aug 1 data but executes after the interval ends (Aug 2 00:00).
{{ ds }}, {{ data_interval_start }}- Template variables — never use datetime.now() in task logic; runs must be reproducible.
Production habits
default_args = {"retries": 2, "retry_delay": ..., "on_failure_callback": alert}- Set retries and alerting at the DAG level; override per task when needed.
max_active_runs=1- Prevent overlapping runs when a task writes non-idempotently.
idempotent tasks: DELETE+INSERT or MERGE by key- A rerun task must produce the same result — the golden rule of reliable pipelines.
From DataLane — tutorials at/blog, practice SQL live in theplayground.