Databricks CLI & Unity Catalog cheat sheet
Bundles, jobs, clusters, and Unity Catalog grants — the commands and SQL for running Databricks from code.
CLI setup
databricks configure --host https://myorg.cloud.databricks.com- Writes ~/.databrickscfg. Use named profiles per workspace and pass --profile to every command.
databricks auth login --host ... --profile prod- OAuth U2M flow, preferred over long-lived personal access tokens for interactive use.
databricks current-user me- The connectivity smoke test. Run it before debugging anything else.
DATABRICKS_HOST / DATABRICKS_TOKEN- Environment variables for CI, where an interactive login is impossible.
Asset bundles
bundle: name: analytics targets: prod: mode: production workspace: host: https://myorg.cloud.databricks.com- databricks.yml is the deployment unit — jobs, pipelines, and clusters as code instead of UI clicks.
databricks bundle validate -t prod- Catches config errors before touching the workspace. Always run it in CI.
databricks bundle deploy -t prod- Idempotent deploy of every resource in the bundle. Production mode blocks concurrent runs and pauses schedules correctly.
databricks bundle run daily_orders -t prod- Trigger a bundle-defined job and stream its output locally.
Jobs and clusters
databricks jobs list --output json- Pipe to jq to audit which jobs still run on all-purpose clusters — usually the top cost finding.
databricks jobs run-now --job-id 1234 --notebook-params '{"run_date":"2026-08-01"}'- Ad-hoc run with parameters. Parameterize the date rather than reading the wall clock inside the notebook.
new_cluster with autotermination_minutes: 20- Job clusters spin up per run and die after. The single biggest Databricks cost saver versus all-purpose clusters.
data_security_mode: SINGLE_USER or USER_ISOLATION- Required for Unity Catalog access. Legacy no-isolation clusters cannot read UC tables.
databricks clusters events --cluster-id ...- Reveals spot-instance reclamation and OOM terminations behind a mysteriously failed job.
Unity Catalog structure
CREATE CATALOG prod MANAGED LOCATION 's3://my-uc-root/prod'- Three-level namespace is catalog.schema.table. One catalog per environment is the layout that ages best.
CREATE EXTERNAL LOCATION lake URL 's3://lake/bronze' WITH (STORAGE CREDENTIAL cred)- External locations plus storage credentials replace instance profiles and mount points.
CREATE SCHEMA prod.silver MANAGED LOCATION 's3://my-uc-root/prod/silver'- Schema-level managed locations keep storage layout predictable and make lifecycle policies possible.
system.access.audit / system.billing.usage- System tables for audit and cost attribution. Query them like any other table — no API export needed.
Grants
GRANT USE CATALOG ON CATALOG prod TO `data-analysts`- USE CATALOG and USE SCHEMA are prerequisites. Without them a SELECT grant appears not to work at all.
GRANT SELECT ON SCHEMA prod.gold TO `data-analysts`- Grant at schema level so new tables inherit access. Table-by-table grants become unmaintainable fast.
GRANT ALL PRIVILEGES ON SCHEMA prod.silver TO `data-engineers`- Ownership plus full privileges for the producing team, read-only for everyone else.
SHOW GRANTS ON TABLE prod.gold.fct_orders- The audit command. Run it before claiming a permission problem is a platform bug.
CREATE FUNCTION prod.gold.mask_email(e STRING) RETURN CASE WHEN is_account_group_member('pii-readers') THEN e ELSE '***' END- Column masking via a UDF applied with SET MASK — UC's dynamic masking equivalent.
Delta maintenance
OPTIMIZE prod.silver.orders- Compacts small files. Streaming and frequent MERGE writes make this mandatory, not optional.
ALTER TABLE prod.silver.orders CLUSTER BY (ordered_at, customer_id)- Liquid clustering. Supersedes ZORDER for new tables and adapts without a full rewrite.
VACUUM prod.silver.orders RETAIN 168 HOURS- Removes files older than the retention window. Never go below your time-travel requirement.
DESCRIBE HISTORY prod.silver.orders- Every commit with operation metrics. The first place to look when row counts change unexpectedly.
From DataLane — tutorials at/blog, practice SQL live in theplayground.