Azure

Solution

ADF moves and orchestrates (IR, linked services). Synapse is the older SQL/Spark analytics plane. Fabric is the SaaS lakehouse (OneLake, warehouse, pipelines). Do not run the same load in all three. Pick the plane the company already pays for, then the service.

Solution

Azure IR talks to public PaaS. A self-hosted IR sits inside the VNet / on-prem so you can reach SQL Server that has no public endpoint. “We opened 1433 to the internet so we could use Azure IR” is the wrong answer.

Solution

Store last_success_ts (or an increasing id) in a control table. Source query: WHERE updated_at > watermark. On success, update the watermark in the same transaction story as the load (or you skip / duplicate). This is still the most common DP-203 answer.

Solution

Shortcut when the source of truth stays in ADLS / S3 / another workspace and you only need to query it. Copy when you need a Fabric-native table, different lifecycle, or a transform that must own the files. Shortcuts are not a backup.

Solution

Serverless: per-TB scanned, great for exploration on a lake. Dedicated: provisioned DWU, predictable BI, you pay for the pool whether you query or not. Mixing both for the same gold table without a reason is how the bill surprises finance.

Solution

Azure Key Vault linked service + secret references. Never commit connection strings in the repo. Managed identity to Key Vault and to Storage. This is a hygiene question; failing it ends the loop.

Solution

A catalog with owners and classifications beats a Confluence page that rot. If you cannot name the producer, you cannot page.

Solution

ADF mapping can work; at serious Spark scale many teams still submit a Databricks/Synapse Spark job and let ADF only orchestrate.

Solution

Self-hosted IR or private endpoints. Opening SQL to the internet is the wrong answer in 2026.

Solution

Pause / scale down / don’t run dedicated compute you do not use. Same FinOps story as Synapse DWU.

Solution

The VM NIC or disk, not “Azure.” Scale out SHIR nodes, or land files in ADLS and transform with Spark. Mapping Data Flows on 2 TB is how the bill and the runtime both surprise you.

Solution

When the lake table is not Delta/Iceberg-shaped for the semantic model, or you need a frozen extract for a board meeting. Direct Lake still needs a clean gold grain. Import is not “legacy” if the model is 400 MB.

Solution

The date dim should be replicated; the fact should distribute on a join key used in large joins. EVEN + broadcast of a huge table is the classic “why is this 2 hours” story.

Solution

Key Vault (or Databricks secrets). Parameters end up in JSON, git, and ARM exports. Rotation is the test: can you rotate without editing 40 pipelines?

Solution

Capture for “land Avro/Parquet to ADLS, batch later.” Streaming job when you need watermarks, joins, or a 2-minute mart. Capture is not a stream processor.

Solution

Wrong collection/scope, the identity cannot list the catalog, or tables live in a workspace Purview is not connected to. A wiki page is not a scan. Fix the connection before buying another catalog.