Azure

Solution

ADF moves and orchestrates (IR, linked services). Synapse is the older SQL/Spark analytics plane. Fabric is the SaaS lakehouse (OneLake, warehouse, pipelines). Do not run the same load in all three. Pick the plane the company already pays for, then the service.

Solution

Azure IR talks to public PaaS. A self-hosted IR sits inside the VNet / on-prem so you can reach SQL Server that has no public endpoint. “We opened 1433 to the internet so we could use Azure IR” is the wrong answer.

Solution

Store last_success_ts (or an increasing id) in a control table. Source query: WHERE updated_at > watermark. On success, update the watermark in the same transaction story as the load (or you skip / duplicate). This is still the most common DP-203 answer.

Solution

Shortcut when the source of truth stays in ADLS / S3 / another workspace and you only need to query it. Copy when you need a Fabric-native table, different lifecycle, or a transform that must own the files. Shortcuts are not a backup.

Solution

Serverless: per-TB scanned, great for exploration on a lake. Dedicated: provisioned DWU, predictable BI, you pay for the pool whether you query or not. Mixing both for the same gold table without a reason is how the bill surprises finance.

Solution

Azure Key Vault linked service + secret references. Never commit connection strings in the repo. Managed identity to Key Vault and to Storage. This is a hygiene question; failing it ends the loop.

Solution

A catalog with owners and classifications beats a Confluence page that rot. If you cannot name the producer, you cannot page.

Solution

ADF mapping can work; at serious Spark scale many teams still submit a Databricks/Synapse Spark job and let ADF only orchestrate.

Solution

Self-hosted IR or private endpoints. Opening SQL to the internet is the wrong answer in 2026.

Solution

Pause / scale down / don’t run dedicated compute you do not use. Same FinOps story as Synapse DWU.