Azure Data Engineering in 2026: Data Factory, Synapse, and Where Fabric Fits

Azure’s data story has changed more in the last few years than any other cloud’s, mostly because of Microsoft Fabric. Here is the current lay of the land and how to choose.

The mistake is treating Fabric, Synapse, ADF, and Databricks as four logos you collect. The job is still: land files, transform to a grain, serve BI, and know who pays the capacity bill.

flowchart LR
  src[Sources] --> adf[Data Factory]
  adf --> lake[ADLS / OneLake]
  lake --> engine[Databricks or Fabric]
  engine --> serve[Power BI / warehouse]

The classic stack (still running most of production)

Azure Data Factory (ADF). The workhorse for ingestion and orchestration. Its 100+ connectors make “copy data from X to the lake” a configuration task rather than code. Mapping Data Flows can transform too, but most teams keep ADF for movement and do transforms elsewhere. IR (integration runtime) — cloud vs self- hosted — is the decision that determines whether on-prem SQL Server is a connector or a project.

ADLS Gen2. The data lake; Azure’s S3. Hierarchical namespace makes it friendlier for analytics than plain blob storage. Use abfss:// in Spark, not wasb leftovers. Container + path layout belongs in git the same way S3 prefixes do.

Azure Databricks. Where serious Spark transformation happens on Azure. The first-party integration (identity, networking, billing) is genuinely smoother than on other clouds. Delta + Unity Catalog is the same lakehouse story as elsewhere — see Databricks + Delta and Delta vs Iceberg.

Synapse Analytics. The former flagship warehouse: dedicated SQL pools, serverless SQL over ADLS, Spark pools, pipelines that look like ADF. It still runs plenty of production. Microsoft’s investment and marketing have clearly shifted to Fabric. Think twice before starting new projects on dedicated SQL pools.

Power BI. The serve layer most Azure shops already paid for. Import vs DirectQuery vs (later) DirectLake is the performance and freshness trade-off. Do not rebuild the warehouse to fix a bad import model.

What Microsoft Fabric actually is

Fabric bundles the analytics stack — data movement (a Data Factory experience), lakehouse (OneLake), warehouse, Spark notebooks, real-time analytics, and Power BI — into one SaaS product with one capacity-based bill.

The key architectural idea is OneLake: a single logical lake (Delta format) that every Fabric engine reads directly. Warehouse tables, lakehouse files, and Power BI datasets all point at the same Delta tables instead of copying data between services. “One copy of data” is the pitch, and it mostly delivers — when you stay inside Fabric and do not also keep a shadow copy in a Synapse dedicated pool “just in case.”

What you are buying operationally:

Capacity (F SKUs / trials / reserved). Spark, warehouse, and BI share a pool. A runaway notebook and a Monday dashboard stampede compete. Someone must own throttling and workspace allocation.
Workspaces as the boundary. Permissions, deployment pipelines, and “who can create a warehouse.” Treat them like environments, not like personal folders.
SaaS networking and identity. Easier than stitching VNets for every service; different than “our Synapse is in this subnet.” Confirm data-residency and private-link needs before you promise Fabric to a regulated domain.

Fabric Data Factory is not a bit-for-bit clone of every ADF quirk. Plan a connector audit. Self-hosted IR patterns may still keep you on ADF for on-prem sources even if gold lives in OneLake.

Synapse is not dead — it is not the greenfield default

Dedicated SQL pools still query. Serverless SQL over ADLS still saves people who never wanted a pool. Spark pools still run.

What changed is the default for new work:

Situation	2022 default	2026 default to evaluate
New analytics platform, Power BI-heavy	Synapse + ADF + PBI	Fabric
Heavy Spark, multi-cloud, strong DE guild	Databricks + ADF + ADLS	Still Databricks + ADF/ADLS (Fabric optional for BI)
Existing dedicated pools, stable	Keep Synapse	Keep; next platform = Fabric or Databricks
Serverless SQL over the lake only	Synapse serverless	Fabric warehouse / lakehouse SQL or stay

A migration is a project: dialect (T-SQL pool quirks), security (logins vs Entra vs workspace roles), pipelines (Synapse pipelines vs ADF vs Fabric pipelines), and cost (DWU vs F capacity). There is no “upgrade” button that preserves all three.

Do not panic-migrate. Synapse works. Plan the next iteration around Fabric or Databricks rather than rewriting this quarter because a keynote said SaaS.

Choosing in practice

Power BI-centric org, want simplicity → Fabric. DirectLake (BI on lake files without a full import) is the feature that changes freshness conversations. If the org already lives in Microsoft 365 identity and Power BI Premium habits, Fabric is the path of least resistance.

Engineering-heavy team, multi-cloud ambitions, big Spark workloads → Azure Databricks + ADF + ADLS. More control, more portability, more established operational practices (job clusters, Unity, repo-based jobs). Fabric can still sit on the side as a consume layer if you shortcut OneLake / mirroring — only with a written writer rule.

Existing Synapse estate → stay, then iterate. Lift a new domain to Fabric or Databricks. Do not dual-run gold orders in a dedicated pool and a Fabric warehouse without an owner for drift.

On-prem SQL Server gravity → ADF (self-hosted IR) first. The lake and the engine come after the copy works on a schedule.

If Power BI is the product     → Fabric-first
If Spark is the product        → Databricks-first
If copy-from-everywhere is the product → ADF-first, engine second
If Synapse dedicated is stable → do not rewrite for sport

Head-to-head

Dimension	ADF + ADLS + Databricks	Synapse (classic)	Fabric
Ingest	ADF (best connectors / SHIR)	Pipelines (ADF-like)	Fabric pipelines / ADF remaining
Lake	ADLS (you own)	ADLS	OneLake (Delta), shortcuts to ADLS
Transform	Databricks Spark	Spark pools / SQL pools	Spark notebooks / warehouse
Warehouse SQL	Databricks SQL or Synapse	Dedicated / serverless	Fabric warehouse
BI	PBI on SQL / DirectQuery	PBI	PBI + DirectLake
Bill	Azure meters + DBUs	DWU + storage + Spark	Capacity (plus Azure if hybrid)
Ops maturity	High (if you staff it)	High (existing estates)	Rising; SaaS constraints
Portability	Spark/Delta travel	T-SQL pool lock-in	OneLake/Delta travel better than pools

Shortcuts (OneLake pointing at ADLS) are how you avoid a second physical copy. They are not “no IAM.” The identity that reads the shortcut still needs the storage grant.

A minimal ADF + Databricks pattern

The most common healthy Azure architecture outside Fabric-only shops:

ADF copies source data (SaaS APIs, SQL Server, files) into ADLS bronze
Databricks jobs build silver/gold Delta tables (medallion)
Power BI reads gold via Databricks SQL warehouse or, if you added Fabric, DirectLake on a shortcut
ADF or Databricks Workflows orchestrates the schedule — pick one owner

# Databricks: silver MERGE, same as any other cloud
from delta.tables import DeltaTable

silver = DeltaTable.forName(spark, "silver.orders")
(
    silver.alias("t")
    .merge(updates.alias("s"), "t.order_id = s.order_id")
    .whenMatchedUpdateAll()
    .whenNotMatchedInsertAll()
    .execute()
)

ADF notes that save weekends:

Parameterize utcNow() and directory dates. Hard-coded folders are how catch-up fails.
Forgive-if-not-exists on optional files; do not fail the whole factory on a missing optional extract.
Integration runtime size ≠ transform engine. Copy is I/O. Mapping Data Flows billed as compute is where ADF invoices surprise people who thought “it’s just copy.”
Source datasets in git (ARM/Bicep or ADF YAML). A factory that exists only in the portal is not a platform.

A minimal Fabric-first pattern

Land in a lakehouse (or shortcut from ADLS bronze ADF still fills)
Notebooks / Dataflow Gen2 / warehouse procedures build silver/gold Delta
Semantic model + DirectLake for the dashboards that can use it; import for the ones that cannot
Deployment pipelines across workspaces (dev/test/prod), not “publish from Desktop to prod on Friday”

-- Fabric warehouse SQL: treat it as a warehouse, not a notebook
create table gold.orders as
select
  order_id,
  customer_id,
  amount,
  cast(ordered_at as date) as order_date
from silver.orders;

Exact T-SQL surface differs from Synapse dedicated pools. Port with tests, not with hope. Incremental patterns (MERGE, watermarks) still matter; SaaS does not make a full reload free — it just hides it in capacity.

Identity, network, and the hybrid reality

Azure shops fail here more than on Spark versions.

Entra ID groups → workspace roles / Databricks groups / ADF access. One mapping document.
Self-hosted IR jump boxes: patch them, do not give them Contributor on the subscription.
Private endpoints for ADLS and Databricks. Fabric private networking is a checklist item, not an assumption.
Key Vault for connection strings. Web activity + secret in the pipeline JSON is how you get a finding.

Synapse workspaces, Databricks workspaces, and Fabric tenants can all exist at once. That is normal during a decade-long estate. It is not a target architecture.

Cost (capacity vs meters)

Fabric: a capacity you sized for “the workspace.” Spark and BI share it. Autoscale / bursting behavior is something you read in the current docs and then watch with actual usage — not a promise from a launch video.

Azure Databricks: DBUs + VMs, same as other clouds. Job clusters should terminate.

ADF: orchestration is cheap-ish; Data Flows and SHIR VMs are not always. Copy of 40 million tiny files is a meter you will meet.

Synapse dedicated: DWU you forgot to pause on nights and weekends is the classic invoice.

Do not compare “Fabric F8” to “Databricks job cluster” using a blog’s guessed hourly rate. Compare your month: capacity utilization, DBU hours, ADF pipeline runs, ADLS storage + transactions.

flowchart TD
  adf[ADF Copy] --> adls[ADLS bronze]
  adf --> meter[Copy and SHIR meters]
  nb[Fabric notebook] --> fsku[F capacity pool]
  pbi[DirectLake / refresh] --> fsku
  fsku --> wait[Queue when CU is gone]

ADF copy does not spend Fabric capacity. A Monday semantic-model refresh and a Spark notebook do. They queue each other.

# ADF: catch-up is run_date, not "whatever landed in Incoming/"
parameters:
  run_date:
    type: string
activities:
  - name: CopySqlOrders
    type: Copy
    typeProperties:
      source:
        type: AzureSqlSource
        sqlReaderQuery: >
          select order_id, amount, updated_at
          from sales.orders
          where cast(updated_at as date) = '@{pipeline().parameters.run_date}'
      sink:
        type: ParquetSink
    outputs:
      - referenceName: adls_bronze_orders
        type: DatasetReference
        parameters:
          dt: "@pipeline().parameters.run_date"

Copy is an Azure meter. It will not throttle the Fabric capacity your dashboards share. A full-history Fabric notebook on Monday morning will.

The certification angle

For job seekers: the DP-700 (Fabric Data Engineer) has replaced the retired DP-203 as Microsoft’s data engineering certification, which tells you where Microsoft thinks the jobs are going. A cert is not an architecture. If the estate is Databricks, study the lakehouse; if it is Fabric, study OneLake and capacities; if it is still Synapse dedicated, that knowledge still pays incident time.

Pitfalls

New dedicated SQL pool in 2026 without a written reason Fabric or Databricks cannot serve.
Panic rewrite of Synapse in a quarter. You will drop security and incremental loads.
Two golds (Synapse pool + Fabric warehouse) for the same grain.
Mapping Data Flows for heavy transforms when Databricks or Fabric Spark was the right engine — then blaming Azure.
DirectLake on a badly laid-out Delta table (tiny files, no clustering). The feature is not a magic index.
OneLake shortcuts with no owner for the source path IAM.
Personal Fabric workspaces as the production lake.
ADF + Synapse pipelines + Databricks Workflows + Fabric pipelines for one domain. One schedule owner.
Ignoring SHIR until the first on-prem copy runs at 4 a.m. on the analyst’s laptop gateway.

Decision rules

Power BI is the center of gravity, new platform, Microsoft identity already → Fabric first. Keep ADF if connectors / SHIR demand it.
Spark-heavy, multi-cloud, or an existing Databricks practice → Databricks + ADF + ADLS. Add Fabric only as a consume layer with a writer rule.
Synapse dedicated, stable, no new domains → keep. Next greenfield domain goes to Fabric or Databricks.
On-prem sources → ADF SHIR first, lake second.
Need a warehouse and a lake without Databricks staff → Fabric lakehouse + warehouse, not a new Synapse pool.
Need portable Spark and a non-Microsoft cloud later → Databricks/Delta (or Iceberg) on ADLS, not a pool-only design.

If Fabric capacity and Databricks are both “maybe,” pick the team you can staff. An empty Fabric workspace and an empty Databricks workspace are the same failure.

FAQ

Is Synapse gone? No. Dedicated pools and serverless SQL still run. What changed is the default for new work: evaluate Fabric or Databricks first. Do not panic-migrate a stable estate because a keynote moved.

Does Fabric replace ADF? For many SaaS and Azure sources, Fabric pipelines are enough. Self-hosted IR and a connector ADF already has often keep copy on ADF even when gold lives in OneLake. Audit connectors before you delete the factory.

Why did Power BI queue when a notebook was running? They share F capacity. A runaway Spark job and a Monday refresh compete. Split workspaces or throttle the notebook; buying another F SKU without an owner just moves the queue.

Can I keep Synapse gold and add a Fabric warehouse for the same grain? Only with a written writer. Two golds drift. Lift a new domain, or pick one table as the source of truth.

ADF Mapping Data Flows or Databricks for transforms? Copy in ADF. Heavy transforms in Databricks or Fabric Spark. Mapping Data Flows billed as compute is how “it’s just copy” becomes an invoice.

Do OneLake shortcuts skip IAM? No. The identity that reads the shortcut still needs the storage grant. A shortcut is a pointer, not a permission.

What this means for data engineers

ADF still moves data. ADLS or OneLake still stores it. The engine is Fabric when the org wants one SaaS analytics surface, and Databricks when Spark and portability are the job. Synapse remains a production estate, not a greenfield slogan.

Write the medallion paths, the writer, and the capacity or DBU owner in the same repo as the pipelines. Azure’s product names will move again. The grain will not.