Azure Data Engineer Interview Questions cheat sheet
ADLS, Data Factory, Synapse, Fabric, and Event Hubs questions with the service trade-offs Azure data interviews probe.
Storage and the lake
What does ADLS Gen2 add over plain Blob Storage?- A hierarchical namespace, so directories are real objects and a rename or delete of a folder is a single atomic metadata operation instead of a per-object loop. That plus POSIX-style ACLs and the abfss driver is what makes it suitable for analytics; flat blob storage makes Spark commit protocols slow and directory ACLs impossible.
How do you control access to a lake in ADLS Gen2?- Azure RBAC at the container or account level for coarse grants, POSIX ACLs on directories and files for fine-grained control, and Entra ID (formerly Azure AD) identities rather than account keys. ACLs cap at 32 entries, so assign groups and never individual users, and remember RBAC is evaluated first and can grant access ACLs cannot revoke.
Compare access keys, SAS tokens, and managed identities.- Account keys are full-control and unrotatable in practice, so treat them as a last resort. SAS tokens are scoped and time-limited, with user delegation SAS backed by an Entra identity being the safer variant. Managed identities are the right default for Azure services, because there is no secret to store, leak, or rotate.
What are storage tiers and lifecycle management for?- Hot, Cool, Cold, and Archive tiers trade storage price against access price and retrieval latency, with minimum retention of 30 days for Cool, 90 for Cold, and 180 for Archive, and Archive requiring a rehydration of hours. Lifecycle rules move or delete blobs automatically by age or last access, which is how you stop a lake growing forever.
How do you make a lake resilient to accidental deletion?- Enable soft delete for blobs and containers, versioning where change frequency allows, and an immutability policy for regulated data. For region loss, geo-redundant storage replicates asynchronously, but read access needs RA-GRS and there is an RPO, so the honest answer includes measuring how much recent data a failover would lose.
Data Factory and orchestration
What is an integration runtime and which type do you pick?- The compute that executes an activity: Azure IR is managed and used for cloud-to-cloud movement and data flows; self-hosted IR runs on your machine or VM to reach on-premises or private network sources; Azure-SSIS IR runs lifted SSIS packages. The self-hosted IR is the answer whenever a source sits behind a firewall, and it needs high availability nodes in production.
Compare the Copy activity, Mapping Data Flows, and calling Databricks.- Copy activity is optimized bulk movement with no transformation beyond simple mapping. Mapping Data Flows give a visual transformation surface that compiles to Spark on a managed cluster, with cluster startup costing several minutes unless you keep a time-to-live. Calling Databricks or Synapse Spark gives full control and testable code, which most engineering teams prefer for anything complex.
How do you parameterize a pipeline so one pipeline serves many tables?- Use a metadata-driven pattern: a control table listing source, target, watermark column, and load type, read with a Lookup activity, iterated with ForEach, and passed into a parameterized dataset and Copy activity. This is the design question interviewers most often ask, because the alternative is 200 near-identical pipelines nobody can change.
How do triggers differ in Data Factory?- Schedule triggers fire on a wall-clock recurrence, tumbling window triggers cover contiguous non-overlapping time slices with dependency and backfill support, and event triggers fire on blob creation or deletion through Event Grid. Tumbling window is the right choice for partitioned incremental loads because it retries and backfills a specific window.
How do you implement incremental loading in ADF?- Store a high-watermark per table, read it in a Lookup, use it in a parameterized source query, and write it back only after the copy succeeds — inside the same pipeline so a failure does not advance the watermark. For sources with reliable change tracking, use SQL Server Change Tracking or CDC instead, which also captures deletes.
How do you manage ADF across environments?- Keep the factory in Git, develop in a collaboration branch, publish to generate ARM templates, and deploy those to test and production with parameter overrides for linked service endpoints. Never edit production in the portal, and store all secrets in Key Vault referenced by linked services so the templates carry no credentials.
Synapse, Fabric, and warehousing
What is the difference between a dedicated SQL pool and a serverless SQL pool?- A dedicated pool is a provisioned MPP warehouse billed by DWU whether you query or not, with distributed tables you must design. Serverless SQL is a query-on-demand engine over files in the lake billed per terabyte processed, with no storage of its own. Serverless suits exploration and lightweight serving; dedicated suits steady heavy BI.
Explain table distribution in a dedicated SQL pool.- Data is spread across 60 distributions. Hash distribution on a join key co-locates matching rows and avoids data movement; round robin spreads evenly and suits staging tables; replicated copies a small dimension to every compute node. Choosing a hash column with heavy skew or many nulls creates uneven distributions and one slow node.
What is Microsoft Fabric and how does it relate to Synapse?- Fabric is the unified SaaS analytics platform that supersedes standalone Synapse for new projects, with OneLake as a single logical Delta-based storage layer and workloads for Data Factory, Data Engineering, Data Warehouse, Real-Time Intelligence, and Power BI. Capacity is bought as F SKUs shared across all workloads rather than provisioned per service.
What is OneLake shortcuts and why does it matter?- A shortcut is a reference to data in another OneLake location, ADLS, or S3 that appears as if it were local, with no copy and no scheduled sync. It matters because the recurring cost of analytics platforms is duplicated copies drifting apart; shortcuts let a Power BI model and a Spark notebook read the same physical Delta files.
What is Direct Lake mode in Power BI?- It reads Delta-Parquet files in OneLake directly into the Power BI engine without importing a refresh copy or paying DirectQuery's per-query translation cost, giving import-like performance with near-live data. It falls back to DirectQuery when it hits unsupported features or capacity limits, so understanding the fallback triggers is the practical part.
Streaming and real time
What is Event Hubs and how does it compare to Kafka?- A managed partitioned event ingestion service with the same log-and-offset model as Kafka and a Kafka-protocol endpoint on Standard tier and above, so Kafka clients work unchanged. Throughput is bought as throughput units or processing units rather than sized by broker, and Capture writes raw events to a lake automatically.
Explain consumer groups and checkpointing in Event Hubs.- A consumer group is an independent view of the stream; each partition should have one active reader per group, and the recommended limit is five concurrent readers per partition. Checkpoints are stored by the client in a blob container, not by the service, so losing that container makes consumers restart from the retention window.
When would you use Stream Analytics over Spark Structured Streaming?- Stream Analytics is a managed SQL-based engine with built-in windowing (tumbling, hopping, sliding, session) and simple sink wiring, good for straightforward filtering, aggregation, and routing with minimal operations. Choose Spark or Flink when you need custom code, complex state, machine learning inference, or portability off Azure.
What are the windowing options in a streaming query?- Tumbling windows are fixed and non-overlapping; hopping windows are fixed length advancing by a smaller hop, so they overlap; sliding windows emit on every event within the interval; session windows group activity separated by a gap. The interview follow-up is late arrival: you need a watermark or late-arrival tolerance or results are silently incomplete.
How do you handle late and out-of-order events?- Configure the tolerance explicitly — Stream Analytics has separate late-arrival and out-of-order policies that either adjust or drop events, and Spark uses withWatermark. Then decide the business rule: drop, count separately, or restate downstream. Any of those is defensible; not knowing which one your pipeline currently does is not.
Governance, security, and operations
What does Microsoft Purview provide?- Automated scanning and cataloging of data assets across Azure, on-premises, and other clouds, with classification, lineage from ADF and Synapse activities, and a glossary. It is discovery and governance, not enforcement — Purview labels sensitive data but access is still enforced by RBAC, ACLs, and database policies.
How do you keep data traffic off the public internet?- Private Endpoints put a service on a private IP in your virtual network, with private DNS resolving the public name to it; service endpoints are the older, coarser alternative that keeps traffic on the backbone but does not give a private IP. Managed virtual networks in Fabric and Synapse handle the compute side of the same problem.
How is Key Vault used in a data platform?- Store connection strings, keys, and certificates there and reference them from linked services, Databricks secret scopes, and Functions using managed identities, so no pipeline definition contains a secret. Enable soft delete and purge protection, and use rotation policies — a Key Vault whose secrets never rotate mostly relocates the problem.
How do you monitor Azure data pipelines?- Send diagnostic settings from ADF, Databricks, and storage to a Log Analytics workspace, then query with KQL and alert on failed activity runs, duration anomalies, and throttling. Azure Monitor alerts route to action groups. Add data quality assertions in the pipeline, because activity success does not mean the load was correct or complete.
How would you cost-optimize an Azure data platform?- Right-size Fabric or Synapse capacity and pause dedicated pools when idle, keep Data Flow cluster time-to-live short but non-zero to avoid repeated startup charges, prefer Copy activity over Data Flows for pure movement, apply storage lifecycle tiering, and use reserved capacity for steady workloads. Then tag everything so each team sees its own spend.
From DataLane — tutorials at/blog, practice SQL live in theplayground.