Python Interview Questions cheat sheet
Data structures, generators, the GIL, decorators, typing, and the Python patterns data engineering interviews actually test.
Core objects and data structures
What is the difference between a list and a tuple beyond mutability?- Tuples are hashable when their contents are, so they work as dict keys and set members; lists do not. Tuples also have a smaller memory footprint and CPython caches small ones, but the real interview point is intent: a tuple signals a fixed-arity record, a list signals a homogeneous sequence you expect to grow.
How does a dict achieve average O(1) lookup, and when does it degrade?- It is an open-addressing hash table storing hash, key, and value; lookup hashes the key, probes, then confirms with equality. Degradation to O(n) requires pathological hash collisions, which PYTHONHASHSEED randomization makes hard to trigger for strings. Since 3.7 dicts also preserve insertion order as a language guarantee, not an implementation detail.
What is the difference between is and ==?- 'is' compares identity (same object in memory), '==' calls __eq__. They coincide for small integers between -5 and 256 and for interned strings because CPython caches those, which is exactly why beginners write 'is' and get away with it until a value falls outside that range. Use 'is' only for None, True, False, and sentinels.
Why is a mutable default argument dangerous?- Defaults are evaluated once at function definition, so 'def f(x, acc=[])' shares one list across every call and accumulates state. The fix is 'acc=None' with 'if acc is None: acc = []' inside. The follow-up is usually whether dataclasses have the same problem: they do, which is why mutable dataclass fields need field(default_factory=list).
Explain shallow copy versus deep copy.- copy.copy duplicates the outer container but shares references to nested objects, so mutating a nested list shows through both copies; copy.deepcopy recursively duplicates and tracks a memo dict to handle cycles. Deepcopy is expensive, so for config objects prefer immutable structures or an explicit constructor over deepcopy in a hot loop.
How do sets and frozensets help in data work?- Set membership is O(1) against O(n) for lists, which turns a nested-loop dedupe into a single pass. frozenset is hashable, so it can key a dict of column groupings. Watch memory: a Python set of ten million strings can exceed a gigabyte, so at that scale reach for a database, a Bloom filter, or a sorted merge instead.
Functions, scope, and decorators
What does a decorator do and how do you write one that takes arguments?- A decorator is a callable that takes a function and returns a replacement, applied with the @ syntax at definition time. Arguments require a third layer: a factory that captures the arguments and returns the actual decorator. Always apply functools.wraps to the inner function or you lose __name__, __doc__, and the signature that logging, docs, and pytest rely on.
Explain closures and the classic loop variable bug.- A closure captures variables by reference, not value, so functions built in a loop all see the final value of the loop variable. Fix it with a default argument that binds at definition time, or functools.partial. Interviewers use this to check that you understand late binding rather than just reciting the definition of a closure.
What is the LEGB rule?- Name resolution goes Local, Enclosing, Global, Built-in. 'global' rebinds a module-level name and 'nonlocal' rebinds one in the nearest enclosing function scope. The common gotcha is that assigning to a name anywhere in a function makes it local for the whole function, so reading it before assignment raises UnboundLocalError rather than falling through to the global.
What are *args and **kwargs, and what does the bare * in a signature mean?- They collect extra positional and keyword arguments into a tuple and a dict. A bare * forces every parameter after it to be keyword-only, and a / forces those before it to be positional-only. Keyword-only parameters are the practical way to add options to a widely used function without breaking callers who pass positionally.
When is functools.lru_cache the right tool, and what are its risks?- It memoizes pure functions with hashable arguments, defaulting to maxsize=128; functools.cache is the unbounded alias. Risks are unbounded memory growth, caching per process so it does nothing across worker restarts, and holding strong references to arguments — which leaks if you decorate a method and cache self.
Iterators, generators, and memory
What is the difference between an iterable, an iterator, and a generator?- An iterable implements __iter__ and can produce a fresh iterator; an iterator implements __next__ and is consumed once; a generator is an iterator built by a yield function or a generator expression. The practical consequence interviewers probe: iterating a generator twice yields nothing the second time, which silently produces empty results in pipeline code.
How would you process a 50 GB file on a 16 GB machine?- Stream it. Iterate the file object line by line, or read fixed-size chunks, and keep only aggregates in memory rather than the rows. For structured data use pyarrow or a chunked reader so you materialize one row group at a time, and push filters and projections down so you never decode columns you do not need.
What does the yield keyword actually do to a function?- It turns the function into a generator factory: calling it runs no body code and returns a generator object; each next() runs to the following yield and freezes the frame, preserving local state. yield from delegates to a sub-generator, and a generator can also receive values through send(), which is the basis of the older coroutine style.
Compare a list comprehension with a generator expression.- A list comprehension builds the whole list in memory immediately; a generator expression is lazy and holds one item at a time. Use the generator when feeding sum, any, max, or a writer. Use the list when you need to iterate more than once, index into it, or take its length — calling len on a generator is a TypeError.
What is itertools good for in a data pipeline?- islice for bounded reads and pagination, chain for concatenating sources without materializing, groupby for run-length grouping (which requires pre-sorted input, the classic trap), and batched in 3.12 and later for building fixed-size insert batches. All of them are C-implemented and constant memory.
Concurrency and parallelism
What is the GIL and how has it changed?- The Global Interpreter Lock lets only one thread execute Python bytecode at a time in a standard CPython build, so threads give concurrency for I/O but no CPU parallelism. Python 3.13 shipped an experimental free-threaded build under PEP 703 and 3.14 made it an officially supported build, though it is not the default and some C extensions still need work.
When do you choose threads, processes, or asyncio?- Threads for blocking I/O with a library that has no async version, since the GIL is released during I/O. Processes for CPU-bound work, accepting pickling overhead and no shared memory. asyncio for very high-concurrency network I/O where thousands of sockets would make one thread each too expensive. Mixing them means running blocking calls in a thread pool via asyncio.to_thread.
What breaks when you call a blocking function inside an async coroutine?- It blocks the whole event loop, so every other task stalls, and throughput collapses without any error message. That includes requests, most database drivers, and time.sleep. Wrap them with asyncio.to_thread or run_in_executor, or switch to an async-native client such as httpx or asyncpg.
How do you parallelize an embarrassingly parallel job correctly?- concurrent.futures.ProcessPoolExecutor with map over independent chunks, sized to os.cpu_count rather than an arbitrary number. Guard the entry point with an __name__ equals __main__ check because Windows and macOS spawn rather than fork and would re-import the module recursively. Return small results: sending large DataFrames back through pickle often costs more than the compute saved.
How do you make a retry safe under concurrency?- Make the operation idempotent first, usually with a deterministic key such as a run ID plus partition so a repeat overwrites rather than appends. Then add exponential backoff with jitter and a cap on attempts, and only retry on transient errors like timeouts and 429 or 5xx responses. Retrying a non-idempotent insert is how duplicates enter warehouses.
Errors, testing, and typing
Explain the difference between except Exception and a bare except.- A bare except also catches KeyboardInterrupt and SystemExit, which are BaseException subclasses, so it swallows Ctrl+C and orchestrator termination signals. Catch the narrowest exception you can handle, re-raise with 'raise' to preserve the traceback, and use 'raise NewError(...) from err' to keep the cause chain visible in logs.
What is a context manager and why write one?- An object with __enter__ and __exit__ that guarantees cleanup even on exception, which is why 'with' is the correct way to handle files, connections, locks, and temporary directories. contextlib.contextmanager turns a generator with a single yield inside a try/finally into one. Returning True from __exit__ suppresses the exception, which is almost always a bug.
How do you structure tests for a data pipeline?- Separate pure transformation logic from I/O so the transformations can be unit tested on small in-memory frames with no cluster or warehouse. Use pytest fixtures for setup, parametrize for edge cases such as nulls, empty input, and duplicate keys, and reserve a handful of slow integration tests against a container or a scratch schema for the wiring.
Do type hints affect runtime behavior?- No. CPython does not enforce them; you need mypy or pyright in CI to get any value. Python 3.14 made annotations lazily evaluated by default under PEP 649, which removed most need for 'from __future__ import annotations' and for quoting forward references. Pydantic is the common way to get actual runtime validation at pipeline boundaries.
How do you mock an external API in tests?- Patch at the point of use, not the point of definition, with unittest.mock.patch, or better, inject the client as a dependency so the test passes a fake. For HTTP, responses or respx record and match requests, which catches wrong URLs and payloads that a hand-rolled MagicMock would happily accept.
Practical data engineering in Python
How do you load a large CSV into a warehouse efficiently from Python?- Do not insert row by row. Stream to a compressed columnar file with pyarrow, stage it in object storage, and let the warehouse bulk-load it with COPY. If you must go through a driver, use the bulk path such as psycopg COPY or executemany with a batch of a few thousand rows, and size files to roughly 100-250 MB compressed.
What does pyarrow give you over plain pandas?- A columnar in-memory format with zero-copy sharing, proper nulls for every type including integers, and fast Parquet and Flight I/O. Since pandas 2.0 you can back a DataFrame with Arrow dtypes, which fixes the old object-dtype string performance and the NaN-versus-None ambiguity. Arrow is also what lets polars, DuckDB, and Spark exchange data without serialization.
How do you manage dependencies and Python versions on a data team in 2026?- Declare dependencies in pyproject.toml and commit a lock file so builds are reproducible; uv has largely replaced pip-tools and virtualenv for speed, while Poetry and pip with venv remain common. Pin the Python version in the image and lock file together, and never rely on the interpreter that ships with the OS.
What is the difference between multiprocessing fork and spawn, and why does it bite data code?- fork copies the parent process cheaply but is unsafe with threads and open connections, which is why forked workers inherit a broken database handle. spawn starts a clean interpreter and re-imports the module, so everything passed must be picklable. Python 3.14 made spawn the default on Linux, so code that relied on inherited globals now has to pass them explicitly.
How do you log properly in a pipeline?- Use the logging module with a configured handler at the entry point only, never logging.basicConfig inside a library, and emit structured JSON so the orchestrator can index fields. Include the run ID, task, and partition in every record, and use logger.exception inside an except block so the traceback is attached rather than a bare message.
From DataLane — tutorials at/blog, practice SQL live in theplayground.