AI / RAG / MLOps

Solution

Ingest, chunk, embed, version, ACL-filter, then retrieve. Failures are usually stale chunks, missing metadata, and documents the model should never see — not the model brand. Treat the index like a serving table with a freshness SLO.

Solution

Reliable feature tables, training-data DAGs, point-in-time joins, and serving logs. Model architecture and eval design belong with ML. Interviewers want the boundary, not a PyTorch lecture. If you cannot explain training/serving skew, you are not done.

Solution

Embeddings from model A and model B are not comparable. A silent model upgrade without a backfill mixes spaces and tanks retrieval. Version the model, the chunker, and the source document hash. This is data versioning, not ML trivia.

Solution

The index will retrieve a document the caller cannot see. Filter on ACL metadata at query time (or partition indexes by tenant). “We will prompt the model not to reveal it” is not a control. This question shows up at every bank and healthcare loop.

Solution

Classify, extract structured fields, summarize into a stored column, route (ticket → queue). Always: schema, retries, cost cap, and a deterministic fallback. Never call a chat API from a BI dashboard on every refresh.

Solution

Point-in-time correct joins (no leakage), online/offline parity, and a serving path with low latency. A gold table is a start. If training used yesterday’s snapshot and serving uses a live join that can see the label, you shipped leakage.

Solution

A labeled retrieval set, freshness of chunks, and logs of what was retrieved. “The model felt better” is not an eval.

Solution

Redact or tokenize before the call, keep a deny-list, and do not paste raw rows into a chat UI. Contracts and DLP, not hope.

Solution

Same definition, two stores, a point-in-time story. If online computes a different join, you shipped skew.

Solution

When you need a specific model, tight latency, or tool-use the warehouse cannot host. Cortex/BigQuery ML are ELT steps, not every product.

Solution

A scheduled embed/refresh job with a watermark on document updated_at, plus an SLA dashboard on index lag. The model vendor is not on-call for a stale index. Treat it like CDC.

Solution

An eval set: retrieval hit-rate at k, and answer faithfulness. Too small loses context; too large dilutes the embedding. Chunking is a modeling choice with a metric, not a default in a tutorial.

Solution

You embedded without tenant_id in the filter/metadata, or the query path does not apply the ACL. Metadata filter at retrieval is mandatory. “The prompt said not to” is not an ACL.

Solution

Training-serving skew: missing features defaulted, or a different join grain online. Diff a production request against the training row for that entity_id and time. Then freeze a schema contract.

Solution

The job pins a prompt table/version at runtime and nobody bumped it, or COMPLETE inlines a string in SQL. Treat prompts like dbt models: version column, PR, and an eval before promoting.

Solution

One model, one batch job, one gold table with a documented as-of join. A store pays off with many models, online serving, and reuse. “We bought Feast” is not a design.

Solution

A labeled holdout that was not used to tune, a threshold policy, and a timestamp. Warehouse UDF metrics are fine; leaking the holdout through a JOIN is not. Say the positive class rate.

Solution

Warehouse when retrieval is batch/ELT and ACL already lives in SQL. Dedicated DB when you need low-latency ANN and the app is the product. Hybrid: warehouse is source of truth, index is a derived store with a rebuild job.