All stacks · AI & MLOps
LLMs, RAG pipelines, vector search, and AI on your data.
AI & GenAIBeyond chatbots: how data engineers use LLMs for entity extraction, data classification, documentation generation, and pipeline triage — with cost controls.
AI & GenAIModel Context Protocol as a tool bus for warehouse and pipeline context — read-only roles, no DROP, and prompt injection that hides in table comments.
AI & GenAIAn honest comparison of vector storage options for AI workloads — and why the right answer is often the database you already run.
AI & GenAIRetrieval-augmented generation is an ETL problem wearing an AI costume. How to build a production RAG ingestion pipeline with the skills you already have.
AI & GenAITreating an LLM call as a transformation step in a batch pipeline. Content-hash idempotency, per-row cost budgets, retry semantics for a non-deterministic API, and how to test output you cannot diff.
AI & GenAINative vector search in Snowflake, BigQuery, and pgvector has quietly become good enough for most RAG workloads. Here are the thresholds, the real costs, and when a dedicated store still wins.
AI & GenAIThe failure modes that only show up after launch — join paths, metric drift, runaway cost — and the semantic layer, constrained generation, and eval harness that make text-to-SQL survivable.
AI & GenAIHow Cortex Analyst semantic models and Cortex Search hybrid retrieval fit together, what the YAML actually controls, and where the accuracy comes from.
AI & GenAIWhy raw text-to-SQL fails on real schemas, when retrieval over curated marts wins, and the semantic model and guardrails that make either safe.
The AI wave landed squarely on data teams: RAG pipelines are data pipelines, vector search is an indexing problem, and every "chat with your data" feature is only as good as the modeling and governance underneath it. Data engineers are becoming the people who make AI features trustworthy.
These articles stay concrete — building RAG over warehouse and document data, running LLM functions inside Snowflake Cortex and friends, embedding pipelines that stay fresh, evaluation that goes beyond vibes, and the security questions agents and MCP servers raise when they touch production data.
Not classical ML depth — you need to understand embeddings, retrieval, prompting, and evaluation. The hard parts of production AI are data engineering parts: pipelines, freshness, access control, and measurement. That is home turf.
Retrieval-Augmented Generation: fetch relevant documents or rows first, then have the model answer using that context. It grounds answers in your actual data, updates instantly when data changes, and keeps sensitive data governable — none of which fine-tuning gives you cheaply.
If your platform already runs Snowflake, Databricks, or Postgres, start with their native vector support — one less system, and governance comes free. Dedicated vector databases earn their keep at high QPS, low latency, or advanced filtering scale.
Build a golden set of questions with known answers, then score retrieval (did the right chunks come back?) separately from generation (was the answer faithful?). Automate it in CI like any other data test — retrieval quality regresses silently when data or chunking changes.
Only with the same discipline you apply to humans: least-privilege roles, read-only by default, row/column policies, query tagging, and audit. Treat prompt injection as a given and design so the blast radius of a bad query is boring.
One email a week with our latest tutorials. No spam.
Newsletter signup is not live yet. Use the contact form if you want to be notified.