What is RAG engineering and how is it different from ML engineering?

Retrieval-augmented generation, introduced by Lewis et al. in 2020, combines two pieces. A retriever finds the most relevant documents for a query, typically via embedding similarity search. A generator (an LLM) reads the retrieved context and produces an answer. RAG engineers own the system that connects them and own the quality metrics that measure whether retrieval surfaced the right material.

The work is closer to search engineering than to ML training. Few RAG engineers train their own models; most use hosted LLMs and concentrate on the indexing pipeline. The hard problems are chunking strategy, embedding model selection, query rewriting, reranking, freshness, deduplication, and citation. Every one of these has multiple credible techniques and the right answer depends on the corpus.

Evaluation is its own subdiscipline. RAG systems get evaluated in two layers: retrieval quality (recall at k, mean reciprocal rank, hit rate on a held-out question set) and end-to-end answer quality (factuality, citation accuracy, hallucination rate). A RAG engineer who cannot describe both layers from memory will not pass an interview.

Production RAG systems run on vector databases (Pinecone, Weaviate, Chroma, pgvector), hybrid search engines (Elasticsearch, OpenSearch), or reranking layers (Cohere Rerank, custom cross-encoders). Most teams use a hybrid of dense vector search plus a keyword fallback because dense retrieval misses rare terms.

The role pays at AI engineering parity. RAG specialists who can ship a production-quality system that includes evaluation, freshness, and citation are in active demand at every enterprise rolling out an AI assistant.

Cybersecurity convergence shows up around document access control. Retrieved chunks must respect the permissions of the user who asked, not the permissions of whatever service indexed the corpus. AI security engineering has named this risk explicitly.