Retrieval-Augmented Generation (RAG)

The pipeline has two phases. Offline, documents are chunked, run through an embedding model, and indexed in a vector store. At query time, the question is embedded, the nearest chunks are retrieved — often refined by a reranker — and the model answers with those passages in context. The model's knowledge is thus extended without any retraining.

RAG earns its ubiquity on three properties: answers stay current because the index updates independently of the model, responses can cite sources, and grounding measurably reduces hallucination on factual queries. It is also the standard answer to private data — the model sees only what retrieval surfaces per query.

Economically, RAG trades a small fixed cost (embedding and storage) for a per-request input-token overhead of the retrieved context. It is far cheaper than fine-tuning for injecting knowledge and far cheaper than stuffing whole corpora into long context windows, though retrieval quality becomes the ceiling on answer quality — a model cannot correctly use what retrieval failed to find.