Retrieval augmented generation: let the model look it up.
A model trained months ago can't know what changed since, and it has never seen the private documents sitting on your drive. RAG hands it the right pages before it answers, instead of asking it to recall everything from memory.
The knowledge base holds hundreds of documents. Most of them are not relevant to this question.
Search first, then answer
Retrieval augmented generation means searching a set of documents for the pieces most relevant to the question, then handing those pieces to the model along with the question itself. The model still writes the answer. It just gets to read the source material first.
It's the same reason you'd hand a coworker the right file before asking them about it, instead of asking them to remember the details from a meeting six months ago.
Closed book versus open book
Same question, two very different ways of answering it.
Answers from memory alone. Confident even when it's wrong, with no way to check where the answer came from. Can't know anything that happened after training, or anything that was never public in the first place.
Answers from memory plus the specific pages it was handed. You can check exactly which document it used. Knows this morning's numbers, if you gave it this morning's file.
Four steps before the model ever sees a question
Retrieval isn't magic. It's a small pipeline that runs once, ahead of time, on your documents, long before anyone types a question.
Clean the data
Strip duplicate pages, boilerplate headers, and outdated versions. Whatever is messy going in stays messy in every chunk that comes out.
Break it into chunks
Split into pieces small enough to retrieve individually, cut along natural breaks like paragraphs, not mid-sentence.
Embed each chunk
Turn each chunk into a vector: a list of numbers standing in for its meaning, not just the words it uses.
Store it in a vector database
An index built to search by similarity, so a question can find chunks with a related meaning even without sharing a single word.
A question gets embedded the same way at answer time, then the database returns the chunks whose vectors sit closest to it. That's the search step in the diagram above, running against an index this pipeline already built.
Retrieval only helps if it finds the right pages
Three ordinary ways this breaks, none of them exotic.
Bad chunking
A document gets cut at an odd point, splitting a table or a sentence in half. Neither half makes sense alone.
Weak ranking
The search returns pages that mention the right keyword but miss the actual answer, and the model works with what it's given.
Stale duplicates
Last year's policy file sits right next to this year's, and nothing tells the model which one is current.
"The model is still writing the answer. Retrieval only controls what it's allowed to write from."
Retrieval quality sets a ceiling on answer quality. Fix the search before blaming the model.
If you can't tell which document an answer came from, you can't trust it either.