RAG / Retrieval-augmented generation

RAG explained: why you need one and how to build it.

2 min 35 sec Published July 12, 2026 Beginner friendly

Your AI can sound certain while inventing facts about your own files. RAG changes the order of operations: retrieve the relevant source first, then answer from it—with the receipt attached.

Watch on YouTube

What is RAG?

Retrieval-augmented generation is a way to make an AI answer from a specific collection of information—your handbook, contracts, support docs, research notes, or product catalog—instead of relying only on what the model absorbed during training.

The key word is retrieval. Before the model writes an answer, the system searches your knowledge base for the passages most relevant to the question. Those passages are placed into the model’s context, and the model generates an answer grounded in that evidence.

RAG in plain English

It is the automated version of pasting a document into Claude and saying, “Answer only from this file, and show me where you found it.”

The RAG workflow

PrepareYour documents are split into useful chunks and converted into searchable representations.
RetrieveA question triggers a search for the chunks whose meaning most closely matches the request.
GenerateThe model receives those passages, answers from them, and can cite the exact source used.

Traditional keyword search looks for matching words. A typical RAG system also uses embeddings—numeric representations of meaning—so a question about “time off after having a child” can still retrieve a section titled “parental leave.” The retrieved evidence matters more than the jargon: the model gets the right text before it responds.

Three reasons RAG is useful

It reduces confident inventions. If the answer must come from retrieved evidence, the system has less room to manufacture a plausible policy or number. A well-designed system should also admit when the source set does not contain an answer.

It can use private and current information. A model’s training data will not contain the contract signed this morning or the wiki page edited ten minutes ago. A RAG index can be refreshed when those files change.

It makes verification practical. The answer can include the document, page, or passage it used. That turns “trust the chatbot” into “check the source in ten seconds.”

Illustration of AI confidently inventing an answer — Stop the confident lie

Diagram of the traditional multi-step RAG engineering workflow — The old engineering stack

Claude Code prompt used to build a RAG pipeline — Build the pipeline

Grounded AI response with a source citation — The answer with a receipt

A practical Claude Code starting prompt

Open Claude Code in a folder containing the documents you want to search. Then give it a concrete implementation brief:

Build a RAG over the files in this folder. Chunk the documents, create embeddings, store the vectors locally, retrieve the top 5 relevant passages for each question, and answer only from those passages with file and page citations. If the retrieved text does not support an answer, say that the answer was not found in the source documents. Include setup instructions and a small evaluation set.

This is a starting specification, not a magic production switch. Review the generated code, choose an embedding model and storage approach appropriate for the sensitivity and size of your data, and test the system with questions whose answers you already know.

Privacy reality check

A local vector store keeps the index on your machine, but your document text may still be sent to an embedding or language-model provider. Check the generated architecture, provider terms, and data-retention settings before indexing sensitive files.

When RAG is—and is not—the right tool

Good fits

Internal policy, handbook, and operations assistants
Support bots grounded in product documentation
Contract, case, or research collections that change over time
Any workflow where a citation is more valuable than a fluent guess

Think twice when

The source set is tiny enough to place directly in the prompt
You need exact database calculations rather than document retrieval
The answer requires actions or live system state—tools or APIs may be the better connection
You cannot define how retrieval quality and answer faithfulness will be evaluated

A five-point quality check

Can it retrieve the correct passage for differently worded versions of the same question?
Does every factual answer point to a real source?
Does it refuse cleanly when the answer is absent?
Does changing a source document update the next answer?
Are access permissions enforced before retrieval, not after generation?

Video chapters

00:00The confident lie 00:41Why RAG stops it 00:59Your private, current data 01:17Answers with citations 01:34The old way to build 01:53Now it is one prompt 02:11The grounded result 02:29Try it on your docs

RAG FAQ

Does RAG completely eliminate hallucinations?

No. It reduces a major source of hallucination by grounding answers in retrieved evidence, but retrieval can return the wrong passage and a model can still misread evidence. Evaluation, citations, refusal behavior, and permission controls remain important.

Is RAG the same as fine-tuning?

No. RAG supplies relevant information at answer time. Fine-tuning changes a model’s behavior or style through additional training. For frequently changing factual knowledge, retrieval is usually easier to update because you can re-index the source documents.

Do I need a vector database?

Not always. Small collections can use simpler full-text or in-memory search. A vector store becomes useful when you need semantic retrieval across a larger collection, but hybrid systems combining semantic and keyword search often perform better.

Can RAG work with PDFs?

Yes, but extraction quality matters. Scanned PDFs may require OCR, tables need careful parsing, and useful citations depend on retaining file and page metadata throughout the indexing pipeline.

More practical AI guides, without the hour-long lecture.

Subscribe on YouTube