How RAG works
Retrieval-augmented generation answers a question in two steps. First, a retrieval step finds the passages in your own documents that are most relevant to the question. Second, a language model writes an answer using only those passages as its source. The model supplies the language; your documents supply the facts.
This is why RAG is preferred over fine-tuning for most company knowledge. Documents change, and updating an index is far easier than retraining a model.
Building the pipeline
The main stages are ingestion, chunking, embedding, indexing, retrieval, generation and citation. Chunk size is the choice with the largest effect: chunks that are too small lose context, and chunks that are too large dilute the match and waste the model's input space.
Most production systems combine semantic retrieval with keyword retrieval so that exact identifiers such as product codes and names are still found. Showing the source passage beside each answer lets users check it.
Evaluating it honestly
Do not judge a RAG system on a handful of demo questions. Collect real questions from support logs or staff, write the correct answer and source for each, and measure how often retrieval finds the right passage and how often the final answer is correct.
Track two failures separately: retrieval misses, where the right passage was never found, and generation errors, where the right passage was found but the answer was still wrong. They have different fixes.
Permissions and freshness
An enterprise system must respect who may see what. A retrieval index that ignores document permissions will leak content, so access rules need to be applied at query time, not only at ingestion.
Set a refresh schedule for each source and show the last-updated date. An answer drawn from a policy that changed last month is worse than no answer.
What you get:
- How a RAG pipeline is put together, stage by stage
- Chunking, embedding and indexing choices that matter
- How to build an evaluation set from real user questions
- Permissions, freshness and citation requirements
- Common causes of wrong or empty answers