In one sentence
A method where an AI system first looks up relevant documents and then writes its answer using them.
Retrieval-augmented generation, or RAG, combines a search step with a language model. When a question comes in, the system retrieves the most relevant passages from a set of documents, such as policies, manuals, past tickets or contracts, and gives them to the model along with the question. The model writes its answer from those passages. The term comes from a 2020 research paper by Lewis and colleagues at Facebook AI Research.
Why use it
Models do not know your internal documents, and their training data has a cutoff. RAG lets a model answer from your current material without retraining it. It also makes answers checkable, because the system can show which passages it used.
How it usually works
Documents are split into chunks. Each chunk is indexed, often as a vector embedding for meaning-based search, sometimes combined with keyword search. At question time the best matches are retrieved, ranked and placed in the prompt.
Where it goes wrong:
Bad retrieval
If the right passage is not retrieved, the model answers from the wrong one or guesses. Most RAG quality problems are search problems.
Stale or conflicting documents
Old versions of a policy sit next to new ones. Clean the source before indexing it.
Access control
A user should only get answers from documents they are allowed to see. Filter by permission at retrieval time.
Unsupported answers
Ask the model to cite passages and to say it does not know when nothing relevant is found.
When not to use it
If the answer lives in a structured system, such as a CRM field or an order status, query that system directly through an API or tool call. RAG is for unstructured text.
In our builds
we test retrieval on real questions before tuning prompts, and we log the passages behind every answer so mistakes can be traced.
Keep following the system
AI Agent
A system that uses a model and tools to pursue a goal across multiple steps within defined permissions and review boundaries.
Tool Calling (Function Calling)
A model feature where the AI returns a structured request to run a defined function, which your code runs and returns.
Model Context Protocol (MCP)
An open standard for connecting compatible AI applications to tools, resources, and prompts.
Semantic Code Search
Searching a codebase by meaning rather than exact text, using embeddings, often combined with keyword search, so a question like "where do we validate refunds" finds the right functions.
Speed to Lead
The time between a new lead reaching a business and the business's first real response to that lead.
Prior Authorization Automation
Software that gathers, submits and tracks prior authorization requests between providers and health plans with less manual work.
Business Associate Agreement (BAA)
A HIPAA contract in which a vendor handling protected health information for a covered entity agrees to protect it.