Every serious AI feature on a business site comes down to the same problem: the model knows a great deal about the world and nothing about you. Retrieval-augmented generation is the standard fix, and it is simpler than the name suggests.
What it is, in one paragraph
Before answering, the system searches your own content for the passages most relevant to the question, and hands those to the model along with an instruction to answer only from them. The model does the language; your data does the facts. When it works, answers cite a source you can click.
What it fixes
- Invention. A grounded system that cannot find an answer can be made to say so, which an ungrounded one will not.
- Freshness. Change a policy page and the answer changes, with no retraining.
- Attribution. Every answer can point at where it came from, which is what makes it defensible.
What it does not fix
If your content is wrong, contradictory or missing, retrieval finds the wrong thing faster. This is the part clients consistently underestimate: most of the work in these projects is content, not machine learning.
It also does not handle questions that require reasoning across many documents, or arithmetic over your live data. "How many orders shipped late last month" is a database query wearing a chat interface, and should be built as one.
The pieces involved
| Piece | What it does | Where it goes wrong |
|---|---|---|
| Chunking | Splits documents into passages | Chunks too large dilute the answer, too small lose context |
| Embeddings | Turns passages into vectors for similarity search | Cheap. Rarely the problem |
| Vector store | Holds and searches the vectors | Over-engineered early. Most catalogues fit in something simple |
| Retrieval | Picks passages for the question | The actual quality bottleneck, and the least glamorous part |
| Generation | Writes the answer from those passages | Needs a strict instruction to refuse when nothing matches |
The costs people miss
- 01Re-indexing. Content changes, and something has to notice and update. This is a pipeline, not a one-off script.
- 02Evaluation. Without a fixed set of questions and expected answers you cannot tell whether a change improved anything. Building that set is real work and it is the difference between engineering and guessing.
- 03Per-answer cost. Small individually, and worth instrumenting from the first week so it never arrives as a surprise.
- 04Content cleanup. Usually the largest line, and the one nobody budgets.
How to start without committing
Build a narrow version against real data and a fixed list of thirty questions you already know the answers to. That tells you within a week or two whether quality is achievable on your content, before anyone commits to a full build.
If it cannot answer thirty known questions reliably, more model will not save it. The problem is your content, and that is worth finding out cheaply.
Working on something like this?
We build websites, stores and custom applications, and we will tell you honestly if the thing you are describing does not need one.