AI, data and trust
How to Tell If Your RAG System Is Making Things Up
"The chatbot gave a wrong answer" is not one problem, it is at least four, and they need different fixes. The document was never retrieved, it was retrieved but ranked below noise, it was retrieved and the model ignored it, or irrelevant context derailed an answerable question. Published evaluation metrics separate these, so the question is measurable rather than a matter of impression. A team without those numbers is guessing which failure they have.
We do this for clients:RAG Systems Development
Why "it hallucinates sometimes" is not a diagnosis
Every retrieval system gets something wrong eventually, and the usual response is to blame the model and try a bigger one. That is expensive, slow, and frequently fixes nothing, because in a retrieval system the model is often not where the failure happened.
A user sees one symptom: a confident answer that is wrong. Underneath it are several distinct mechanisms, and telling them apart is the whole job. Swapping models when the real problem is that your chunking cut a table in half will not help, and you will have spent a month finding that out.
The four failures, and what each one actually is
These are the standard evaluation dimensions, as defined by the Ragas framework. The point is not the tool: it is that each dimension isolates a different stage of the pipeline.
| What went wrong | The measure | Where the fix is |
|---|---|---|
| The right passage was never retrieved | Context recall | Ingestion: chunking, coverage, the index |
| It was retrieved but buried below noise | Context precision | Ranking: embeddings, reranking, top-k |
| It was retrieved and the answer ignored it | Faithfulness | Generation: the prompt, the model |
| Irrelevant context derailed the answer | Noise sensitivity | Filtering, and knowing when to abstain |
Faithfulness, and what it does not tell you
Faithfulness "measures how factually consistent a response is with the retrieved context", scored from 0 to 1, where "a response is considered faithful if all its claims can be supported by the retrieved context". It is computed by breaking the answer into claims, checking each against what was retrieved, and dividing supported claims by total claims.
That is the closest thing to a direct measure of making things up, and it is worth understanding precisely, because it is narrower than it sounds.
Faithfulness measures whether the answer follows from what was retrieved. It does not measure whether the answer is true. A system can score a perfect 1.0 while being completely wrong, if retrieval confidently returned the wrong document and the model faithfully summarised it.
That is exactly why it has to be read against the retrieval measures rather than alone. High faithfulness with low context recall is a system that is reliably telling you what the wrong document said.
The two retrieval measures do different jobs
Context recall "measures how many of the relevant documents (or pieces of information) were successfully retrieved. It focuses on not missing important results." Low recall means the answer was never available. No prompt engineering fixes that, and no better model fixes it either. The problem is upstream in ingestion.
Context precision "evaluates the retriever's ability to rank relevant chunks higher than irrelevant ones", calculated as the mean of precision at each rank. Low precision with acceptable recall means the right passage was in there but buried, which is a ranking problem and often the cheapest of all these to fix.
Noise sensitivity "measures how often a system makes errors by providing incorrect responses when utilizing either relevant or irrelevant retrieved documents", scored 0 to 1 where lower is better. It catches the case where retrieval technically worked but the surrounding junk pulled the answer off course.
Building the evaluation set, which is the actual work
None of the above helps without a set of questions with known answers. This is unglamorous and it is the difference between managing a system and hoping about it.
- 1
Collect real questions from the people who will use it
Not questions you invented to demonstrate the system. The questions your team actually asks, in the words they actually use, including the badly phrased ones. Fifty real questions beat five hundred synthetic ones.
- 2
Write down the passage that should answer each
This is the reference the retrieval measures compare against, and it is the tedious part. It is also what converts "it feels worse since the reindex" into a number that either moved or did not.
- 3
Include questions the corpus cannot answer
The system should say it does not know. Most demos quietly omit this case, and it is the one that destroys trust fastest in production, because a confident answer to an unanswerable question teaches users they cannot rely on any answer.
- 4
Score before you change anything
A baseline is what makes the next change legible. Without it you are comparing a new impression against a remembered one.
- 5
Re-score on a schedule, not on suspicion
Corpora drift as documents are added and edited. A system that scored well at launch and was never measured again is the most common way a trusted tool quietly stops being trustworthy.
What to ask a vendor, including us
If someone is building a retrieval system for you, these questions separate the people who have done it from the people who have demoed it.
What is in the evaluation set, and who wrote the questions? If the answer is that the builder wrote them, the set is measuring whether the system does what the builder expected, not whether it answers your questions.
What does it do when retrieval comes back empty? The correct behaviour is to say so. A system that always produces an answer will always produce an answer, including when there is nothing to base one on.
Can I see the passage behind any given answer? If a claim cannot be traced back to its source in one click, nobody will check it, and unchecked answers are how a wrong one survives to become a decision.
What is the re-indexing story? A stale index is the most common reason a system that worked at launch is distrusted six months later.
What this does not cover
- These are framework definitions, not empirical claims about how often each failure occurs. How your system fails is a question about your corpus, which only your own evaluation set can answer.
- Ragas is used here because it publishes precise definitions and formulas. It is not the only framework and this is not a recommendation of a tool over its alternatives.
- No failure-rate percentages appear in this post. The commonly circulated ones could not be traced to any named study, so they are deliberately absent rather than accidentally omitted.
- Evaluation tells you where a system fails, not how to fix it. The fix is specific to the corpus, the chunking and the domain.
Sources
Every figure above comes from one of these, read on the date shown. Prices change: if you are reading this long after that date, check before you rely on it.
- Ragas documentation, Faithfulness
The definition of faithfulness and how the score is computed.
Read 11 September 2026
- Ragas documentation, Context Precision
The ranking measure and its precision-at-k calculation.
Read 11 September 2026
- Ragas documentation, Context Recall
The retrieval-coverage measure and its need for a reference.
Read 11 September 2026
- Ragas documentation, Noise Sensitivity
The measure of errors induced by relevant and irrelevant retrieved documents.
Read 11 September 2026
Questions
The questionspeople ask
How do I know if my chatbot is hallucinating?
Measure faithfulness, which scores whether each claim in the answer is supported by the passages that were retrieved. But read it alongside context recall, because a system can be perfectly faithful to the wrong document. High faithfulness plus low recall means it is reliably reporting the wrong source.
Is hallucination a model problem or a retrieval problem?
Usually retrieval, in a RAG system. If the passage that answers the question was never retrieved, no model can answer correctly and a larger one will simply be wrong more fluently. Measure recall first, because it tells you whether the answer was even available.
What is a good faithfulness score?
There is no universal threshold, and anyone quoting one without knowing your corpus is guessing. What matters is your own baseline and whether changes move it. A score that drops after a reindex tells you something specific; a score compared against someone else's number tells you nothing.
Do I need an evaluation set if the system seems to work?
That is precisely when you need one, because "seems to work" is an impression formed from the handful of questions someone happened to try. The set exists so that six months and three reindexes later you can tell whether it still works, rather than waiting for a user to lose confidence.
Can a RAG system be made to admit it does not know?
Yes, and it should be built that way from the start. It requires deciding what happens when retrieval returns nothing relevant, and treating abstention as correct behaviour rather than a failure. Most demonstrations skip it because an answer always looks better than a refusal, right up until the answer is invented.
Want a number instead of an impression?
We build the evaluation set from your team's own questions and report the scores, so quality is something you can check rather than something we assert. If you have a retrieval system nobody quite trusts any more, that is a good place to start.