Why compare hallucinations and not just ‘good and bad models’
When someone asks about the limitations of hallucinations in LLMs, the useful answer is not a yes or no, but a practical comparison: which model hallucinates less depending on the task, which techniques mitigate it, and at what cost. According to the official documentation of the main providers and industry studies on faithfulness evaluation, none of the current models eliminates hallucination; they reduce it in specific domains and shift it to others.
This article compares popular mitigation options and models from an operational perspective: what works, what doesn’t, and where teams usually get surprised. It is not a list of ‘best AIs’; it is a comparison of approaches with their real limits.
Comparison criteria that actually discriminate
1. Rate of unsupported claims (faithfulness)
The key metric is not ‘does it get it right?’ but ‘does it invent facts that are not in the source?’. In summarization tasks with long context, large models with better adherence to context usually generate fewer unsupported claims than small models, but they lose that advantage when the question requires external knowledge not included in the prompt. Comparing models without fixing the scenario (closed context vs. open question) produces misleading conclusions.
2. Calibration of uncertainty
A calibrated model expresses doubt when it doesn’t know. In practice, almost all popular commercial models are poorly calibrated on niche questions: they answer with the same confidence a verifiable fact as an invented one. According to industry studies, the model’s verbal confidence does not reliably correlate with accuracy, which invalidates the strategy of ‘asking if it is sure’.
3. Cost and latency of mitigation
RAG, verification with web search, self-critique, and multiple voting reduce hallucinations, but each layer adds latency and cost. The honest comparison includes the price per reliable answer, not just the price per token.
Comparison of popular mitigation approaches
RAG (retrieval-augmented generation)
What it does well: it anchors answers to proprietary documents. In questions about internal regulations, manuals, or catalogs, it drastically reduces data invention when the retriever brings the correct fragment.
Where it fails: if the retriever does not find the fragment, the model does not always say ‘I don’t know’; sometimes it fills in with parametric knowledge. It also hallucinates when citing: it invents page numbers or mixes two documents. A common error that only shows up with real use is that RAG over duplicated documents or old versions produces coherent but outdated answers, and the team detects it late because the answer ‘sounds good’.
Verification with real-time web search
What it does well: it improves currency and allows citing sources. Useful for questions about prices, launches, or news.
Where it fails: it introduces dependence on the quality of search results. If the source is a forum or a speculative blog, the model can present a dubious claim as fact with a link included, which paradoxically increases the perceived credibility of the error. It is not useful for counterfactual or hypothetical questions, where there is no source to verify.
Self-critique and verification chains
What it does well: asking the model to review its own answer detects arithmetic errors and internal contradictions in structured tasks.
Where it fails: self-critique shares the same blind spots as generation. A model that doesn’t know a fact also doesn’t know that it doesn’t know it when reviewing, and can ‘confirm’ its own invention. It works better with external verification (calculator, database) than with introspection.
Answers with forced abstention
What it does well: instructing the model to answer ‘not available’ when there is no evidence reduces false claims in bounded domains.
Where it fails: it increases false negatives. In customer service, an excess of abstention degrades the experience as much as an incorrect answer. The threshold must be calibrated per use case, and there is no universal value.
Comparison by type of model: large vs. small vs. specialized
Large general-purpose models
Lower hallucination rate in common knowledge and better following of complex instructions. However, their fluency is a double-edged sword: they invent more elegantly. In highly technical or recent domains, their parametric knowledge becomes outdated and they fill gaps with plausibility.
Small and distilled models
Cheaper and faster, and sufficient for extraction tasks with closed context. Their limitation is generalization: outside the format for which they were tuned, the invention rate rises. Not recommended for open questions without RAG.
Specialized models or models tuned with proprietary data
They can reduce hallucinations in a narrow domain, but fine-tuning does not guarantee faithfulness: it teaches style and format more than facts. A model tuned with incomplete data learns to answer with confidence within that domain, including the gaps. Evaluation with out-of-distribution questions is mandatory before deployment.
What the comparison does not solve
No approach eliminates hallucination in open generative tasks. The reason is structural: the model predicts plausible text, it does not verify truth. Mitigations reduce the probability of verifiable error, but do not eliminate it. Any promise of ‘zero hallucinations’ should be treated with skepticism.
Furthermore, comparison between models expires quickly: versions change and the results of public evaluations are not always replicated in internal domains. The only valid comparison for a team is the one it makes with its own corpus and its own questions.
How to set up a useful comparison in practice
Assemble a set of 50 to 100 real business questions, with known answers and plausible distractors. Run each option (model + technique) on the same set and measure three things: accuracy, unsupported claims, and abstention rate. Manually review a sample of the answers, because automatic faithfulness metrics also get it wrong.
A detail that only appears with experience: the most dangerous hallucinations are not the obvious ones, but those that mix a correct fact with an invented one in the same sentence. The human reviewer tends to validate the entire sentence because the first half is true. The review must be done claim by claim, not answer by answer.
Recommendation by scenario
For customer service with a documentary base, RAG with calibrated abstention and citation review. For research and current affairs, web search with a requirement for a primary source and an explicit warning to the user. For structured data extraction, small models with rule-based validation and external verification. For creative content, assume that facts are not reliable and separate generation from human verification.
The conclusion of the comparison is uncomfortable but honest: the best option is not the model with the fewest hallucinations in the abstract, but the combination that makes the error visible before it causes harm.