Mitigating LLM Hallucinations through Domain-Grounded Tiered Retrieval
Organizations: Computational Unit, Z.H. College of Engineering & Technology, Aligarh Muslim University, India · Interdisciplinary Center for Artificial Intelligence, Aligarh Muslim University, India · Department of Computer Engineering, Aligarh Muslim University, India
Abstract
Large Language Models (LLMs) have achieved unprecedented fluency but remain susceptible to "hallucinations" - the generation of factually incorrect or ungrounded content. This limitation is particularly critical in high-stakes domains where reliability is paramount. We propose a domain-grounded tiered retrieval and verification architecture designed to systematically intercept factual inaccuracies by shifting LLMs from stochastic pattern-matchers to verified truth-seekers. The proposed framework utilizes a four-phase, self-regulating pipeline implemented via LangGraph: (I) Intrinsic Verification with Early-Exit logic to optimize compute, (II) Adaptive Search Routing utilizing a Domain Detector to target subject-specific archives, (III) Refined Context Filtering (RCF) to eliminate non-essential or distracting information, and (IV) Extrinsic Regeneration followed by atomic claim-level verification. The system was evaluated across 650 queries from five diverse benchmarks: TimeQA v2, FreshQA v2, HaluEval General, MMLU Global Facts, and TruthfulQA. Empirical results demonstrate that the pipeline consistently outperforms zero-shot baselines across all environments. Win rates peaked at 83.7% in TimeQA v2 and 78.0% in MMLU Global Facts, confirming high efficacy in domains requiring granular temporal and numerical precision. Groundedness scores remained robustly stable between 78.8% and 86.4% across factual-answer rows. While the architecture provides a robust fail-safe for misinformation, a persistent failure mode of "False-Premise Overclaiming" was identified. These findings provide a detailed empirical characterization of multi-stage RAG behavior and suggest that future work should prioritize pre-retrieval "answerability" nodes to further bridge the reliability gap in conversational AI.
Figures & tables
| Type | Nature of Error | Example | Primary Cause |
|---|---|---|---|
| Factual | Incorrect statements contradicting truth | “Albert Einstein won two Nobel Prizes in Physics.” | Inaccurate memory / data bias |
| Numerical | Wrong figures or quantities | “The Earth’s atmosphere contains 40% oxygen.” | Lack of numerical reasoning |
| Logical | Invalid reasoning chain | “All doctors are scientists; Li is a scientist; therefore, Li is a doctor.” | Weak inferential ability |
| Commonsense | Violates basic knowledge | “The sun rises in the west and sets in the east.” | Lack of world grounding |
| Temporal | Outdated or mixed-up timelines | “Barack Obama is the current President of the United States.” | Static knowledge |
| Attributional | Wrong author or source | “The theory of relativity was proposed by Isaac Newton.” | Entity confusion |
| Benchmark | N | Domain | Query Type | Key Challenges |
| TimeQA v2 | 150 | Temporal biographical | ‘Who held role X between dates A–B?’ | Sparse time-window documentation; correct refusal on unknowable queries |
| FreshQA v2 | 150 | Current events + false premises | Factual + denial queries | 39% of queries embed false or unresolved premises; requires refusal, not inference |
| HaluEval General | 150 | Open domain (mixed) | General knowledge, science, coding | Broad coverage; retrieval advantage lower vs. parametrically-strong baseline |
| MMLU Global Facts | 50 | Global statistics | Precise numerical recall | Exact figures unavailable in small-model memory; short answers |
| TruthfulQA | 150 | Myths / misconceptions | Myth-busting; denial of false premises | Must resist confabulation; false-premise resistance essential |
| Benchmark | N | Proposed Wins | Tie | Baseline Wins | Win Rate | Hallucination | Groundedness |
| TimeQA v2 | 86* | 72 | 10 | 4 | 83.7% | 13.6% | 86.4% |
| MMLU Global Facts | 50 | 39 | 8 | 3 | 78.0% | 33.1% | 66.9% |
| FreshQA v2 | 150 | 97 | 37 | 16 | 64.7% | 3.5%† | 19.2%† |
| TruthfulQA | 150 | 82 | 56 | 12 | 54.7% | 15.1% | 84.9% |
| HaluEval General | 150 | 75 | 45 | 30 | 50.0% | 21.2% | 78.8% |
| Combined (650) | 586 | 365 | 220** | 65 | 62.3% |
| Failure Mode | Frequency | Primary Benchmarks | Primary Root Cause |
|---|---|---|---|
| Open-Domain Parametric Competition | 30 | HaluEval | High-quality training data in the baseline model matches the quality of retrieved web sources for common knowledge. |
| False-Premise Overclaiming | 14 | TimeQA, FreshQA, TruthfulQA | Lack of a pre-retrieval answerability check; the system answers the premise instead of correcting it. |
| Vagueness (Verbose vs. Concise) | 12 | FreshQA, TruthfulQA | The generator produces long, hedged responses that the judge finds less useful than the baseline’s concise denial. |
| Retrieval Distraction | 9 | TruthfulQA, HaluEval | Tangential information in a retrieved document leads the generator away from the direct answer. |
| Numerical precision / Data mismatch | 5 | MMLU, FreshQA | The retrieved source utilizes different aggregation methods, timeframes, or units than the generated claim. |
| Structured Data Extraction Errors | 4 | FreshQA, MMLU | Small-model limitations (Llama 3.1 8B) leading to misreading of markdown tables or extracting wrong values from documents. |
LLM Hallucination Mitigation
Large language model hallucination mitigation develops ways to detect and reduce unsupported, fabricated, or inconsistent outputs.
See all 179 papers