Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

Zequn Xie; Zhengyang Sun

cs.CLMar 28, 2026

Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

Authors: Zequn Xie, Zhengyang Sun

Organizations: Polytechnic Institute, Zhejiang University, Hangzhou, China · School of Computer Science and Technology, Xinjiang University, Urumqi, China

Abstract

Retrieval-Augmented Generation (RAG) aims to reduce hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, RAG introduces a critical challenge: hallucination on hallucination," where flawed retrieval results mislead the generation model, leading to compounded hallucinations. To address this issue, we propose VOTE-RAG, a novel, training-free framework with a two-stage structure and efficient, parallelizable voting mechanisms. VOTE-RAG includes: (1) Retrieval Voting, where multiple agents generate diverse queries in parallel and aggregate all retrieved documents; (2) Response Voting, where multiple agents independently generate answers based on the aggregated documents, with the final output determined by majority vote. We conduct comparative experiments on six benchmark datasets. Our results show that VOTE-RAG achieves performance comparable to or surpassing more complex frameworks. Additionally, VOTE-RAG features a simpler architecture, is fully parallelizable, and avoids the problem drift" risk. Our work demonstrates that simple, reliable ensemble voting is a superior and more efficient method for mitigating RAG hallucinations.

Figures & tables

Fig. 1: An overview of our VOTE-RAG framework. It leverages parallel ensemble voting to enhance retrieval breadth and generation robustness against ”Hallucination on Hallucination.”

Method	NQ		TriviaQA		PopQA		2Wiki		HotpotQA		StrategyQA
Method	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM
Naive RAG	38.20	50.08	60.80	69.55	37.60	45.69	14.80	24.27	25.80	35.80	62.60
IRCoT	28.60	37.36	47.20	54.56	27.00	33.02	22.80	31.19	25.20	34.40	53.60
Iter-RetGen	$\textdagger$ 40.80	$\textdagger$ 52.31	63.00	72.23	$\textdagger$ 39.60	46.41	15.00	24.75	27.80	38.93	62.00
FLARE	19.40	27.68	53.60	63.05	21.60	24.35	9.20	20.13	16.60	23.74	42.80
SuRe	34.80	51.35	47.60	64.11	41.80	48.99	10.20	18.60	19.00	32.39	0.00

TABLE I: Overall evaluation results of VOTE-RAG and other baselines on six benchmarks. Bold marks the best-performing method.

Method	HotpotQA (EM)			TriviaQA (EM)
	$N=3$	$N=5$	$N=7$	$N=3$	$N=5$	$N=7$
VOTE-RAG	29.80	30.10	30.70	61.10	62.40	63.20

TABLE II: Ablation study on the number of agents (

N

) on the HotpotQA and TriviaQA datasets (EM Score). As hypothesized, VOTE-RAG’s performance, which relies on voting, scales robustly with more agents.In contrast, the performance of DRAG degrades at

N=7

, consistent with the ”problem drift” caused by excessive debate. [ 8 ]

Hallucination in RAG

See all 19 papers

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
Apr 17, 2026Fabian Ridder, Laurin Lessel, Malte SchillingHallucination in Language Models RAG Robustness
Computer Science Department, University of Münster, Münster, Germany
MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
Mar 25, 2026Zhuo Li, Yupeng Zhang, Pengyu Cheng +8LLM Hallucination Mitigation LLM Hallucination Detection
Qwen Large Model Application Team, Alibaba · The Chinese University of Hong Kong, Shenzhen
Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems
Oct 28, 2025Yihan Li, Xiyuan Fu, Ghanshyam Verma +2Retrieval-Augmented Generation LLM Hallucination Mitigation
Electronic Information School, Wuhan University, Wuhan, China · School of Computing, Dublin City University, Dublin, Ireland · School of Public Health, Wuhan University, Wuhan, China +2

Apr 17, 2026J/K move · Enter open · S save

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

Fabian Ridder, Laurin Lessel, Malte Schilling

Computer Science Department, University of Münster, Münster, Germany

Hallucination in Language Models RAG Robustness LLM Hallucination Mitigation Retrieval-Augmented Generation Language Modeling LLM Hallucination Detection+1

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) is widely used to augment the input to Large Language Models (LLMs) with external information, such as recent or domain-specific knowledge. Nonetheless, current models still produce closed-domain hallucinations and generate content that is unsupported by the retrieved context. Current detection approaches typically treat hallucination as a post-hoc problem, relying on black-box consistency checks or probes over frozen internal representations. In this work, we demonstrate that hallucination detection based on internal state representation can also serve as a direct training signal. We introduce RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and RAGognizer, a hallucination-aware fine-tuning approach that integrates a lightweight detection head into an LLM, allowing for the joint optimization of language modeling and hallucination detection. This joint objective forces the model to improve the separability of its internal states regarding hallucinations while simultaneously learning to generate well-formed and meaningful responses. Across multiple benchmarks, RAGognizer achieves state-of-the-art token-level hallucination detection while substantially reducing hallucination rates during generation, without degrading language quality or relevance.

Abstract

Hallucination in RAG

Explore similar work

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination

Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems