Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

Dayong Liang; Xiao-Yong Wei; Linhui Xiao; Changmeng Zheng

cs.AINov 14, 2025

Multi-agent Undercover Gaming: Hallucination Removal via Counterfactual Test for Multimodal Reasoning

Authors: Dayong Liang, Xiao-Yong Wei, Linhui Xiao, Changmeng Zheng

Organizations: School of Future Technology, South China University of Technology, Guangzhou, China · Department of networked intelligence, Peng Cheng Laboratory, Shenzhen, China · College of Computer Science, Sichuan University, Chengdu, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China

Abstract

Hallucination continues to pose a major obstacle in the reasoning capabilities of large language models (LLMs). Although the Multi-Agent Debate (MAD) paradigm offers a promising solution by promoting consensus among multiple agents to enhance reliability, it relies on the unrealistic assumption that all debaters are rational and reflective, which is a condition that may not hold when agents themselves are prone to hallucinations. To address this gap, we introduce the Multi-agent Undercover Gaming (MUG) protocol, inspired by social deduction games like "Who is Undercover?". MUG reframes MAD as a process of detecting "undercover" agents (those suffering from hallucinations) by employing multimodal counterfactual tests. Specifically, we modify reference images to introduce counterfactual evidence and observe whether agents can accurately identify these changes, providing ground-truth for identifying hallucinating agents and enabling robust, crowd-powered multimodal reasoning. MUG advances MAD protocols along three key dimensions: (1) enabling factual verification beyond statistical consensus through counterfactual testing; (2) introducing cross-evidence reasoning via dynamically modified evidence sources instead of relying on static inputs; and (3) fostering active reasoning, where agents engage in probing discussions rather than passively answering questions. Collectively, these innovations offer a more reliable and effective framework for multimodal reasoning in LLMs. The source code can be accessed at https://github.com/YongLD/MUG.git.

Figures & tables

Figure 1: Overview of the Counterfactual Undercover Debate Framework, illustrating the dynamic interactions between agents, the current game state, and the decision-making process influenced by counterfactual information. The diagram highlights the roles of normal and undercover agents, as well as the flow of information and reasoning throughout the game.

Figure 2: Comparison of Chain-of-thought (CoT), Multi-agent Debate (MAD), and our proposed Multi-agent Undercover Gaming (MUG). Q: input question,

I^{+}

: input image,

I^{-}

: edited counterfactual image, A: MLLM agent, R: output response.

Figure 3: Illustration of counterfactual editing constraints, including visual similarity, semantic change, and naturalness.

Model	MMMU	MMStar	HallusionBench				POPE
Model	Acc.	Acc.	aAcc.	fAcc.	qAcc.	Avg.	Acc.	Pre.	Rec.	F1
DeepSeek-VL-7B	38.3	40.5	53.9	24.9	24.6	34.5	86.8	94.3	78.4	85.6
LLaVA-OneVision-7B	47.9	61.9	48.4	21.4	25.1	31.6	87.9	96.7	78.5	86.6
LLaVA-Next-Llama3-8B	43.1	43.9	52.3	25.7	21.3	33.1	87.1	88.1	95.1	87.1
InternVL2-8B	50.2	61.5	63.9	35.0	36.0	45.0	86.0	96.6	74.7	84.2
Gemini-1.5-Pro	60.6	59.1	63.0	36.1	37.6	45.6	88.6	93	83.9	88.2

Table 1: Performance comparison across multiple benchmarks. The improvements against baselines are statistically significant by one-tailed paired

t

-test with

p

<

0.01. Acc.: accuracy, aAcc.: average accuracy, fAcc.: unique figure accuracy, qAcc.: unique quetion accuracy, Avg.: average score of the three metrics. Pre.: precision, Rec.: recall.

Figure 4: Performance comparison for different models across hallucination categories in the POPE and HallusionBench datasets.

Figure 5: Ablation results demonstrating the impact of counterfactual editing and undercover mechanisms on performance across three benchmarks.

Round	HallusionBench	MMMU	MMStar
0	67.31	47.88	61.93
1	69.40	50.33	63.80
2	65.89	48.02	63.92
3	66.95	47.56	62.76

Table 2: Model performance with respect to the iteration round of game.

Figure 6: Statistics of survived undercover agents across three game rounds.

Figure 7: Case study of our proposed MUG framework compared with traditional chain-of-thought reasoning (CoT) and multiagent debate (MAD).

Multi-Agent LLM Systems

Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.

See all 1,039 papers

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Feb 3, 2026Hao Fang, Jinyu Li, Jiawei Kong +4Hallucination Mitigation Multimodal Reasoning
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen
Multi-Agent Debate with Memory Masking
Mar 3, 2026Hongduan Tian, Xiao Feng, Ziyuan Zhao +3Multi-Agent Debate LLM Reasoning
TMLR Group, Hong Kong Baptist University · Search Algorithm Group, WeChat, Tencent
When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?
Oct 23, 2025Yongqiang Chen, Gang Niu, James Cheng +2Multi-Agent Debate Multi-Agent Collaboration
1The Chinese University of Hong Kong · 2RIKEN Center for Advanced Intelligence Project · 3Hong Kong Baptist University +1

Feb 3, 2026J/K move · Enter open · S save

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models

Hao Fang, Jinyu Li, Jiawei Kong, Tianqu Zhuang+3

Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen

Hallucination Mitigation Multimodal Reasoning Multimodal CoT Reasoning Modality Reliance in Multimodal Models CoT Compression Preference Optimization

Open paper AskPDF arXiv

While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this paper, we experimentally analyze the hallucination cause and propose C3PO, a training-based mitigation framework comprising Chain-of-Thought Compression and Contrastive Preference Optimization. Firstly, we identify that introducing reasoning mechanisms exacerbates models' reliance on language priors while overlooking visual inputs, which can produce CoTs with reduced visual cues but redundant text tokens. To this end, we propose to selectively filter redundant thinking tokens for a more compact and signal-efficient CoT representation that preserves task-relevant information while suppressing noise. In addition, we observe that the quality of the reasoning trace largely determines whether hallucination emerges in subsequent responses. To leverage this insight, we introduce a reasoning-enhanced preference tuning scheme that constructs training pairs using high-quality AI feedback. We further design a multimodal hallucination-inducing mechanism that elicits models' inherent hallucination patterns via carefully crafted inducers, yielding informative negative signals for contrastive correction. We provide theoretical justification for the effectiveness and demonstrate consistent hallucination reduction across diverse MLRMs and benchmarks.

Abstract

Multi-Agent LLM Systems

Explore similar work

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models

Multi-Agent Debate with Memory Masking

When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?