ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning

Gongli Xi; Kun Wang; Zeming Gao; Huahui Yi; Haolang Lu; Qianshan Wei; Ye Tian; Wendong Wang

cs.CVFeb 2, 2026

ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning

Authors: Gongli Xi, Kun Wang, Zeming Gao, Huahui Yi, Haolang Lu, Qianshan Wei, Ye Tian, Wendong Wang

Organizations: Beijing University of Posts and Telecommunications, Beijing, China · Nanyang Technological University, Singapore · West China Biomedical Big Data Center, West China Hospital, SCU

Abstract

Large multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual tokens. Yet this capability also increases hallucinations, where the model generates content that is not supported by the input image or the question. To understand this failure mode, we identify reasoning drift: during clue gathering, the model over-focuses on question-irrelevant entities, diluting focus on task-relevant cues and gradually decoupling the reasoning trace from visual grounding. As a consequence, many inference-time localization or intervention methods developed for non-reasoning models fail to pinpoint the true clues in reasoning settings. Motivated by these insights, we introduce ClueRecall, a metric for assessing visual clue retrieval, and present ClueTracer, a training-free, parameter-free, and architecture-agnostic plugin for hallucination suppression. ClueTracer starts from the question and traces how key clues propagate along the model's reasoning pathway (question $\rightarrow$ outputs $\rightarrow$ visual tokens), thereby localizing task-relevant patches while suppressing spurious attention to irrelevant regions. Remarkably, without any additional training, ClueTracer improves all reasoning architectures (including R1-OneVision, Ocean-R1, MM-Eureka, etc.) by $\mathbf{1.21\times}$ on reasoning benchmarks

Figures & tables

Figure 1: (a) Example outputs from a reasoning model; hallucinated content is highlighted in red. (b) Reasoning drift in a multimodal reasoning model, where attention during inference is allocated to task-irrelevant regions. (c) An intuitive illustration: by progressively zooming in on task-critical regions, hallucinations (in red) diminish and the answers become correct. (d) Radar-chart comparisons among different reasoning models.

Figure 2: Our Motivations. From Failure Modes to Output-Guided Clue Tracing.

Figure 3: Overview of our method. Top : ClueRecall uses bbox/category annotations from COCO or lightweight detectors, enabling extension to arbitrary datasets. Middle : layer-wise ClueRecall identifies $L_{\max}$ , the layer with strongest visual clue retrieval. Bottom : ClueTracer traces the question $\rightarrow$ output $\rightarrow$ vision pathway to localize task-relevant regions. Best viewed in color.

Model	HallusionBench (Accuracy %)				VMCBench (Accuracy %)						Average $\Delta$ Acc
Model	qAcc	fAcc	hAcc	aAcc	Gen.	Reason.	OCR	Math	Doc&Chart	Overall	Average $\Delta$ Acc
R1-OneVision	8.57	12.43	31.86	35.43	56.64	45.14	65.39	32.44	52.35	52.99	+22.20%
+ ClueTracer	18.18	30.79	41.16	49.16	76.11	61.81	89.97	51.21	78.88	73.91	+22.20%
Ocean-R1	19.34	27.76	40.00	48.89	77.74	61.69	88.91	46.25	76.58	73.75	+10.60%
+ ClueTracer	30.11	43.35	54.88	63.51	80.31	62.72	96.24	52.00	81.07	77.02	+10.60%
MM-Eureka	23.07	31.50	49.30	58.46	76.31	58.79	90.46	42.44	76.21	72.44	+7.26%

Table 1: Comparison on reasoning-oriented hallucination benchmarks with and without ClueTracer . qAcc, fAcc, hAcc, and aAcc denote, respectively, the accuracy per question pair , per figure , on hard questions , and the overall average . Case studies are provided in Fig. 9 - 12 .

	ClueRecall per Layer (%)						POPE (%)		MMVP (%)
Model	0	6	12	18	24	27	w/o CT	w/ CT	w/o CT	w/ CT
R1-OneVision	30.78%	33.68%	43.11%	50.60%	47.28%	44.11%	70.22%	76.97%	23.33%	46.00%
Ocean-R1	31.05%	32.64%	48.56%	53.79%	50.92%	49.23%	86.77%	85.77%	47.33%	50.00%
MM-Eureka	29.30%	30.57%	45.45%	51.06%	51.38%	48.11%	75.00%	81.38%	44.67%	50.00%
ORSTA-R1	31.14%	32.38%	47.55%	51.66%	55.32%	50.56%	71.36%	82.98%	38.00%	53.33%

Table 2: Per-layer ClueRecall (left) and accuracy on POPE / MMVP with and without ClueTracer (right).

Figure 4: Visualization of reduced reasoning drift and traced visual clues. (a) Comparative scatter plot of attention allocation versus output mentions for task-relevant objects. (b) Case studies showing that ClueTracer traces implicit visual clues required by the question, with red boxes highlighting the discovered conditions in the image.

Panel of Figure 5 — Figure 5: Ablation Study and Comparative Evaluations. Default settings in the experiments are shown in bold . Additional ablations on hyperparameters (e.g., DBSCAN and z-score thresholds) are provided in the Appendix F .

Notation	Description
$\mathcal{RM},\,L$	Multimodal reasoning model (MLRM) and its number of Transformer layers
$I,\,q$	Input image and textual que
$\texttt{cat},\,\texttt{bbox}$	Category of an entity in $I$ and its bounding box in the image plane
$\mathbf{X}_{c},\,\mathbf{X}_{s},\,\mathbf{X}_{q},\,\mathbf{X}_{v}$	Complete input, system prompt tokens, query tokens, and visual tokens
$N_{c},\,N_{s},\,N_{q},\,N_{v}$	Lengths of $\mathbf{X}_{c}$ , $\mathbf{X}_{s}$ , $\mathbf{X}_{q}$ , and $\mathbf{X}_{v}$
$\mathbf{X}_{q_{\texttt{cat}}},\,\mathbf{X}_{\texttt{bbox}}$	Query tokens for category cat ; visual tokens within the bounding box bbox

Table 3: Notations used throughout ClueRecall and ClueTracer .

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Parameter	Value
min_new_tokens	50
max_new_tokens	1024
do_sample	True

Appendix

Table 9

Model	MMMU	MathVista	MMStar	AI2D	ScienceQA	SEEDBench	MM-Vet	MathVision	TVQA	RWQA
R1 - OneVision	36.30	36.63	40.38	46.70	61.54	54.81	48.12	28.99	48.31	46.79
+ ClueTracer (Ours)	50.49	67.00	55.96	68.06	82.35	72.35	74.29	36.41	73.79	54.62
Ocean - R1	56.28	60.89	58.81	74.94	80.09	73.58	73.85	32.35	69.02	60.09
+ ClueTracer (Ours)	55.42	69.50	62.53	79.68	84.39	80.99	79.44	36.97	79.09	66.67
MM - Eureka	51.20	58.42	56.06	71.07	79.64	72.84	73.38	27.42	68.02	54.59
+ ClueTracer (Ours)	60.58	74.75	63.66	80.41	83.48	77.78	85.61	41.78	80.41	56.42

Appendix

Table 4: Accuracy (%) on selected VMCBench . Including: MMMU, MathVista, MMStar, AI2D, ScienceQA, SEEDBench, MM-Vet, MathVision, TableVQABench (TVQA), RealWorldQA (RWQA). Rows are paired as baseline vs. + ClueTracer for each model.

Model	TextVQA	InfoVQA	DocVQA	OCRVQA	VizWiz	ChartQA	GQA	A - OKVQA	OKVQA	VQAv2
R1 - OneVision	64.72	56.36	58.56	66.06	64.18	56.65	61.86	56.71	63.95	66.44
+ ClueTracer	90.56	77.14	94.44	89.38	84.42	82.11	80.93	80.24	81.98	84.03
Ocean - R1	91.01	71.19	92.00	87.31	86.86	75.92	80.68	81.65	86.67	83.56
+ ClueTracer	97.75	82.16	100.00	96.63	90.91	84.63	88.75	89.41	92.35	88.89
MM - Eureka	88.54	73.73	88.42	91.71	83.09	79.36	82.40	79.06	85.68	83.56
+ ClueTracer	94.83	82.75	96.88	93.26	90.44	88.99	87.04	86.82	93.33	87.96

Appendix

Table 5: Accuracy (%) on the remaining VMCBench datasets: TextVQA, InfoVQA, DocVQA, OCRVQA, VizWiz, ChartQA, GQA, A - OKVQA, OKVQA, VQAv2. Rows are paired as baseline vs. + ClueTracer for each model.

Model	Adversarial		Popular		Random		Average
Model	Acc	F1	Acc	F1	Acc	F1	Acc	F1
R1 - OneVision	69.20%	0.6629	70.33%	0.6711	71.12%	0.6769	70.22%	0.6703
+ ClueTracer	80.90%	0.7731	81.60%	0.7825	82.43%	0.7881	81.64%	0.7805
Ocean - R1	85.53%	0.8481	86.80%	0.8592	87.97%	0.8707	86.77%	0.8593
+ ClueTracer	84.43%	0.8271	86.13%	0.8458	86.70%	0.8496	85.76%	0.8408
MM - Eureka	74.17%	0.7107	74.21%	0.7108	76.63%	0.7308	75.00%	0.7174

Appendix

Table 6: POPE : accuracy (%) and F1 by split (Adversarial / Popular / Random / Average).

Figure 6: Left: Accuracy versus output–token length on MMVP across models; Right: attention allocation with and without ClueTracer .

Dataset	Setting	Ocean-R1	ORSTA	MM-Eureka
RWQA	w/o ClueTracer	63.27%	65.23%	60.39%
RWQA	with ClueTracer	70.33%	67.84%	65.49%
3DSRBench	w/o ClueTracer	37.66%	43.44%	56.25%
3DSRBench	with ClueTracer	40.06%	45.14%	59.24%

Appendix

Table 7: Results on RWQA and 3DSRBench with and without ClueTracer .

Figure 7: Case study on clustering methods. Each subfigure has two parts. Left: ClueTracer attention, i.e., a visualization of the attention map, where the ClueTracer -selected visual patches $\mathcal{V}_{k}^{\star}$ are outlined with black boxes. Right: clustering results from different methods. Because K - means is sensitive to noise, the resulting crops include large amounts of question-irrelevant content induced by only two noisy points, whereas DBSCAN avoids this issue.

Figure 8: Case study on attention choices. Attention identifies question-relevant tokens and correctly crops the hand, thereby guiding the model to the correct answer. Question to visual denotes cross-attention between the input question and input visual tokens; because the model is not trained for this alignment, the signal is weak. Output to visual directly leverages generation-side attention, which is mainstream in MLLMs; however, in MLRMs, reasoning drift introduces numerous distractors.

Method	0.5	1	1.5	2	3
HallusionBench + MM-Eureka	63.51	65.10	63.51	63.15	61.65
POPE + MM-Eureka	79.86	81.38	80.32	79.13	80.92
POPE + Orsta	83.32	83.67	85.01	83.12	82.91

Appendix

Table 8: Sensitivity of accuracy to

\tau_{q}

on HallusionBench and POPE.

Method	0.5	1	1.5	2	3
HallusionBench + MM-Eureka	66.43	65.10	63.86	63.42	63.77
POPE + MM-Eureka	82.32	81.38	79.22	79.62	80.62
POPE + Orsta	84.42	83.67	85.02	83.12	79.02

Appendix

Table 9: Sensitivity of accuracy to

\tau_{o}

on HallusionBench and POPE.

Method	0.5	1	1.5	2	3
HallusionBench + MM-Eureka	64.22	65.46	65.10	63.95	63.45
POPE + MM-Eureka	82.92	82.41	81.38	80.92	79.12
POPE + Orsta	81.82	83.22	83.67	85.81	83.22

Appendix

Table 10: Sensitivity of accuracy to

\tau_{v}

on HallusionBench and POPE.

Method	1.1	1.9	2.1	2.9	3.1	3.9
HallusionBench + MM-Eureka	63.31	64.57	64.48	65.10	65.20	65.37
POPE + MM-Eureka	78.52	79.62	80.62	81.38	81.32	81.22
POPE + Orsta	84.52	84.62	85.41	83.67	84.52	83.02

Appendix

Table 11: Sensitivity of accuracy to DBSCAN-

\epsilon

on HallusionBench and POPE.

Mode / sequence length	offline/–	1	20	50	128	256
Latency (hours / 1000 items)	0.91	0.97	1.08	1.36	1.62	1.79
Throughput (tokens/s)	39.80	35.52	31.24	27.71	20.91	18.08

Appendix

Table 12: Latency and throughput under different online and offline settings.

Model	0	12	24	30	36	40	42	48	54	63
MM-Eureka-32B	0.3060	0.3377	0.4978	0.5358	0.5781	0.6271	0.5474	0.4854	0.3992	0.3099
Orsta-32B	0.3165	0.3397	0.4922	0.5381	0.5669	0.6350	0.5596	0.4961	0.4059	0.3172

Appendix

Table 13: Layer-wise ClueRecall on 32B models.

model	Arch	0	6	12	17	18	24	31
LLaVA-1.5-7B	LLaVA-1.5	33.81%	38.00%	43.04%	50.59%	48.29%	47.03%	39.02%
LLaVA-Mistral-7B	LLaVA-NeXT	10.37%	23.95%	46.00%	49.04%	49.72%	41.10%	35.95%

Appendix

Table 14: Layer-wise ClueRecall on LLaVA-1.5-7B and LLaVA-Mistral-7B.

Figure 9: Case study (Part One). The visual crops selected by ClueTracer are highlighted with red bounding boxes in the figure.

Figure 10: Case study (Part Two). The visual crops selected by ClueTracer are highlighted with red bounding boxes in the figure.

Figure 11: Failure case study (Part One). The visual crops selected by ClueTracer are highlighted with red and yellow bounding boxes. We observe that, in most cases, ClueTracer crops the correct regions, but it fails to do so in the third example.

Figure 12: Failure case study (Part Two). The visual crops selected by ClueTracer are highlighted with red/yellow boxes. We observe that these failures are driven by object-level semantics: in these examples, the model recognizes semantically similar but incorrect concepts (e.g., glass vs. cup , seat vs. chair ).

Method	Ocean-R1	MM-Eureka
Standard Grad-CAM	0.267	0.274
ClueTracer + Grad-CAM	0.377	0.390

Appendix

Table 15: Compatibility of ClueTracer with Grad-CAM. Incorporating question-relevant reasoning tokens substantially improves visual clue localization over standard final-token attribution.

VLM Hallucination Mitigation

Vision-language model hallucination mitigation keeps generated claims grounded in visual evidence.

See all 229 papers

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models
Feb 3, 2026Hao Fang, Jinyu Li, Jiawei Kong +4Hallucination Mitigation Multimodal Reasoning
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen
Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization
May 27, 2026Jiawei Kong, Hao Fang, Shunxiang Liao +5Multimodal Grounding Direct Preference Optimization
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen
Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models
Jul 24, 2026Junlin Fang, Do Nguyen-Thanh, Xiaogang Xu +2LLM Hallucination Detection Large Reasoning Models
College of Computing and Data Science, Nanyang Technological University · Zhejiang University · Australian Artificial Intelligence Institute, University of Technology Sydney +1

Feb 3, 2026J/K move · Enter open · S save

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models

Hao Fang, Jinyu Li, Jiawei Kong, Tianqu Zhuang+3

Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen

Hallucination Mitigation Multimodal Reasoning Multimodal CoT Reasoning Modality Reliance in Multimodal Models CoT Compression Preference Optimization

Open paper AskPDF arXiv

While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this paper, we experimentally analyze the hallucination cause and propose C3PO, a training-based mitigation framework comprising Chain-of-Thought Compression and Contrastive Preference Optimization. Firstly, we identify that introducing reasoning mechanisms exacerbates models' reliance on language priors while overlooking visual inputs, which can produce CoTs with reduced visual cues but redundant text tokens. To this end, we propose to selectively filter redundant thinking tokens for a more compact and signal-efficient CoT representation that preserves task-relevant information while suppressing noise. In addition, we observe that the quality of the reasoning trace largely determines whether hallucination emerges in subsequent responses. To leverage this insight, we introduce a reasoning-enhanced preference tuning scheme that constructs training pairs using high-quality AI feedback. We further design a multimodal hallucination-inducing mechanism that elicits models' inherent hallucination patterns via carefully crafted inducers, yielding informative negative signals for contrastive correction. We provide theoretical justification for the effectiveness and demonstrate consistent hallucination reduction across diverse MLRMs and benchmarks.

Abstract

VLM Hallucination Mitigation

Explore similar work

Seeing Through the Chain: Understanding and Mitigating Hallucinations in Multimodal Large Reasoning Models

Reasoning Matters: Mitigate Hallucination in Multimodal Large Reasoning Models via Reasoning-Conditioned Preference Optimization

Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models