3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

Makanjuola Ogunleye; Eman Abdelrahman; Ismini Lourentzou

cs.CVApr 9, 2026

3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding

Authors: Makanjuola Ogunleye, Eman Abdelrahman, Ismini Lourentzou

Organizations: Virginia Tech · University of Illinois Urbana-Champaign

Abstract

Large multimodal models are increasingly used as the reasoning core of embodied agents operating in 3D environments, yet they remain prone to hallucinations that can produce unsafe and ungrounded decisions. Existing inference-time hallucination mitigation methods largely target 2D vision-language settings and do not transfer to embodied 3D reasoning, where failures arise from object presence, spatial layout, and geometric grounding rather than pixel-level inconsistencies. We introduce 3D-VCD, the first inference-time visual contrastive decoding framework for hallucination mitigation in 3D embodied agents. 3D-VCD constructs a distorted 3D scene graph by applying semantic and geometric perturbations to object-centric representations, such as category substitutions and coordinate or extent corruption. By contrasting predictions under the original and distorted 3D contexts, our method suppresses tokens that are insensitive to grounded scene evidence and are therefore likely driven by language priors. We evaluate 3D-VCD on the 3D-POPE and HEAL benchmarks and show that it consistently improves grounded reasoning without any retraining, establishing inference-time contrastive decoding over structured 3D representations as an effective and practical route to more reliable embodied intelligence.

Figures & tables

Figure 1: Split-wise results on 3D-POPE benchmark. Each split (Random, Popular, Adversarial) shows Precision, F1, and Accuracy. Our 3D-VCD training-free hallucination mitigation method consistently improves precision, F1, and accuracy while substantially reducing over-affirmation rates across all splits.

Figure 2: 3D- VCD Overview. Given 3D environment observations, 3D- VCD builds a structured 3D scene graph ( ${\mathcal{G}}$ ) encoding object categories, centroids, and extents, and injects controlled semantic and geometric perturbations to obtain a distorted version of the environment ( $\hat{\mathcal{G}}$ ). The MLLM agent processes both contexts in parallel, given the textual query ( $\mathbf{x}$ ). 3D-VCD contrastively fuses their logits to reveal and suppress hallucination-prone tokens. This training-free procedure enforces 3D-grounded reasoning at inference time.

3D-POPE	Model	Training-Free	Precision $\uparrow$	Recall $\uparrow$	F1 Score $\uparrow$	Accuracy $\uparrow$	Yes (%) $\downarrow$
Random	Random Baseline	-	50.00	50.00	50.00	50.00	50.00
	3D-LLM [ 12 ]	✘	50.03	99.88	66.67	50.07	99.81
	3D-VisTA [ 46 ]	✘	50.12	53.58	51.79	49.66	53.95
	LEO [ 15 ]	✘	51.95	77.65	62.25	52.91	74.73
	3D-VCD (Ours)	✔	62.16	92.90	74.48	67.99	75.15
Popular	Random Baseline	-	50.00	50.00	50.00	50.00	50.00

Table 1: Results on the 3D-POPE benchmark. Across all three evaluation categories (Random, Popular, and Adversarial) 3D-VCD achieves the highest precision, accuracy, and F1-score, surpassing prior 3D language models (3D-LLM, 3D-VisTA, and LEO). The substantial reduction in Yes-rate (e.g., 99.81% → 75.15% in the Random set) alongside consistent gains in precision and accuracy demonstrates that VCD effectively mitigates over-affirmation bias and hallucination, yielding more balanced and reliable predictions in 3D reasoning.

Model	$C_{O}$ (%) $\downarrow$	$C_{S}$ (%) $\downarrow$
Llama-3-8B-Instruct-VCD	2.39	12.43
Llama-3-8B-Instruct	2.58	9.49
Qwen-14B-Instruct-VCD	3.55	5.00
Qwen-14B-Instruct	4.13	16.45
Gemma-2-9b-it	11.60	5.70
DS-R1-Distil-LLaMA-8B	13.90	27.20

Table 2: Hallucination rates (CHAIR) on the HEAL probing set under the distraction injection prompt. Applying 3D-VCD significantly reduces hallucinations and improves grounding for both Llama-3 and Qwen models to the factual objects and states present in the environment.

Panel of Figure 3 — Figure 3: Ablation on distortion types for VCD in 3D-POPE (F1) . $x$ -axis shows distortion tags (ascending overall F1).

Figure 4: Qualitative comparison on 3D-POPE. The baseline incorrectly predicts the absence of a dining table, missing the true table object. In contrast, 3D-VCD correctly grounds and identifies the dining table by aligning contrastive decoding with the scene graph.

Figure 5: Qualitative comparison on HEAL benchmark. For the task brushing_lint_off_clothing , the baseline Qwen-14B-Instruct model hallucinates a nonexistent microwave.n.01_1 in its symbolic goal prediction. In contrast, 3D- VCD produces clean symbolic goals with no hallucinated objects, correctly grounding all sweaters on the bed and removing dust-related states as required by the instruction. The right panel shows the first-person view of the agent interacting with the sweaters in the scene.

Figure 6: 3D-VCD inference time as a function of scene complexity (number of objects).

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

3D-POPE	Geometric Distortions ( $\boldsymbol{\varepsilon}$ )	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	Yes % $\downarrow$
Random	3D-VCD $\varepsilon{=}0.01$	59.20	98.02	73.81	63.86	86.05
	3D-VCD $\varepsilon{=}0.05$	61.89	95.17	75.00	67.65	78.41
	3D-VCD $\varepsilon{=}0.15$	61.68	95.28	74.88	67.39	78.82
	3D-VCD $\varepsilon{=}0.45$	59.21	98.25	73.89	63.79	86.55
Popular	3D-VCD $\varepsilon{=}0.01$	51.54	97.66	67.47	52.45	95.69
	3D-VCD $\varepsilon{=}0.05$	52.35	94.09	67.27	53.34	91.61

Appendix

Table 3: Effect of Geometric Distortion Strength (

\varepsilon

) on 3D-POPE Performance. Lower yes-rate indicates reduced hallucination.

3D-POPE	Method	Precision $\uparrow$	Recall $\uparrow$	F1 $\uparrow$	Accuracy $\uparrow$	Yes % $\downarrow$
Random	Low-SemSub-Geom ( $\alpha$ =1.0)	62.16	92.90	74.48	67.99	75.15
	Low-SemSub-Geom ( $\alpha$ =0.5)	59.19	96.22	73.29	64.39	82.56
	High-SemSub-Geom ( $\alpha$ =1.0)	59.52	95.65	73.38	64.58	81.99
	High-SemSub-Geom ( $\alpha$ =0.5)	59.42	95.95	73.39	64.56	82.26
Popular	Low-SemSub-Geom ( $\alpha$ =1.0)	52.35	92.86	66.95	54.00	89.02
	Low-SemSub-Geom ( $\alpha$ =0.5)	51.39	96.23	66.99	52.72	93.39

Appendix

Table 4: Ablation on Semantic and Geometric Distortions under varying contrastive strengths

\alpha

on 3D-POPE.

Env	Models	Base Prompt		TaskDescMod				SceneMod
		Base Prompt		DistInj		TaskObjRem		SynonymSub		SceneTaskCon
		$C_{O}\downarrow$	$C_{S}\downarrow$	$C_{O}\downarrow$	$C_{S}\downarrow$	$C_{O}\downarrow$	$C_{S}\downarrow$	$C_{O}\downarrow$	$C_{S}\downarrow$	$C_{O}\downarrow$	$C_{S}\downarrow$
BEHAVIOR	Gemma-2-9b-it	0.87	1.80	11.60	5.70	10.8	4.8	11.4	2.8	73.1	8.2
	DS-R1-Distil-LLaMA-8B	1.26	8.16	13.90	27.20	6.8	13.3	20.4	10.1	60.0	19.6
	Llama-3-8B-Instruct	1.93	2.15	2.58	9.49	5.0	6.7	2.2	7.4	17.8	7.7
	Qwen-14B-Instruct	1.1	11.0	4.1	16.5	3.5	10.0	3.9	10.0	53.9	5.7

Appendix

Table 5: Comparative Hallucination Rates (CHAIR) on HEAL Probes. We report Object Hallucination (

C_{O}

) and State Hallucination (

C_{S}

) rates. Our proposed 3D-VCD method (applied to Qwen-14B) shows reduced hallucination, particularly under Distractor Injection.

Figure 7: Qualitative comparison on 3D-POPE. The baseline 3D-LLM hallucinates the presence of a bed. In contrast, 3D-VCD correctly answers No by contrasting logits under perturbed 3D scene graphs, effectively suppressing hallucinated object activations. The right panel shows the object-level scene segmentation for reference.

Figure 8: Qualitative comparison on 3D-POPE. The baseline 3D-LLM hallucinates a desk object and incorrectly predicts its presence. In contrast, 3D-VCD correctly determines that no desk exists by suppressing spurious category matches through contrastive decoding aligned with the object-centric scene graph.

Figure 9: 3D-VCD model input and output.

VLM Hallucination Mitigation

Vision-language model hallucination mitigation keeps generated claims grounded in visual evidence.

See all 228 papers

Geo3R: Mitigating Spatial Reasoning Hallucination in Multimodal Large Language Models
Jul 23, 2026Mingyu Wang, Weilin Jin, Wenbo Li +33D Spatial Reasoning Hallucination in Language Models
Peking University Beijing, China · Joy Future Academy Beijing, China
Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs
Jul 28, 2026Arnav Bendre, Guneesh Gupta, Kavish Grover +2Object Hallucination in VLMs Multimodal Large Language Models
Department of Hydro and Renewable Energy Indian Institute of Technology, Roorkee · Department of Electrical Engineering Indian Institute of Technology, Roorkee · Department of Mechanical Engineering Indian Institute of Technology, Roorkee +2
Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correction
Jul 30, 2026Lei Yang, Xinze Liu, Dayan Wu +7Hallucination Detection in VLMs LLM Hallucination Mitigation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA

Jul 23, 2026J/K move · Enter open · S save

Geo3R: Mitigating Spatial Reasoning Hallucination in Multimodal Large Language Models

Mingyu Wang, Weilin Jin, Wenbo Li, Haoyang Huang+2

Peking University Beijing, China · Joy Future Academy Beijing, China

3D Spatial Reasoning Hallucination in Language Models Multimodal Large Language Models VLM Hallucination Mitigation

Open paper AskPDF arXiv

Despite remarkable progress in visual understanding, Multimodal Large Language Models (MLLMs) remain prone to hallucinations when reasoning about spatial relationships, often producing judgments that contradict the true 3D structure of the scene. Though several existing works have proposed to mitigate hallucinations, our analysis indicates that they show limited effectiveness in spatial reasoning, as they fail to bridge the fundamental gap between 2D visual representations and 3D spatial reality. Based on this finding, we define hallucinations arising from insufficient spatial structure modeling as spatial reasoning hallucination, a subcategory of relation hallucination that existing mitigation methods fail to address. We further identify three typical scenarios where such hallucinations frequently occur: perspective effects, object orientation, and viewpoint changes. To this end, we propose Geo3R, a training-free, plug-and-play framework that incorporates geometric evidence and structured 3D reasoning to mitigate spatial reasoning hallucination. Experiments on three benchmarks, covering 18 tasks across all three scenarios, show that Geo3R substantially reduces spatial reasoning hallucination across diverse MLLMs without additional training, outperforming existing models and methods.

Abstract

VLM Hallucination Mitigation

Explore similar work

Geo3R: Mitigating Spatial Reasoning Hallucination in Multimodal Large Language Models

Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs

Hallucinations Leave a Grounding Signature:Verifier-Guided Decoding for Selective Object Correction