MAGIC: Modality-Aware Gini Inspection for Clinical Hallucination Detection in LMMs
Authors: Sachin Acharya, Niyoj Oli, Shishir Adhikari, Aminu Lawal, Yash Raj Shrestha, Prashnna K Gyawali, Binod Bhattarai
Organizations: Nepal Applied Mathematics and Informatics Institute for research, Nepal · Fogsphere (Redev.AI Ltd, UK) · Global College of Technology, Nepal · University of Aberdeen, UK · University of Lausanne, Switzerland · University of West Virginia, USA
Recent advances in medical large multimodal models (LMMs) have brought promising progress to clinical decision support, yet hallucination, the generation of confident but clinically unverifiable outputs, poses a critical threat to the reliability demanded in high-stakes medical settings. Existing hallucination detection methods, predominantly designed for unimodal settings, fail to capture the visual-linguistic dynamics inherent to medical imaging. We formulate the problem of hallucination detection in LMMs as disproportionate distribution of attention weights across the modalities during answer token generation. We quantify modality-specific attention concentration across layers and heads using the Gini coefficient and demonstrate statistically significant redistribution of attention mass between visual and textual tokens in hallucinated responses. We leverage these features to train a logistic regression-based classifier and evaluate it on two medical LMMs across three medical imaging modalities, demonstrating that our method achieves competitive or superior performance against contemporary hallucination detection baselines.
Figures & tables
Figure 1: Image, visual attention heatmap, and response attention graph (left to right). (a) Hallucinated: diffuse visual attention (low visual Gini) and concentrated text attention (high text Gini). (b) Non-hallucinated: focused visual attention (high visual Gini) and diffuse text attention (low text Gini).
Model
Modality
Head ( L,H )
Mean Gini
Diff.
Cohen’s d
H
NH
Med-Gemma
Text
17,2
0.8773
0.8283
+0.0490
+0.719
23,2
0.8703
0.8304
+0.0399
+0.739
17,1
0.8215
0.7771
+0.0444
+0.754
11,3
0.8246
0.7895
+0.0351
+0.724
23,0
0.9048
0.8685
+0.0363
+0.694
Table 1: Mean Gini coefficients for hallucinated (H) and non-hallucinated (NH) responses, their difference (Diff. = H − NH), and Cohen’s d for the five most statistically significant attention heads in each modality. Heads are selected based on p-values comparing the two response groups.
Figure 2: MAGIC framework. Given a multimodal input (image-question pair), (a) attention matrices are extracted across L layers and H heads. (b) Each matrix is decomposed into text and visual components, isolating response-token attention to prompt and image tokens. (c) Modality-specific Gini coefficients ( Gt , Gv ) quantify attention concentration. (d) The coefficients are aggregated into a 2×L×H feature tensor. (e) A lightweight logistic regression probe σ predicts hallucination.
While medical Multimodal Large Language Models (MLLMs) have shown promise in assisting diagnosis, they still frequently generate hallucinated responses that appear linguistically plausible but lack visual evidence. Such hallucinations pose risks to clinical decision-making and necessitate effective detection. Existing introspective detection methods primarily perform uncertainty estimation or logical verification by analyzing model responses conditioned on original or perturbed inputs. However, such external perturbations are often heuristic and context-agnostic, which overlooks the internal cross-modal dependency between generated tokens and related visual tokens during decoding. To address this issue, we propose VIHD, a Visual Intervention-based Hallucination Detection method that leverages targeted visual token masking to calibrate semantic entropy for more effective hallucination detection. VIHD locates visually dominant decoder layers via Visual Dependency Probing (VDP), executes Visual Intervention Decoding (VID) via token masking to calibrate the semantic distribution, and quantifies the resulting Calibrated Semantic Entropy (CSE) as a reliable hallucination signal. Extensive experiments on three medical VQA benchmarks with two medical MLLMs demonstrate that VIHD consistently outperforms state-of-the-art methods, underscoring the importance of fine-grained visual dependency for hallucination detection. The code will be available at https://github.com/Jiayi-Chen-AU/VIHD
Jiayi Chen, Benteng Ma, Zehui Liao +3
Department of Data Science & AI, Faculty of Information Technology, Monash University, Melbourne, VIC 3800, Australia · Hong Kong Polytechnic University, Hong Kong SAR, China · Alfred Health Radiology, Alfred Health, Melbourne, VIC 3004, Australia +1
Although Multimodal Large Language Models have achieved strong performance across a wide range of vision-language tasks, they still suffer from hallucinations, where model outputs become inconsistent with the visual content, textual context, or commonsense knowledge. Existing studies primarily address this problem through coarse-grained detection. However, these approaches often provide insufficient diagnostic information for understanding hallucination types and supporting downstream hallucination mitigation. To bridge this gap, we propose fine-grained hallucination diagnosis for MLLMs, a new unified task that jointly performs hallucination detection, classification, and interpretable explanation generation. We develop an automated data generation pipeline and construct HalluScope-30K, a large-scale diagnostic dataset covering eight sources and five task categories. Based on this dataset, we design a multi-granular joint reward function and train two diagnosis models, HalluScope-4B and HalluScope-8B, which achieve state-of-the-art performance on both the MHALO benchmark and our fine-grained hallucination classification benchmark. Notably, detection and classification are mutually beneficial under joint optimization. Furthermore, diagnosis-driven feedback experiments show that the fine-grained diagnostic explanations produced by our model effectively guide target models to correct their hallucinations, with full diagnosis substantially outperforming all baselines on both Qwen3-VL-8B-Instruct and LLaVA-1.5-7B. Our code, data, and models are available at https://github.com/wkinglin/HalluScope.
Weilin Jin, Mingyu Wang, Wenbo Li +5
Peking University Beijing, China · Joy Future Academy Beijing, China
Large vision-language models (LVLMs) are increasingly used for clinical image understanding, yet they remain vulnerable to hallucinations--producing textual findings or attributes not supported by the image. We present a vision-traceable hallucination detection framework that audits arbitrary LVLM responses via visual evidence grounding, requiring neither modification nor internal access to the hidden states of LVLMs. Given an LVLM response, we extract visually verifiable entities and use a medical-domain-adapted Qwen-VL grounding verifier to localize each entity on the input image. To enhance the robustness of our detection method, we introduce a counterfactual entity perturbation method and estimate visual evidence uncertainty by contrasting factual and counterfactual grounding results. Specifically, we compute an entity-level uncertainty score from the positive confidence, counterfactual confidence, and their grounding overlap for binary hallucination decision-making. Experiments on multiple medical imaging modalities and LVLM backbones demonstrate that our method consistently improves hallucination detection performance over recent baselines, while providing interpretable localization evidence and strong cross-model transferability. Code and dataset are available at https://github.com/Agentic-CliniAI/CounterVHD.
Xiao Song, Haonan Qin, Zhaoxu Zhang +3
School of Intelligent Science and Technology, Nanjing University, Suzhou, China · School of Biomedical Engineering, Nanjing University, Suzhou, China · National Institute of Healthcare Data Science at Nanjing University, Nanjing University, Nanjing, China