Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification
Authors: Tao Huang, Rui Wang, Xiaofei Liu, Yi Qin, Li Duan, Liping Jing
Organizations: State Key Laboratory of Advanced Rail Autonomous Operation, China · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, China · School of Computer Science and Technology, Beijing Jiaotong University, China · School of Automation and Intelligence, Beijing Jiaotong University, China · Beijing Key Laboratory of Security and Privacy in Intelligent Transportation, China
%Large vision-language models (LVLMs) have shown substantial advances in multimodal understanding and generation. However, when presented with incompetent or adversarial inputs, they frequently produce unreliable or even harmful content, such as fact hallucinations or dangerous instructions. This misalignment with human expectations, referred to as misbehaviors of LVLMs, raises serious concerns for deployment in critical applications. These misbehaviors are found to stem from epistemic uncertainty, specifically either conflicting internal knowledge or the absence of supporting information. However, existing uncertainty quantification methods, which typically capture only overall epistemic uncertainty, have shown limited effectiveness in identifying such issues. To address this gap, we propose Evidential Uncertainty Quantification (EUQ), a fine-grained method that captures both information conflict and ignorance for effective detection of LVLM misbehaviors. In particular, we interpret features from the model output head as either supporting (positive) or opposing (negative) evidence. Leveraging Evidence Theory, we model and aggregate this evidence to quantify internal conflict and knowledge gaps within a single forward pass. %We extensively evaluate our method across four categories of misbehavior, including hallucinations, jailbreaks, adversarial vulnerabilities, and out-of-distribution (OOD) failures, using state-of-the-art LVLMs, and find that EUQ consistently outperforms strong baselines, showing that hallucinations correspond to high internal conflict and OOD failures to high ignorance. Furthermore, layer-wise evidential uncertainty dynamics analysis helps interpret the evolution of internal representations from a new perspective. The source code is available at https://github.com/HT86159/EUQ.
Figures & tables
Figure 1: The example shows how our evidential uncertainty, visualized as a token-level heatmap over the Chain-of-Thought (CoT) ( Kojima et al., 2022 ) traces, shows that the misbehavior may stem from internal conflict and a lack of knowledge.
Figure 2: The overall framework of the proposed method applies basic belief assignment to the pre-logits feature to obtain evidence weights. These weights are then decomposed into positive and negative components, which are fused to estimate the final uncertainties that can detect different types of misbehaviors, respectively.
Scenarios
Methods
Size
Question Type
Hallucination
( Li et al., 2023b )
1000
Multiple-choice
Hallucination
( Wu et al., 2024a )
1000
Multiple-choice
Jailbreak
( Gong et al., 2025 )
200
Open-ended
Jailbreak
( Li et al., 2024b )
200
Open-ended
Jailbreak
( Qi et al., 2024 )
600
Open-ended
Jailbreak
( Goh et al., 2021 )
1800
Multiple-choice
Table 1: Overview of datasets and evaluation types.
Figure 3: Layer-wise changes of evidential uncertainty and analysis of conflict vs. ignorance across four dataset types using Intern. Results for other models are provided in Appendix A.8 .
Figure 4: Density distributions of CF , IG , and entropy for each type of misbehavior in Intern, comparing the target misbehavior against others. Results for other models are provided in Appendix A.8 .
Table 6
Models → Method ↓
DeepSeek
Qwen
Intern
MoF
Average
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
Hallucination datas from ( Li et al., 2023b ) and ( Wu et al., 2024a ) .
SC
0.660
0.734
0.640
0.815
0.696
0.883
0.500
0.758
0.624
0.798
SE
0.517
0.649
0.501
0.554
0.775
0.582
0.722
0.510
0.629
0.574
PE
0.771
0.574
0.742
0.741
0.755
0.634
0.701
0.619
0.742
0.642
LN-PE
0.758
0.570
0.574
0.576
0.755
0.634
0.702
0.619
0.697
0.600
Table 4: The AUROC and AUPR of our methods and baselines on DeepSeek-VL2 (DeepSeek), Qwen2.5-VL (Qwen), InternVL2.5 (Intern), and MoF-Models (MoF) in adversarial, OOD, hallucination, and jailbreak settings. Best and next-best results are marked in bold and underlined , respectively.
Figure 5: Ablation study on temperature (left) and model scale (right) across all datasets using Intern.
Method
Model Inference
SC
SE
PE
LN-PE
HiddenDetect
CF
IG
Time (s)
9.6×10−2
8.9×10−1
9.0×10−1
3.1×10−6
6.1×10−6
2.0×10−2
9.1×10−4
4.5×10−3
AUROC
—
0.626
0.624
0.701
0.704
0.707
0.812
0.783
Table 6: Comparison of AUROC and average runtime per example across Intern.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Representative examples of four types of misbehaviors.
Figure 7: Analysis of conflict and ignorance, as quantified measures of evidential uncertainty, across four dataset types using four LVLMs.
Figure 8: Density distribution comparison between our method (conflict and ignorance) and predictive entropy across various misbehavior groupings on four LVLMs.
Table 7: Ablation study on the impact of model scale using DeepSeek. We separately report AUROC and AUPR for clarity. The best and second-best results are highlighted in bold and underlined , respectively.
Table 8: Ablation study on the impact of model scale using Qwen. We separately report AUROC and AUPR for clarity. The best and second-best results are highlighted in bold and underlined , respectively.
Method
DeepSeek
Qwen
Intern
MoF
POPE
RBench
POPE
RBench
POPE
RBench
POPE
RBench
CF
0.776
0.518
0.908
0.501
0.860
0.593
0.999
0.672
IG
0.962
0.596
0.586
0.576
0.691
0.840
0.999
0.673
Appendix
Table 9: AUROC and AUPR of our method under the impact of hallucination type using four LVLMs. We separately report AUROC and AUPR for clarity. The best results are highlighted in bold .
Method
DeepSeek
Qwen
Intern
MoF
Open-ended
Yes-No
Open-ended
Yes-No
Open-ended
Yes-No
Open-ended
Yes-No
CF
0.720
0.966
0.920
0.997
0.871
0.623
0.702
0.995
IG
0.587
0.789
0.862
0.872
0.921
0.637
0.708
0.751
Appendix
Table 10: AUROC and AUPR of our method under the impact of jailbreak type using four LVLMs. We separately report AUROC and AUPR for clarity. The best results are highlighted in bold .
Scenario
Dataset
MC Dropout
Deep Ensemble
EDL
CF
IG
Adversarial
MNIST
0.927
0.933
0.892
0.935
0.701
GTSRB
0.970
0.980
0.912
0.962
0.894
OOD
MNIST
0.937
0.985
0.802
0.972
0.995
GTSRB
0.907
0.969
0.802
0.944
0.995
Appendix
Table 11: AUROC performance comparison on adversarial and OOD detection tasks.
Figure 9: Layer-wise changes of evidential uncertainty and analysis of conflict vs. ignorance across four dataset types using Qwen-2.5-VL-72B.
Misbehaviors → Method ↓
Hallucination
Jailbreak
Adversarial
OOD
Average
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
AUROC
AUPR
SC
0.701
0.874
0.566
0.833
0.712
0.640
0.674
0.884
0.663
0.808
SE
0.609
0.856
0.543
0.818
0.502
0.670
0.602
0.786
0.564
0.781
PE
0.783
0.558
0.618
0.759
0.766
0.692
0.714
0.727
0.720
0.684
LN-PE
0.783
0.553
0.645
0.628
0.655
0.658
0.717
0.720
0.700
0.639
HiddenDetect
0.622
0.659
0.854
0.762
0.823
0.662
0.613
0.719
0.728
0.701
Appendix
Table 12: The AUROC and AUPR of our methods and baselines on Qwen-VL-72B, in adversarial, OOD, hallucination, and jailbreak settings. Best and next-best results are marked in bold and underlined , respectively.
Method
Accuracy
Precision
Recall
F1 Score
CF
0.851
0.929
0.885
0.907
IG
0.859
0.921
0.906
0.914
CF & IG
0.835
0.929
0.866
0.896
CF ∣ IG
0.873
0.922
0.924
0.923
Appendix
Table 13: Performance comparison of CF, IG, and their fusion strategies for hallucination detection.
Large vision--language models (LVLMs) demonstrate strong multimodal reasoning capabilities but remain prone to hallucination, where model predictions are not grounded in visual evidence. Existing black-box hallucination detection methods estimate uncertainty through a single consistency metric, implicitly assuming that model uncertainty can be adequately characterized by a single measure. However, hallucinations exhibit diverse manifestations of uncertainty across different behavioral probes, making a single measure insufficient to characterize their underlying behavior. We propose Unique Hallucination Pattern (UHP) Detection, a fully black-box framework that models hallucination as a structured uncertainty pattern defined by two axes: perturbation modality (image vs.\ text) and logical polarity (a statement vs.\ its negation). Their intersection produces four complementary consistency groups that capture distinct manifestations of model uncertainty, from which both within-group and between-group features are extracted to train a lightweight classifier. Through comprehensive experiments on AMBER and PhD across three LVLMs, UHP Detection consistently outperforms prior black-box and white-box baselines, with improvements of up to +18.72% AUC-ROC and +20.07% AUC-PR over the strongest black-box methods. Extensive ablation studies demonstrate that each consistency group contributes complementary information and that their combination forms a structured hallucination pattern. Furthermore, cross-dataset evaluation shows that this learned pattern generalizes across benchmarks, indicating that hallucination behavior reflects a model-specific consistency pattern. Code is publicly available at https://github.com/amirezzati/uhpdet.
Amir Mohammad Ezzati, Kiyan Rezaee, Bardiya Kariminia +4
Sharif University of Technology · Michigan State University · Shahid Beheshti University +2
Large Vision-Language Models (LVLMs) frequently hallucinate, limiting their safe deployment in real-world applications. Existing LLM self-evaluation methods rely on a model's ability to estimate the correctness of its own outputs, which can improve deployment reliability; however, they depend heavily on language priors and are therefore ill-suited for evaluating vision-conditioned predictions. We propose VAUQ, a vision-aware uncertainty quantification framework for LVLM self-evaluation that explicitly measures how strongly a model's output depends on visual evidence. VAUQ introduces the Image-Information Score (IS), which captures the reduction in predictive uncertainty attributable to visual input, and an unsupervised core-region masking strategy that amplifies the influence of salient regions. Combining predictive entropy with this core-masked IS yields a training-free scoring function that reliably reflects answer correctness. Comprehensive experiments show that VAUQ consistently outperforms existing self-evaluation methods across multiple datasets.
Seongheon Park, Changdae Oh, Hyeong Kyu Choi +2
University of Wisconsin-Madison · Nanyang Technological University
Vision-language models (VLMs) are increasingly deployed in high-stakes settings where reliable uncertainty quantification (UQ) is as important as predictive accuracy. Extended reasoning via chain-of-thought (CoT) prompting or reasoning-trained models has become ubiquitous in modern VLM pipelines, yet its effect on UQ reliability remains poorly understood. Our results show that reasoning tends to degrade the quality of many uncertainty estimates, even when it improves task accuracy. We identify implicit answer conditioning as the primary mechanism: as reasoning traces converge on a conclusion before the final answer is generated, token probabilities increasingly reflect consistency with the model's own reasoning trace rather than uncertainty about correctness. In effect, the model becomes overconfident in its answer. In contrast, agreement-based consistency remains robust and often improves under reasoning, making it a practical choice for uncertainty estimation in reasoning-enabled VLMs.
Robert Welch, Emir Konuk, Kevin Smith
KTH Royal Institute of Technology, Stockholm, Sweden · Science for Life Laboratory, Stockholm, Sweden