Every evaluation run of a vision-language model (VLM) already produces per-question artifacts (predictions, judged correctness, metadata) that leaderboards reduce to a single score and then discard. We reuse those artifacts to audit the benchmark itself, and add an audit layer to VLMEvalKit: six detectors that run as post-processing over saved outputs, plus one additional text-only rerun with visual inputs removed, emitting a per-question report. The layer reports per-question findings together with aggregate audit statistics; for the headline quantities, it attaches permutation-null excess where applicable, interval estimates for proportions, and sensitivity to the model pool. Auditing MMStar, a benchmark curated specifically so that the image is indispensable, with three models, we find zero aggregate accuracy gain from the image on 20.2% of its questions. An item-matched permutation null accounts for 17.0%, leaving a +3.2 pp excess (corrected permutation p = 0.001). This excess is concentrated in the mathematics and science categories and is absent from coarse perception, a localization consistent with knowledge substituting for visual evidence; confirming that mechanism would require item-level review. A second detector flags 134 questions on which every model that answered in parseable form picked the same non-gold option, and these candidates peak in the perception categories, whereas the zero-gain excess is largest in mathematics and science & technology. Replacing the LLM judge with exact string matching changes no conclusion, and every scoring discrepancy between the two traces to one model whose free-form answers a parser cannot read. On a binary yes/no benchmark the detectors measurably fall out of calibration, a scope boundary that we measure and report. The audit layer, the prediction artifacts and the generated reports accompany this submission.
Figures & tables
Figure 1: The audit pipeline in five stages. (1) Inputs: the full model runs with predictions and judged hits, one additional text-only blind run with images removed, and the dataset itself with questions, options and gold labels. (2) All frames are aligned by dataset question ID. (3) Six detectors process the aligned frame. (4) The audit report lists per-question findings with severity levels and summary statistics, including permutation-null excess values and a reproducibility manifest. (5) The report yields actionable insights for improving benchmarks and evaluation practice. The blind run is the only step that requires inference, and it runs without images.
Detector
Signal
Pathology flagged
Judge-dep.
visual_dependency
full vs. blind accuracy
no-gain-from-image items
yes
consensus_error
majority vote vs. label
label-conflict candidates
no
fleiss_kappa_agreement
inter-model κ
unstable / ambiguous items
no
correctness_agreement
judged-correctness profile
difficulty / discrimination
yes
answer_options_distribution
label-position histogram
position bias
no
distractor_similarity
pairwise option similarity
near-duplicate options
no
Table 1: Detector taxonomy. “Judge-dep.” marks detectors whose output depends on the answer-extraction protocol.
Figure 2: The zero-gain signal on MMStar. (a) Model-averaged full against blind accuracy per question (judged arm, jittered; three models give the 4×4 grid). Questions on the diagonal gain nothing from the image. (b) The observed 20.2% share against 999 item-shuffled permutations: the null centres at 17.0%, leaving a +3.2 pp excess that no permutation reaches ( p=0.001 ). (c) The no-gain set is heterogeneous: on 164 of 303 questions every model is wrong with and without the image, so the conservative reading is zero aggregate accuracy gain from the image for this model pool , and genuinely text-answerable items are only one possible component.
Model pool
Observed (%)
Null mean (%)
Excess (pp)
p
All three models
20.2
17.0
+3.2
0.001
without InternVL3-8B
27.9
24.1
+3.7
0.001
without Ristretto-3B
25.5
21.9
+3.6
0.001
without gpt-5-nano
34.4
26.7
+7.7
0.001
Table 2: Zero-gain share against the permutation null on MMStar (999 permutations, one-sided, (g+1)/(N+1) correction). The excess is markedly more pool-stable than the raw share: raw spread 14.2 pp, excess spread 4.6 pp.
Figure 3: Per-category audit of MMStar (250 items per category), sorted by zero-gain excess. (a) The excess concentrates in mathematics and science & technology and is absent from coarse perception, a pattern consistent with greater scope for non-visual knowledge in the former categories. (b) Unanimous label-conflict candidates peak in the perception categories. The two detectors localize different candidate failure modes.
Figure 4: Judge dependence of the audit on MMStar. (a) Category shares under the gpt-4o-mini judge and under exact letter matching, on identical predictions: every share moves by at most 2.1 pp and no conclusion changes. (b) The disagreement has a single source: for the two open models the judge and the letter parser score every item identically, while gpt-5-nano, whose answers are unparseable on 10% of items, accounts for all of it.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: HallusionBench (951 yes/no items, same three models). (a) Ranking items by the continuous visual gain does not recover the benchmark’s own design labels (AUC 0.49 against 0.50 for chance). (b) The one directional signal that survives is consistent with the benchmark’s design: zero-gain excess is approximately 2.2 times larger on visual-supplement items than on visual-dependent items.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image tokens only degrades model performance very slightly on a widely used hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs. Our analysis spans multiple levels of granularity, spanning global visual degradation, localized occlusion, question reformulation, answer-space expansion, and decision-level analyses beyond standard accuracy. We further complement these behavioral results with a layer-wise analysis of vision-token geometry. Throughout the experiments, we find that although VLMs do incorporate visual input, their predictions are less sensitive to the loss of fine-grained visual evidence that standard accuracy should have suggested. Even when the final prediction remains unchanged, the model's internal support for the correct answer may already be weakened. We further complement a representation-level analysis, which shows increasing similarity among visual tokens in deeper layers, providing a possible explanation for our findings. Together, these results suggest that current benchmarks are not sufficient to reliably evaluate fine-grained visual grounding in VLMs.
Zixuan Lan, Luzhe Sun, Matthew R. Walter +1
University of Chicago · Toyota Technological Institute at Chicago · Stony Brook University
Large Vision-Language Models (VLMs) are increasingly used to evaluate outputs of other models, for image-to-text (I2T) tasks such as visual question answering, and text-to-image (T2I) generation tasks. Despite this growing reliance, the reliability of these Evaluator VLMs remains under explored. In this work, we systematically evaluate the reliability of Evaluator VLMs across both I2T and T2I tasks. We introduce targeted perturbations that degrade output quality along key error dimensions, including object hallucinations, spatial reasoning, factual grounding, and visual fidelity. These perturbations test whether Evaluator VLMs can reliably account for these quality degrading errors in their evaluations. Using a comprehensive benchmark of over 4000 perturbed instances spanning 40 perturbation dimensions, we evaluate 4 prominent VLMs using single-answer scoring, pairwise comparison, and reference-guided paradigms. Our findings reveal that current VLM evaluators exhibit substantial blind spots: they often fail to detect perturbed outputs - in some cases exceeding 50%, struggle particularly with fine-grained compositional and spatial errors, and are often insensitive to hallucinated content that contradicts the input image. Pairwise comparison proves more reliable, though failure rates persist. These results highlight the unreliable nature of current Evaluator VLMs and urge caution in their deployment for benchmarking and development decisions. Code and data have been made publicly available.
Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand +1
♠Nilekani Centre at AI4Bharat ♢Indian Institute of Technology Madras · ♠Nilekani Centre at AI4Bharat · ♣BITS Pilani, Hyderabad
Vision-language models (VLMs) increasingly act as judges that pick the best of several generated images, so their choices decide what users see. Such judges are usually validated by score agreement with human ratings, not by the images they return. We audit VLM judges as decision-makers: on 300 culturally situated prompts, we compare the returned image with human ratings the judge never sees and with random choice from the same candidates, and repeat every decision with the candidates reordered. A 4B-parameter judge barely beats random and falls short of a CLIP similarity baseline. It picks the first image shown in 49% of calls (chance: 28%), and reordering changes its choice on 60% of prompts. For this judge, agreement across orders is informative: decisions that survive reordering are much better than random, whereas agreement with a weaker second judge keeps the wrong ones. An 8B judge shows almost no position bias and outperforms CLIP, yet for it the same filter mostly discards good decisions. Agreement helps only when it targets the judge's failure mode, so filters must be re-audited whenever the judge changes. The 4B judge's slight rise in stereotype ratings is no longer detectable after aggregating across orders or with the larger judge.