Vision-language models (VLMs) have achieved strong performance on tasks such as visual question answering, yet small image resizes can turn correct answers into errors. We investigate whether changes in visual configuration, such as image tiling and token arrangement, contribute to this instability. Across seven checkpoints and four benchmarks, equally small resizes cause more correctness flips when they switch configurations. Surprisingly, in over half of these cases, models answer the question incorrectly but can still read the correct answer when told what to read. Furthermore, attention interventions in LLaVA-NeXT suggest that configuration changes can weaken the use of readable information during answering. We therefore guide models using field cues and their own transcriptions. With annotation assistance, these forms of guidance together correct 97.2% of errors with readable information. These findings show that configuration changes can affect how models use information they can still read.
Figures & tables
Figure 1: A one-pixel height difference changes MiniCPM’s tiling from four local tiles to two and flips the answer. Keeping the visual representations that produced the error fixed, guided reading extracts the target and the model’s own field record restores the original answer. This example is from TextVQA, and all responses are generated by MiniCPM-V-4.5 using greedy decoding.
Task
Qwen3-VL-8B
MiniCPM-V-4.5
LLaVA-NeXT-7B
InternVL3-8B
Qwen2.5-VL-7B
InternVL3.5-8B
Qwen3-VL-32B
Mean
VQAv2
+1.37
+3.12 †
+3.91 †
+5.26
+3.52
+4.21 †
+5.47 †
+3.84
TextVQA
+4.69 †
+4.30 †
+6.84 †
+10.87
+7.03 †
+8.70 †
+3.91 †
+6.62
DocVQA
+6.25 †
+7.81 †
+6.25 †
+16.67
+7.03 †
+3.33
+3.91 †
+7.32
ChartQA
+2.73 †
+4.30 †
+5.08 †
+9.09
+3.52
+0.00
+7.42 †
+4.59
Mean
+3.76
+4.88
+5.52
+10.47
+5.27
+4.06
+5.18
+5.59
Table 1: Excess correctness-flip rate Δ (percentage points, pp). † indicates that the unadjusted paired 95% confidence interval excludes zero. The per-task sample size is n=512 for Qwen3-8B/MiniCPM/LLaVA and n=256 for Qwen2.5/Qwen3-32B. For both InternVL checkpoints, n=95 (VQAv2), 46 (TextVQA), 30 (DocVQA), and 22 (ChartQA).
Figure 2: Bars show four-task mean correctness-flip rates when changing only the image or configuration; error bars indicate 95% confidence intervals clustered by source.
Model
Config.
Flip Δ
Acc. Δ
Tokens Δ
Qwen3
Low
-3.32
-0.34
-3.4
High
-4.05
-0.02
+3.4
MiniCPM
Low
-4.83
-2.66
-37.6
High
-7.03
+3.03
+37.6
LLaVA
Low
-6.49
-2.37
-24.0
High
-6.74
+2.00
+24.0
Table 2: Changes in disagreement and accuracy (pp) and visual token count (%) under common Low/High configurations, relative to native processing.
Figure 3: Target readability in 300 configuration-induced errors. (a) Bars show total errors and the subset with complete target transcription through guided reading for each model. (b) The pie shows the overall distribution; remaining cases are classified by outputs under a common reading prompt.
Configuration
Original Q
Guided reading R
Correct
82.1%
71.8%
Incorrect
38.5%
74.4%
Table 3: Target-attention knockout effects in LLaVA-NeXT, after subtracting non-target effects (%).