Despite the rapid advancements in Multimodal Large Language Models (MLLMs), a critical question regarding their visual grounding mechanism remains unanswered: do these models genuinely read'' text embedded in images, or do they merely rely on parametric shortcuts in the text prompt? In this work, we diagnose this issue by introducing the Visualized-Question (VQ) setting, where text queries are rendered directly onto images to structurally mandate visual engagement. Our diagnostic experiments on Qwen2.5-VL reveal a startling capability-utilization gap: despite possessing strong OCR capabilities, models suffer a performance degradation of up to 12.7% in the VQ setting, exposing a deep-seated modality laziness.'' To bridge this gap, we propose SimpleOCR, a plug-and-play training strategy that imposes a structural constraint on the learning process. By transforming training samples into the VQ format with randomized styles, SimpleOCR effectively invalidates text-based shortcuts, compelling the model to activate and optimize its visual text extraction pathways. Empirically, SimpleOCR yields robust gains without architectural modifications. On four representative OOD benchmarks, it surpasses the base model by 5.4% and GRPO based on original images by 2.7%, while exhibiting extreme data efficiency, achieving superior performance with 30x fewer samples (8.5K) than recent RL-based methods. Furthermore, its plug-and-play nature allows seamless integration with advanced RL strategies like NoisyRollout to yield complementary improvements. Code is available at https://github.com/aiming-lab/SimpleOCR.
Figures & tables
Figure 1: (a) Visualized-Question (VQ) Format. We render the question text into the image as the only question source, removing text-channel shortcuts and requiring visual reading. (b) Capability–Utilization Gap. On Qwen2.5-VL-7B, performance drops markedly under VQ versus standard inputs, indicating that OCR capability is not reliably utilized during reasoning.
Figure 2: The SimpleOCR framework. During training, all inputs are transformed into visual question contexts Cvq , where question text is rendered onto images. This structurally eliminates text-based shortcuts and forces visual OCR engagement. At inference, models trained this way demonstrate robust performance on standard inputs Corig . The method integrates seamlessly as an augmentation branch in existing RL frameworks.
In-Domain
Out-of-Distribution
Method
Data Size
Geo3K
MMK12
Avg.
MathVerse
MathVision
MathVista
HallusionBench
Avg.
Open-source Baselines (SFT / General)
InternVL-2.5-8B-Instruct* Chen et al. (2024)
-
-
-
-
39.5
19.7
64.4
67.3
47.7
LLaVA-OneVision-7B* Li et al. (2024a)
-
-
-
-
26.2
-
63.2
48.4
-
Kimi-VL-16B* Team et al. (2025b)
-
-
-
-
44.9
21.4
68.7
66.2
50.3
Mulberry-7B* Yao et al. (2024)
-
-
-
-
-
-
63.1
-
-
Table 1: Performance on mathematical reasoning and visual perception benchmarks. Models marked with “*” are cited from original papers. Bold and underlined numbers indicate the best and second-best performance, respectively. Data sizes for SFT and RL are respectively marked in blue and red .
Figure 3: Performance on OCR-intensive benchmarks. SimpleOCR demonstrates superior performance, achieving 81.6% on ChartQA and 69.1% on HallusionBench.
In-Domain
Out-of-Distribution
Method Configuration
Geo3K
MMK12
Avg.
MathVerse
MathVision
MathVista
Hallusion
Avg.
Qwen2.5-VL-3B-Instruct
26.0
45.9
36.0
32.5
18.2
50.8
59.1
40.2
+ GRPO (baseline) ( n=6 )
35.1
51.2
43.2
36.6
18.5
53.0
58.6
41.7
+ SimpleOCR ( n=6 )
36.1
53.6
44.9
40.4
20.1
53.3
61.7
43.9
+ HybridRollout ( n1=3 , n2=3 )
34.8
53.6
44.2
41.4
20.6
57.3
58.7
44.5
Qwen2.5-VL-7B-Instruct
37.6
53.6
45.6
43.9
23.4
64.2
68.2
49.9
Table 2: Analysis of Integration & Scalability: SimpleOCR integrates seamlessly with HybridRollout across model scales. The combination yields consistent gains, particularly on the 3B model, validating that SimpleOCR (focused on text reading) and HybridRollout (focused on visual robustness) are orthogonal and complementary.
Figure 4: The “U-Shaped” Optimization Conflict. We report the average performance across four representative OOD benchmarks. The mixed strategy (50% VQ) results in a net performance loss, illustrating that contradictory modality signals hinder generalization.
In-Domain
Out-of-Distribution
Rendering Strategy
Geo3K
MMK12
M-Verse
M-Vision
M-Vista
WeMath
Fixed style
41.4
61.3
46.9
23.4
65.9
61.6
Random style
43.4
62.3
47.7
24.9
68.7
64.0
Table 3: Ablation on rendering style. Randomization prevents overfitting to specific visual patterns. (Note: “Random style” corresponds to the full SimpleOCR method used in main results.)
Figure 5: Left: On MathVista, the GRPO baseline is misled by hallucinated semantic priors, while SimpleOCR correctly identifies material properties. Right: On ChartQA, the baseline relies on superficial keyword spotting, whereas SimpleOCR performs holistic visual analysis. Blue : correct grounding; red : heuristic errors.
In-Domain
Out-of-Distribution
Configuration
Geo3K
MMK12
Avg.
M-Verse
M-Vision
M-Vista
WeMath
Hallusion
Avg.
n=3
40.4
62.5
51.5
46.2
24.0
67.5
60.5
70.4
53.7
n=6
43.4
62.3
52.9
47.7
24.9
68.7
64.0
69.1
54.9
n=9
41.4
63.0
52.2
47.4
24.6
66.4
61.6
67.9
53.6
Table 4: Impact of Group Sampling Size n . We analyze the effect of the number of generations per prompt during GRPO training.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Source
Domain
Size
Geometry3K
( Lu et al., 2021 )
Plane Geometry
2,100
MMK12
( Meng et al., 2025 )
K-12 Mathematics
6,400
Total
-
Mixed
8,500
Appendix
Table 5: Training Data Statistics. We combine geometry-focused and general K-12 math datasets to construct a diverse training corpus.
VQ Ratio
MathVerse
MathVision
MathVista
WeMath
Avg.
Standard (0% VQ)
46.4
22.5
66.9
65.3
50.3
Mixed (25% VQ)
47.8
22.9
67.2
62.8
50.2
Mixed (50% VQ)
46.2
23.7
65.0
62.4
49.3
Mixed (75% VQ)
48.0
23.9
65.7
62.2
50.0
SimpleOCR (100% VQ)
47.7
24.9
68.7
64.0
51.3
Appendix
Table 6: Impact of VQ Training Ratio (Detailed Breakdown). We report the performance on four reasoning-heavy OOD benchmarks. The mixed strategy (50% VQ) consistently underperforms or stagnates compared to the baseline (Avg. 49.3 vs 50.3), supporting the hypothesis of optimization conflict. Only the full VQ strategy (SimpleOCR) achieves robust generalization gains (Avg. 51.3).
Method
ChartQA
HallusionBench
InfoVQA
Base Model
79.8
68.2
79.7
GRPO (Original images)
79.5
68.9
80.1
GRPO + SimpleOCR
81.6
69.1
80.5
Appendix
Table 7: Performance on OCR-Intensive Benchmarks. Exact numbers corresponding to Figure 3 .
Parameter
Configuration
Model Base
Qwen2.5-VL-Instruct
Vision Encoder
Frozen
Global Batch Size
128
Rollout Batch Size
512
Rollout Temperature
1.0
Learning Rate
1×10−6
Appendix
Table 8: Summary of hyperparameter configurations.
Multimodal LLMs can fail to read text embedded in an image's coarse structure (text a human recovers by squinting) and report that no text is present. We name this class Squint Text and study it as a recognition deficiency, motivated by cases from a production moderation pipeline. We introduce SquintBench, comprising 1076 production-derived moderation images, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis. Frozen-weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; identity becomes substantially more accessible when the correct spatial regions are supplied, implicating spatial selection as a major bottleneck. Suppressing high-frequency content also improves reading at a fixed visual-token budget, while visual token count may also contribute. These findings motivate Squint OPSD: a frozen teacher receives a downscaled view and the ground-truth transcription as privileged context, and its token-level distributions supervise a student along on-policy trajectories. At inference, the student reads the original image alone. Squint OPSD reaches 88.61 ANLS on SquintBench, versus 26.50 for the unadapted backbone, 24.44 for the best general-purpose model on the original view, and 68.60 at best for an off-the-shelf model under a downscaled view. These results substantially narrow the gap between available visual evidence and its generated read-out without auxiliary views at inference.
Chen Peng, Shan Huang, Jianbo Huang +3
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · Tencent Anti-Cheat Expert, China
Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across words, layout, fields, charts, and visual correspondences. Existing evaluations often conflate extraction with reasoning and rarely test whether models follow the required reasoning direction: applying visible rules, abstracting hidden regularities, or recovering missing premises. We introduce OCR-MetaReasoning, a controlled single-image benchmark that treats deduction, induction, and abduction as distinct directions and separates final-answer correctness from reasoning-process compliance. The benchmark contains 1,500 verified samples in a balanced 3×5 taxonomy crossing three reasoning types with five OCR-object categories, along with reference reasoning steps, automatic answer scoring, the Meta-Reasoning Macro Score (MRMS), and the Reasoning Process Compliance Score (RPCS). Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation. The code is available at https://github.com/gengxuli/OCR-MetaReasoning.
Gengxu Li, Yuan Wu, Yi Chang
School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · 3International Center of Future Science, Jilin University
Vision-language models (VLMs) can read text from images, but where does this optical character recognition (OCR) information enter the language processing stream? We investigate the OCR routing mechanism across three architecture families (Qwen3-VL, Phi-4, InternVL3.5) using causal interventions. By computing activation differences between original images and text-inpainted versions, we identify architecture-specific OCR bottlenecks whose dominant location depends on the vision-language integration strategy: DeepStack models (Qwen) show peak sensitivity at mid-depth (about 50%) for scene text, while single-stage projection models (Phi-4, InternVL) peak at early layers (6-25%), though the exact layer of maximum effect varies across datasets. The OCR signal is remarkably low-dimensional: PC1 captures up to 72.9% of variance. Crucially, principal component analysis (PCA) directions learned on one dataset transfer to others, demonstrating shared text-processing pathways. Surprisingly, in models with modular OCR circuits (notably Qwen3-VL-4B), OCR removal can improve counting performance (up to +6.9 percentage points), suggesting OCR interferes with other visual processing in sufficiently modular architectures.