Organizations: State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · Tencent Anti-Cheat Expert, China
Multimodal LLMs can fail to read text embedded in an image's coarse structure (text a human recovers by squinting) and report that no text is present. We name this class Squint Text and study it as a recognition deficiency, motivated by cases from a production moderation pipeline. We introduce SquintBench, comprising 1076 production-derived moderation images, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis. Frozen-weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; identity becomes substantially more accessible when the correct spatial regions are supplied, implicating spatial selection as a major bottleneck. Suppressing high-frequency content also improves reading at a fixed visual-token budget, while visual token count may also contribute. These findings motivate Squint OPSD: a frozen teacher receives a downscaled view and the ground-truth transcription as privileged context, and its token-level distributions supervise a student along on-policy trajectories. At inference, the student reads the original image alone. Squint OPSD reaches 88.61 ANLS on SquintBench, versus 26.50 for the unadapted backbone, 24.44 for the best general-purpose model on the original view, and 68.60 at best for an off-the-shelf model under a downscaled view. These results substantially narrow the gap between available visual evidence and its generated read-out without auxiliary views at inference.
Figures & tables
Figure 1: Squint Text , and why the deployed defence does not remove it. (a) Pipeline images whose embedded string a human reads—directly or by stepping back—while GPT, Gemini and Claude each answer “No text in this image” , on the original and the scaled view alike. (b) Downscaling before review recovers only part of the hidden text for every off-the-shelf model.
Figure 2: Three diagnostics on frozen off-the-shelf weights. (a) Restricting the read-out to glyph-bearing tokens improves glyph-identity prediction, supporting spatial selection as a major bottleneck (per-layer expansion in Appendix A ). (b) At a fixed 64-token budget, suppressing high-frequency content raises squint-text ANLS from 29.2 to 67.0 (per- σ results in Appendix B ). (c) For the same target glyph 搜, adding a 1/8 -scale view of the same image shifts the base model’s phonetic predictions toward visually matching glyphs, indicating stronger visual grounding.
D1
D2
D3
ground-truth SFT
✓
✗
✗
sparse-reward RL
✓
✗
✗
downscaled view at inference
✗
—
—
ground-truth-conditioned teacher
✓
✗
✓
Squint OPSD (view + hint teacher)
✓
✓
✓
Table 1: Candidate solutions under the requirements of § 3.4 : D1 uses original-only inference, D2 uses localization-bearing supervision, and D3 uses a distributional target. A dash denotes a method with no training target. Only Squint OPSD satisfies all three.
Figure 3: Overview of Squint OPSD . The teacher receives the original image, the downscaled view v=g(x) , the task instruction, and the ground-truth output as privileged context. Its token distributions supervise the student along stop-gradient, student-generated trajectories. Gradients flow only through the student logits, and inference uses the original image alone.
SquintBench
general
AI2D
DocVQA
OCRBench
Base
26.50
86.66
95.93
879
Base +1/8 view (two-view)
58.96
52.36
95.37
834
SFT (full, single)
72.47
85.52
64.73
722
DAPO
73.55
82.12
92.31
728
Squint OPSD
88.61
85.33
92.98
833
Table 2: SquintBench performance and general vision-language capability.
benchmark
cond.
Squint OPSD
Qwen3.5- 9B base
claude- opus-5
gemini- 3.1-pro
gpt-6- astra
qwen3-vl- 235b-think
SquintBench
orig.
88.61
26.50
22.26
18.15
8.33
24.44
1/8 only
49.64
38.22
22.41
27.90
8.13
34.37
40×40 only
76.12
68.60
11.07
44.91
6.26
66.88
IllVQA Illusion
WER ↓
67.00
178.10
166.50
190.13
119.60
177.97
IllVQA Filtered
WER ↓
64.87
76.80
83.53
70.00
65.27
91.33
AdvOCR †
orig.
66.7
3.7
3.7
3.7
11.1
3.7
Table 3: Comparison with the base model and off-the-shelf MLLMs on SquintBench and three external hidden-text benchmarks. WER is lower better. The “only” conditions replace the original image with the reduced view, applied before the model preprocessor; further protocol details are in Appendix B.1 .
configuration
SquintBench
teacher input
x (neither)
27.96
x+h (hint only)
41.79
x+v (privileged view only)
54.68
x+v+h (default)
88.61
x+vmis+h (mismatched view) †
78.60
Table 4: Ablations on SquintBench . We vary teacher input, target form, and trainable modules. † The mismatched-view row is paired with a matched-view run under the same setting (84.60), giving a 6.0-point drop.
Figure 4: What changes after training. (a) Example next-token distributions for Base , SFT, and Squint OPSD . (b) Attention concentration on glyph-bearing regions and a causal visual-token embedding knockout. (c) Candidate-distribution structure on 120 matched glyph positions.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Glyph identity is more accessible at the correct locations. Glyph-identity macro-AUC on SquintCtrl-Glyph by encoder block under the three poolings of § 3.2 . random uses the same number of tokens as oracle but remains close to global across depth and script.
script
checkpoint
before
after
Δ
Chinese
Base
0.707
0.700
−0.007
Chinese
Squint OPSD
0.989
0.985
−0.004
Latin
Base
0.555
0.573
+0.018
Latin
Squint OPSD
0.699
0.713
+0.014
Appendix
Table 5: Glyph separability before and after the 2×2 patch merger.
same pipeline
different pipelines
block
Base
Squint OPSD
Base
Squint OPSD
L6
0.964
0.977
0.996
0.996
L12
0.977
0.995
0.995
0.998
L18
0.981
0.995
0.993
0.996
L24
0.967
0.991
0.991
0.995
L27
0.956
0.991
0.984
0.996
Appendix
Table 6: Presence-probe AUC on SquintCtrl-Pair . The controlled columns use the same generation path for both members of each pair.
script
Base
Squint OPSD
Δ
Chinese
0.718
0.975
+0.257
Latin
0.612
0.779
+0.167
Appendix
Table 7: Glyph-identity macro-AUC at L18 before and after training.
band (c/p)
orig.
1/8
40×40
0 – 1/32
91.3%
94.3%
96.5%
1/32 – 1/16
6.5%
4.8%
3.3%
1/16 – 1/8
1.7%
0.87%
0.21%
>1/8
0.5%
0.08%
0.03%
Appendix
Table 8: Share of non-DC spectral energy by band, in cycles per pixel, for the three SquintBench viewing conditions.
Figure 6: Frequency at constant token count. Hidden-text reading ANLS on all 1076 SquintBench images under Gaussian low-pass filtering. The dashed line is the unfiltered input; the visual-token count is unchanged throughout the sweep.
transform
what it does
ANLS
Δ
CLAHE
raises contrast, spectrum intact
25.91
−0.53
bilateral filter
flattens texture, keeps glyph edges
32.22
+5.79
posterise ( 3 levels)
compresses range and texture
37.21
+10.77
band-stop
removes mid, keeps low + high
11.22
−15.22
Appendix
Table 9: Contrast and band-selective controls on SquintBench , against the unfiltered 26.44 ANLS baseline.
teacher configuration
corrective view v
SquintBench
default
1/8
88.61
fixed view
40×40
88.09
fixed view
1/4
78.51
probability mean
1/8,1/4,40×40
83.05
max-entropy selection
1/8,1/4,40×40
80.51
min-entropy selection
1/8,1/4,40×40
74.30
Appendix
Table 10: Teacher-view variants. All Squint OPSD teachers receive (x,v,q,h) ; only the corrective view v or its aggregation is varied. Student inference always uses (x,q) only.
Figure 7: Recoverability versus hiding strength. Reading accuracy for the base model and Squint OPSD as conditioning strength varies, over the same 200 scenes. Stars mark strength levels at which the 95% CI of the paired base-vs- Squint OPSD difference excludes zero (paired bootstrap, 100 resamples); unmarked levels are not significant at this threshold, and 0.7 is borderline and is left unmarked. The method provides its clearest benefit in the partially recoverable regime; the two ends saturate for different reasons.
metric
Base
SFT
Squint OPSD
ρvis , own valid set
0.079 (254)
0.160 (259)
0.243 (164)
ρvis , common set
0.126
0.242
0.286
ρlang , common set
0.555
0.334
0.544
Appendix
Table 11: Candidate-distribution structure before and after matching glyph positions across checkpoints. Parentheses give the number of valid positions.
intervention
Base
Squint OPSD
none (baseline)
26.0
90.5
replace glyph tokens, d=0
19.5
87.0
replace glyph tokens, d=1
16.0
77.0
replace glyph tokens, d=2
15.0
71.5
replace random tokens, d=0
25.0
91.5
replace random tokens, d=1
25.0
91.0
Appendix
Table 12: Reading accuracy after visual-token embedding replacement. Glyph-region embeddings are replaced by the mean embedding of non-glyph tokens from the same image; the random control replaces an equal number of randomly selected background-token embeddings using the same operation. d denotes the number of 3×3 dilations applied to the known glyph mask.
Table 13: Training mixture, 5905 prompts after removing 65 samples with corrupted annotations. DocVQA contributes train-split samples only; its reported evaluation uses a disjoint test split.
hyperparameter
value
optimizer / learning rate
AdamW ( Loshchilov & Hutter, 2019 ) , 1×10−5
schedule / warmup
constant / 10 steps
weight decay / grad clip
0.01 / 1.0
prompt batch
56 prompts per step
rollouts per prompt
n=8 (448 sequences/step)
inner epochs
1
Appendix
Table 14: Main Squint OPSD optimization and rollout settings.
trainable modules
SquintBench
AI2D
DocVQA
OCRBench
LM only; ViT frozen
27.29
86.66
50.39
741
ViT + LM
72.47
85.52
64.73
722
Appendix
Table 15: Effect of visual adaptation under three-epoch supervised fine-tuning.
Figure 8: Two examples from each of the four SquintCtrl subsets. The embedded strings are carried in the images’ coarse structure: they are hard to notice at full size but become legible when the page is squinted at or viewed at reduced scale. SquintCtrl-Pair : a text-present image and its text-free counterpart, regenerated through the same ControlNet branch with an all-black conditioning image. SquintCtrl-Glyph : conditioning on the closed vocabularies of ten Chinese glyphs and ten Latin letters. SquintCtrl-Strength : the same type of content under weak versus strong conditioning. SquintCtrl-Read : arbitrary open-vocabulary strings.
Figure 9: SquintBench deployment examples that commercial models miss. Four production-derived moderation images, shown together with the answers of two frontier commercial MLLMs and Squint OPSD to the transcription instruction. Both commercial models report that no text is present on the original image, whereas Squint OPSD reads out the embedded promotional strings; the same strings become legible to a human when the page is squinted at or viewed at reduced scale. These examples come from the production-derived portion of SquintBench and contain no personal data.
Despite the rapid advancements in Multimodal Large Language Models (MLLMs), a critical question regarding their visual grounding mechanism remains unanswered: do these models genuinely read'' text embedded in images, or do they merely rely on parametric shortcuts in the text prompt? In this work, we diagnose this issue by introducing the Visualized-Question (VQ) setting, where text queries are rendered directly onto images to structurally mandate visual engagement. Our diagnostic experiments on Qwen2.5-VL reveal a startling capability-utilization gap: despite possessing strong OCR capabilities, models suffer a performance degradation of up to 12.7% in the VQ setting, exposing a deep-seated modality laziness.'' To bridge this gap, we propose SimpleOCR, a plug-and-play training strategy that imposes a structural constraint on the learning process. By transforming training samples into the VQ format with randomized styles, SimpleOCR effectively invalidates text-based shortcuts, compelling the model to activate and optimize its visual text extraction pathways. Empirically, SimpleOCR yields robust gains without architectural modifications. On four representative OOD benchmarks, it surpasses the base model by 5.4% and GRPO based on original images by 2.7%, while exhibiting extreme data efficiency, achieving superior performance with 30x fewer samples (8.5K) than recent RL-based methods. Furthermore, its plug-and-play nature allows seamless integration with advanced RL strategies like NoisyRollout to yield complementary improvements. Code is available at https://github.com/aiming-lab/SimpleOCR.
Yibo Peng, Peng Xia, Ding Zhong +6
UNC-Chapel Hill · Carnegie Mellon University · University of Michigan +1
We introduce MMTR-Bench, a benchmark designed to evaluate the intrinsic ability of Multimodal Large Language Models (MLLMs) to reconstruct masked text directly from visual context. Unlike conventional question-answering tasks, MMTR-Bench eliminates explicit prompts, requiring models to recover masked text from single- or multi-page inputs across real-world domains such as documents and webpages. This design isolates the reconstruction task from instruction-following abilities, enabling a direct assessment of a model's layout understanding, visual grounding, and knowledge integration. MMTR-Bench comprises 2,771 test samples spanning multiple languages and varying target lengths. To account for this diversity, we propose a level-aware evaluation protocol. Experiments on representative MLLMs show that the benchmark poses a significant challenge, especially for sentence- and paragraph-level reconstruction. The homepage is available at https://mmtr-bench-dataset.github.io/MMTR-Bench/.
Multimodal large language models (MLLMs) can process text presented as images, yet they often perform worse than when the same content is provided as textual tokens. We systematically diagnose this "modality gap" by evaluating seven MLLMs across seven benchmarks in five input modes, spanning both synthetically rendered text and realistic document images from arXiv PDFs to Wikipedia pages. We find that the gap is highly sensitive to rendering choices such as font and resolution, and that natural document images often exhibit much smaller gaps, suggesting the performance difference partly reflects evaluation artifacts rather than fundamental limitations. Through a grounded-theory error analysis of over 4,000 examples, we identify the primary cause: image input alone suppresses reasoning effort, with models producing 5--19x shorter outputs that skip step-by-step computation or reasoning. The reluctance to reason, not a failure of perception or knowledge retrieval, drives the performance gap, particularly on tasks requiring multi-step reasoning. We show that a simple, lightweight on-policy self-distillation method by fine-tuning models on their own text-mode reasoning traces paired with image inputs closes this gap, raising image-mode accuracy to match or exceed text-mode performance with over 50% improvement, and the gains transfer to unseen benchmarks without catastrophic forgetting. Overall, our results and analyses provide a systematic understanding of the modality gap and suggest a practical path toward improving visual text understanding in multimodal language models.
Kaiser Sun, Xiaochuang Yuan, Hongjun Liu +4
Johns Hopkins University · Amazon · New York University +1