Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of Multimodal Large Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense Benchmark to ensure low semantic correlation of testing samples. By eliminating contextual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.
Figures & tables
Figure 1: Decoupled analysis of confounding factors in visual-text compression. (a) Semantic Priors Compensation: The model leverages contextual inference to rectify typographical errors in the source image. (b) Raw Perceptual Bottleneck: In a semantic vacuum, the model exhibits failures on incoherent alphanumeric strings despite high input resolution, revealing the intrinsic ceiling of its raw recognition capability. (c) Performance Decomposition Framework: End-to-end performance is deconstructed into the synergistic interaction between The text preserved by visual-text compression, The model’s raw recognition capability, and the model’s contextual inference capability.
Figure 2: Pipeline for semantically irrelevant text generation and rendering. The system comprises three modules: an Analyzer that extracts key visual attributes (e.g., word counts and bounding-box coordinates) from input images using OCR Tool or ground-truth annotations; a Text Generator that produces semantically irrelevant text with LLM; and a Renderer that synthesizes the final images by combining the extracted visual attributes with the generated text.
Compression
Omni
Fox
Our framework
DeepSeek-OCR
Our framework
DeepSeek-OCR
7.5 ×
97.1%
89.2%
93.9%
95.5%
10 ×
91.5%
86.4%
77.2%
93.4%
12.5 ×
81.5%
84.2%
57.1%
90.6%
15 ×
67.5%
78.8%
38.9%
83%
17.5 ×
55.7%
67.4%
27.4%
81.3%
Table 1: Text preservation metrics across the Fox and Omni datasets at varying compression ratios. The table contrasts the accuracy of the DeepSeek-OCR with Kquality computed via our decoupled framework. Results highlight the difference between the two frameworks.
Figure 3: (a) The text token distribution density of the Fox and Omni datasets. The data distribution for Fox is highly concentrated around 900 tokens. Conversely, Omni exhibits a prominent long-tail characteristic; a substantial volume of its data is clustered between 300 and 600 tokens, yet the proportion exceeding 2,100 tokens remains non-negligible. (b) and (c) illustrate typical document examples with extremely low and high token counts, respectively.
Compression
Fprior+OCRraw⋅Kquality
OCRraw⋅Kquality
Fprior
Omni
Fox
Omni
Fox
Omni
Fox
7.5 ×
77.4%
95.5%
40.4%
71.7%
37%
23.8%
10 ×
75.7%
93.4%
30.4%
51.9%
45.3%
41.5%
12.5 ×
67.2%
90.6%
23.3%
36.6%
43.9%
54%
15 ×
56.8%
83%
14.6%
19.5%
42.2%
63.5%
17.5 ×
42.4%
81.3%
10.7%
13.3%
31.7%
67%
Table 2: Quantification of semantic priors reliance ( Fprior ) under increasing compression ratios. The values indicate the proportion of predictions driven by semantic context rather than visual features. Data is reported for both the Fox and Omni datasets across multiple compression intervals.
Compression
OCRraw
Omni
Fox
7.5 ×
39.5%
76.1%
10 ×
34%
68.6%
12.5 ×
28.5%
61%
15 ×
22.9%
53.5%
17.5 ×
17.4%
46%
Table 3: Decay of raw visual extraction capability ( OCRraw ) across increasing compression ratios ρ(θ) . The table reports the baseline performance isolated from the semantic priors, calibrated via the Zero-Sense benchmark. Results track the decline of OCRraw for the Fox and Omni datasets from 7.5× to 17.5× compression.
Compression
Precision
Compression
Precision
Original
Shuffled
Δ (Drop)
Original
Shuffled
Δ (Drop)
2.5 ×
97.4%
97.4%
0.0%
12.5 ×
88.0%
77.6%
−10.4%
5 ×
97.0%
96.2%
−0.8%
15 ×
80.8%
66.1%
−14.7%
7.5 ×
95.6%
92.5%
−3.1%
17.5 ×
70.5%
51.9%
−18.6%
10 ×
92.8%
86.3%
−6.5%
Table 4: Performance of DeepSeek-OCR under semantic interference with shuffled word orders. Herein, Original denotes natural document images, Shuffled denotes document images with shuffled word orders, and Δ (Drop) indicates the precision drop.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Compression
Fprior+OCRraw⋅Kquality
OCRraw⋅Kquality
Fprior
Omni
Fox
Omni
Fox
Omni
Fox
7.5 ×
0.2
0.04
0.5
0.13
-0.3
-0.09
10 ×
0.21
0.07
0.58
0.22
-0.37
-0.15
12.5 ×
0.34
0.06
0.63
0.32
-0.29
-0.26
15 ×
0.46
0.28
0.73
0.49
-0.27
-0.21
17.5 ×
0.59
0.2
0.78
0.63
-0.19
-0.43
Appendix
Supplementary Table 1: Comparison of Edit Distance metrics with and without semantic relevance. Note: The negative sign denotes performance degradation under non-semantic conditions, not the absolute magnitude of the metric difference. This confirms that the model leverages strong semantic priors to perform inference.
Supplementary Figure 1: Comparison between original and ZeroSense document images. Top: Full page views demonstrate that our generation pipeline perfectly preserves the document’s structural context. Bottom: Transcribed zoomed-in regions highlight the semantic decoupling; true semantic priors are systematically replaced with tokens sampled from a low-probability vocabulary subset, isolating the visual layout characteristics.
Supplementary Figure 2: We compared the posterior probability distribution plots of textual content between Fox, Omni, and their corresponding rendered ZeroSense data. Yellow denotes the original data, and blue represents the rendered data. It can be clearly observed that the text in the original dataset exhibits strong semantic priors, whereas the textual content in ZeroSense is highly semantically irrelevant.
Supplementary Figure 3: The figures respectively illustrate the distribution of the number of text tokens for Fox and Omni. Fox is distributed almost entirely within a reasonable range, whereas Omni lies mostly in extreme regions. This aligns with the results presented in Section 5.2 . Owing to the extreme nature of Omni’s rendering scheme, after disentangling semantic priors and OCR recognition accuracy, the difference in text preservation between OmniStyle-ZeroSense and Omni is smaller than that between FoxStyle-ZeroSense and Fox.
Supplementary Figure 4: Illustration of the two-stage shuffling process applied to an original document image: (1) Inter-line Permutation based on Geometric Constraints and (2) Intra-line Word Shuffling with Visual Density Preservation.
Supplementary Figure 5: Comparison between original and shuffled document images. Dislike the original image, the shuffled image exhibits a scrambled word sequence.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of Multimodal Large Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense Benchmark to ensure low semantic correlation of testing samples. By eliminating textual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.
Yonghan Gao, Zehong Chen, Lijian Xu +3
Shenzhen University of Advanced Technology · Shenzhen Technology University
Long-context reasoning in large language models incurs substantial computation and memory costs. Visual text compression (VTC) reduces input length by rendering text as images, but fixed-resolution rendering creates a compression-performance trade-off: low DPI saves tokens at the expense of legibility, whereas high DPI spends tokens on irrelevant content. We introduce FocusVTC, which breaks this trade-off through adaptive resolution while preserving general multimodal capabilities. It combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing reasoning. We construct 29.4K high-quality Reasoning-Evidence Localization (REL) chain-of-thought examples (REL-CoT) that link reasoning traces to page indices and bounding boxes. Multi-resolution REL supervised fine-tuning (REL-SFT) teaches the model to localize relevant regions, and Group Relative Policy Optimization learns when to enhance resolution and how to use the resulting observations, without a separate continual-pretraining stage. At 72 DPI on RULER v1, FocusVTC scores 87.4 at 2.9× input compression, including tool observations, versus 57.5 for Glyph at 3.0× input compression. It surpasses its text-input backbone on LongBench (56.40 versus 55.86), improves the MRCR macro-average by 13.91 points, and achieves a 51.19 macro-average on VTCBench. The MRCR latency evaluation also shows a 2.79× online end-to-end speedup over Text. General multimodal capabilities are preserved, with MMMU increasing from 65.12 to 66.73 and MME from 2424.02 to 2457.62.
FangZhi Zhong, Xuerui Qiu, Yuqi Pan +4
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +1
Vision Language Models (VLMs) offer the exciting possibility of processing text as rendered images, bypassing the need for tokenizing the text into long token sequences. Since VLM image encoders map fixed-size images to a fixed number of visual tokens, varying rendering resolution provides a fine-grained compression knob. However, accuracy deteriorates quickly as compression increases: characters shrink below the vision encoder's effective resolution, making them indistinguishable. To address this, we propose LensVLM, an inference framework and post-training recipe that enables VLMs to scan compressed images, then selectively expand only the relevant images to their uncompressed form via learned tools. Building on Qwen3.5-9B-Base, LensVLM maintains accuracy comparable to the full-text upper bound at 4.3× effective compression and outperforms retrieval-based, text- and visual-compression baselines up to 10.1× effective compression across seven text QA benchmarks. LensVLM also generalizes to multimodal document and code understanding tasks, with the accuracy gain over baselines growing as compression increases. Our analysis validates this approach: training makes visual compression robust to rendering choices, and as compression grows the model increasingly relies on expanded content rather than unreliable visual reading. The analysis also yields practical tool-choice guidance: text expansion is preferable for rendered text, while high-resolution image expansion suits native documents whose layout cues carry task-relevant information.