cs.CVMar 12, 2026
SaveZeroSense:How Vision matters in Long Context Compression
Organizations: Shenzhen University of Advanced Technology · Shenzhen Technology University
Abstract
Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of Multimodal Large Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense Benchmark to ensure low semantic correlation of testing samples. By eliminating contextual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.
Figures & tables
Figure 1: Decoupled analysis of confounding factors in visual-text compression. (a) Semantic Priors Compensation: The model leverages contextual inference to rectify typographical errors in the source image. (b) Raw Perceptual Bottleneck: In a semantic vacuum, the model exhibits failures on incoherent alphanumeric strings despite high input resolution, revealing the intrinsic ceiling of its raw recognition capability. (c) Performance Decomposition Framework: End-to-end performance is deconstructed into the synergistic interaction between The text preserved by visual-text compression, The model’s raw recognition capability, and the model’s contextual inference capability.
Figure 2: Pipeline for semantically irrelevant text generation and rendering. The system comprises three modules: an Analyzer that extracts key visual attributes (e.g., word counts and bounding-box coordinates) from input images using OCR Tool or ground-truth annotations; a Text Generator that produces semantically irrelevant text with LLM; and a Renderer that synthesizes the final images by combining the extracted visual attributes with the generated text.
| Compression | Omni | Fox | ||
|---|---|---|---|---|
| Our framework | DeepSeek-OCR | Our framework | DeepSeek-OCR | |
| 7.5 | 97.1% | 89.2% | 93.9% | 95.5% |
| 10 | 91.5% | 86.4% | 77.2% | 93.4% |
| 12.5 | 81.5% | 84.2% | 57.1% | 90.6% |
| 15 | 67.5% | 78.8% | 38.9% | 83% |
| 17.5 | 55.7% | 67.4% | 27.4% | 81.3% |
Table 1: Text preservation metrics across the Fox and Omni datasets at varying compression ratios. The table contrasts the accuracy of the DeepSeek-OCR with computed via our decoupled framework. Results highlight the difference between the two frameworks.
Figure 3: (a) The text token distribution density of the Fox and Omni datasets. The data distribution for Fox is highly concentrated around 900 tokens. Conversely, Omni exhibits a prominent long-tail characteristic; a substantial volume of its data is clustered between 300 and 600 tokens, yet the proportion exceeding 2,100 tokens remains non-negligible. (b) and (c) illustrate typical document examples with extremely low and high token counts, respectively.
| Compression | ||||||
|---|---|---|---|---|---|---|
| Omni | Fox | Omni | Fox | Omni | Fox | |
| 7.5 | 77.4% | 95.5% | 40.4% | 71.7% | 37% | 23.8% |
| 10 | 75.7% | 93.4% | 30.4% | 51.9% | 45.3% | 41.5% |
| 12.5 | 67.2% | 90.6% | 23.3% | 36.6% | 43.9% | 54% |
| 15 | 56.8% | 83% | 14.6% | 19.5% | 42.2% | 63.5% |
| 17.5 | 42.4% | 81.3% | 10.7% | 13.3% | 31.7% | 67% |
Table 2: Quantification of semantic priors reliance ( ) under increasing compression ratios. The values indicate the proportion of predictions driven by semantic context rather than visual features. Data is reported for both the Fox and Omni datasets across multiple compression intervals.
| Compression | ||
|---|---|---|
| Omni | Fox | |
| 7.5 | 39.5% | 76.1% |
| 10 | 34% | 68.6% |
| 12.5 | 28.5% | 61% |
| 15 | 22.9% | 53.5% |
| 17.5 | 17.4% | 46% |
Table 3: Decay of raw visual extraction capability ( ) across increasing compression ratios . The table reports the baseline performance isolated from the semantic priors, calibrated via the Zero-Sense benchmark. Results track the decline of for the Fox and Omni datasets from to compression.
| Compression | Precision | Compression | Precision | ||||
|---|---|---|---|---|---|---|---|
| Original | Shuffled | (Drop) | Original | Shuffled | (Drop) | ||
| 2.5 | 97.4% | 97.4% | 0.0% | 12.5 | 88.0% | 77.6% | |
| 5 | 97.0% | 96.2% | 15 | 80.8% | 66.1% | ||
| 7.5 | 95.6% | 92.5% | 17.5 | 70.5% | 51.9% | ||
| 10 | 92.8% | 86.3% | |||||
Table 4: Performance of DeepSeek-OCR under semantic interference with shuffled word orders. Herein, Original denotes natural document images, Shuffled denotes document images with shuffled word orders, and (Drop) indicates the precision drop.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Compression | ||||||
|---|---|---|---|---|---|---|
| Omni | Fox | Omni | Fox | Omni | Fox | |
| 7.5 | 0.2 | 0.04 | 0.5 | 0.13 | -0.3 | -0.09 |
| 10 | 0.21 | 0.07 | 0.58 | 0.22 | -0.37 | -0.15 |
| 12.5 | 0.34 | 0.06 | 0.63 | 0.32 | -0.29 | -0.26 |
| 15 | 0.46 | 0.28 | 0.73 | 0.49 | -0.27 | -0.21 |
| 17.5 | 0.59 | 0.2 | 0.78 | 0.63 | -0.19 | -0.43 |
Supplementary Table 1: Comparison of Edit Distance metrics with and without semantic relevance. Note: The negative sign denotes performance degradation under non-semantic conditions, not the absolute magnitude of the metric difference. This confirms that the model leverages strong semantic priors to perform inference.
Supplementary Figure 1: Comparison between original and ZeroSense document images. Top: Full page views demonstrate that our generation pipeline perfectly preserves the document’s structural context. Bottom: Transcribed zoomed-in regions highlight the semantic decoupling; true semantic priors are systematically replaced with tokens sampled from a low-probability vocabulary subset, isolating the visual layout characteristics.
Supplementary Figure 2: We compared the posterior probability distribution plots of textual content between Fox, Omni, and their corresponding rendered ZeroSense data. Yellow denotes the original data, and blue represents the rendered data. It can be clearly observed that the text in the original dataset exhibits strong semantic priors, whereas the textual content in ZeroSense is highly semantically irrelevant.
Supplementary Figure 3: The figures respectively illustrate the distribution of the number of text tokens for Fox and Omni. Fox is distributed almost entirely within a reasonable range, whereas Omni lies mostly in extreme regions. This aligns with the results presented in Section 5.2 . Owing to the extreme nature of Omni’s rendering scheme, after disentangling semantic priors and OCR recognition accuracy, the difference in text preservation between OmniStyle-ZeroSense and Omni is smaller than that between FoxStyle-ZeroSense and Fox.
Supplementary Figure 4: Illustration of the two-stage shuffling process applied to an original document image: (1) Inter-line Permutation based on Geometric Constraints and (2) Intra-line Word Shuffling with Visual Density Preservation.
Supplementary Figure 5: Comparison between original and shuffled document images. Dislike the original image, the shuffled image exhibits a scrambled word sequence.
More in this topic
VLM Evaluation
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
See all 961 papers