Organizations: College of Computer Science and Artificial Intelligence, Fudan University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory · ByteDance, LarkAI · Shanghai Key Lab of Intelligent Information Processing
Compared with typical vision-language tasks, document parsing places stronger demands on fine-grained visual perception. Existing vision-language model (VLM)-based parsing approaches rely on globally compressed visual tokens, where fine-grained details are entangled within a single representation and repeatedly accessed during decoding. However, we observe that the visual evidence for each prediction is typically localized and conditioned on the current decoding state, whereas such representations must be accessed in full at every decoding step, resulting in inefficient computation. To address this mismatch, we formulate perception as state-conditioned visual evidence retrieval (SCVER) during autoregressive decoding. The model operates on a compact global representation for coarse structure and retrieves a small set of relevant high-resolution regions conditioned on the current token state. This coarse-to-fine design enables on-demand access to fine-grained visual cues, relieving globally shared representations from encoding all fine-grained details. We further find that learning such state-conditioned retrieval in VLMs is challenging and unstable. To stabilize this process, we introduce a Spatially-Guided Learning Objective (SGLO) to guide the retrieval process. Experiments on document parsing benchmarks show that SCVER improves robustness under reduced input resolution and achieves a better accuracy-efficiency trade-off, demonstrating the effectiveness of on-demand visual evidence retrieval for fine-grained perception.
Figures & tables
Figure 1: Accuracy vs. Input Size on UniMER-CPE Wang et al. (2024a) . Circle size indicates the average GFLOPs per generated token. SCVER reduces the reliance on high-resolution inputs, achieving a more favorable accuracy–efficiency trade-off compared to the dynamic-resolution MinerU2.5 Niu et al. (2025) and the fixed-resolution Dolphin Feng et al. (2025) .
Figure 2: (a) Most VLMs only use final layer output. (b) LLaMA-Adapter Zhang et al. (2024) injects multi-scale visual features into the decoder. (c) VIVL Cao et al. (2024) uses a small set of fixed queries to gather multi-level features, but these sparse visual samples stay constant throughout decoding, offering limited information. (d)Taking MinerU2.5 Niu et al. (2025) as an example, SCVER dynamically supplies pre-downsampling visual information required at each decoding step.
Figure 3: Illustration of State-Conditioned Visual Evidence Retrieval. SCVER efficiently retrieves fine-grained visual information from pre-downsampling features conditioned on the current decoding state.
Method
R
Param
Formula CDM↑
Table TEDS↑
Text 1-Edit↑
Page-Level
SPE
SCE
CPE
HWE
PubTabNet
FinTabNet
DocLaynet
B-MOD
OmniDoc
General VLMs
GPT-4o
1.0
-
96.1
88.2
83.3
85.9
76.6
83.9
91.7
0.318
75.0
Qwen3-VL-235B
1.0
235B
98.4
96.2
97.4
94.2
85.4
84.9
92.3
0.321
89.7
Specialized VLMs
DeepSeek-OCR-2
1.0
3B
95.5
77.1
92.0
81.6
-
-
92.1
81.2
90.2
Table 1: Performance of different methods under varying image resolutions. R denotes the down-scaling ratio applied to each image dimension (width and height), corresponding to R2 of the original image area. Values are approximate, as some models require padding or rounding to match implementation constraints. Bold numbers indicate the best results in each column.
Model
Params
Encoder Type
Resolution
Dolphin1.5 ( 2025 )
394M
Swin
Fixed
MinerU2.5 ( 2025 )
1.2B
ViT
Dynamic
MonkeyOCR-Pro ( 2025 )
3B
ViT
Dynamic
Table 2: Base models used in our experiments.
Figure 4: SGLO with two shared-weight hidden-state streams. The frozen base stream produces detached attention-centroid targets, while the SCVER stream predicts the corresponding reference points.
Figure 5: Accuracy and efficiency analysis. The x-axis shows input image size (0.4, 0.6, 0.8, 1.0) corresponding to per-token GFLOPs. Lines on the left y-axis indicate model performance and bars on the right y-axis indicate tokens per second. Integrating SCVER achieves a better accuracy-efficiency trade-off compared to the baseline (MinerU2.5).
HWE
PubTabNet
B-MOD
FLOPs/Token
MinerU2.5-Pro
95.3
90.1
85.6
1.15e+10
+VIVL
94.1( -1.2 )
90.7( +0.7 )
91.7( +6.1 )
1.19e+10
+HVFA
96.2( +0.9 )
91.4( +1.3 )
92.1( +6.5 )
1.34e+10
+SCVER (Ours)
97.4( +2.1 )
95.2( +5.1 )
95.1( +9.5 )
1.21e+10
Table 3: Comparison of different methods.
Method
Tok
DocVQA
TextVQA
InfoVQA
Qwen2.5-VL
576
92.7
80.9
71.6
DocVLM
320
92.3( -0.4 )
79.7( -1.2 )
67.3( -4.3 )
SCVER (Ours)
320
90.4( -2.3 )
78.0( -2.9 )
69.4( -2.2 )
DocVLM
576
94.2( +1.5 )
83.8( +2.9 )
72.8( +1.2 )
SCVER (Ours)
576
93.5( +0.8 )
82.9( +2.0 )
73.1( +1.5 )
Table 4: Results on document VQA benchmarks under different visual token budgets. Tok denotes the maximum number of visual tokens after image resizing.
Figure 6: Training loss curves with and without the Spatially-Guided Learning Objective (SGLO). λ denotes the weight of the SGLO term.
Figure 7: Visualization of state-conditioned retrieval points. Multi-head retrieval points (after applying offsets to the reference point) and their attention weights. Each circle represents a retrieval point, with color indicating its attention weight.
SCVER
SGLO
SCE
PubTabNet
B-Mod
95.1
85.9
82.7
✓
95.4
88.1
85.9
✓
✓
97.6
91.9
94.0
Table 5: Ablation study on the effect of SGLO.
Figure 8: Effect of retrieval module application frequency on accuracy. The x-axis indicates the interval of decoder layers at which sampling module is inserted.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Issue
Original
Standardized
Bracket
{
\lbrace
Subsup
a^1_2
a_2^1
Prime
a ′
a^{\prime}
Fraction
\over
\frac
Space
\array{l c}
\array{lc}
Appendix
Table 6: Examples of LaTeX standardization for various symbols and expressions.
Method
Textual
Image
Table
Equation
Page Margins
Full Page
D4LA
DocLayout-YOLO Zhao et al. (2024)
90.8
62.6
89.8
91.1
-
88.5
PP-StructureV3 Cui et al. (2025b)
90.0
67.9
89.7
92.1
79.1
88.3
MinerU2.5 Niu et al. (2025)
94.6
72.8
91.4
91.0
84.2
91.4
MinerU2.5+SCVER
95.8
74.9
93.2
92.9
88.5
93.2
DocLaynet
Appendix
Table 7: Comparison of layout analysis performance across different methods and content types on multiple benchmark datasets.
Method
Formula CDM↑
Table TEDS↑
Text Edit↓
SPE
SCE
CPE
HWE
PubTabNet
FinTabNet
DocLaynet
B-MOD
Dolphin-1.5
97.8
95.1
87.7
-
85.9
87.4
0.017
0.304
Dolphin-1.5*
97.9
95.3
87.4
-
86.1
87.0
0.014
0.244
Dolphin-1.5 + SCVER
98.4
97.6
90.2
-
91.9
92.4
0.011
0.068
MonkeyOCR-Pro
97.6
94.9
91.4
92.2
87.4
86.4
0.020
0.204
MonkeyOCR-Pro*
97.4
95.3
91.7
92.0
88.0
86.2
0.018
0.154
Appendix
Table 8: Analysis of potential data leakage to verify whether the observed performance gains are caused by training data overlap. The symbol * indicates that the model is further fine-tuned on the same data used to train SCVER.
Configuration
Performance
Efficiency
H
P
F
Formula
Table
Text
FLOPs
8
2
1
95.6
87.6
0.028
9.58e+08
16
4
1
97.3
91.9
0.011
9.90e+08
24
8
2
97.6
92.4
0.011
1.27e+09
Appendix
Table 9: Performance and efficiency under different configurations. H , P , and F correspond to the number of heads, sampling points, and high-scale feature maps used in the SCVER computation.
λ
Formula
Table
Text
0
96.4
95.6
0.218
0.01
96.7
95.3
0.166
0.1 (selected)
98.7
95.9
0.028
1.0
98.2
95.7
0.017
Appendix
Table 10: Performance under different λ values on SGLO.
Figure 10: Comparison between the base model and the model equipped with the SCVER module on text recognition. The base model tends to rely on the decoder’s language prior, while the model with SCVER follows the visual evidence more faithfully.
Figure 11: Impact of the SCVER strategy on formula recognition. This figure compares model outputs with and without the SCVER module across various types of mathematical expressions, including simple formulas, matrices, degraded formulas, and formulas containing Chi- nese text. The highlighted regions indicate the positions where the model without SCVER fails to preserve structural correctness or symbol fidelity. These comparisons demonstrate that SCVER effectively mitigates such errors by maintaining fine-grained structural details and improving symbol-level accuracy, resulting in outputs that more closely align with the original expressions.
Figure 12: Impact of the SCVER strategy on table recognition.This figure compares model outputs with and without the SCVER module on a complex table containing multi-row, multi-column, and merged-cell structures. The highlighted regions indicate structural and content errors produced by the model without SCVER, including misaligned cell boundaries, incorrect row–column associations, and incorrectly recognized text. These comparisons demonstrate that SCVER effectively preserves table-structure integrity and improves cell-level content fidelity, enabling the model to generate outputs that more accurately reflect the original tabular layout and data.