Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern Vision-Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via diffusion models. However, we show that existing masked diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.
Figures & tables
Figure 1: DODO: High-throughput parallel generation. Unlike autoregressive models constrained to a strict left-to-right sequence, DODO generates text across the entire canvas simultaneously (with same color) based on visual confidence. In this example, it resolves 148 tokens in just 20 forward passes ( ≈7 tokens/step on average). Simultaneously generated tokens are adaptively distributed across steps.
Figure 2: Semantically flexible vs . semantically rigid vision–language tasks. Left: Image captioning admits multiple, semantically equivalent descriptions of the same image. Different decoding trajectories can converge to distinct but equally valid captions, and lexical or structural variations are naturally absorbed. Right: OCR requires a single, exact transcription determined by the image. Even minimal local deviations, such as an incorrect token choice or boundary, render the output incorrect. As a result, conditioned on the image, OCR exhibits extremely low output variability, which makes it a natural candidate for parallel decoding, but also a demanding setting in which errors cannot be compensated by alternative phrasings or later corrections.
Figure 3: Conditional independence assumption. (Top) In open-ended tasks, ambiguity between valid options risks sampling incoherent mixtures. (Bottom) In OCR, the strong visual signal resolves this ambiguity, enabling conflict-free parallel decoding.
Figure 4: Full vs . block diffusion . In standard full diffusion ( left ), MDM sampling is applied globally to the entire sequence. In contrast, block diffusion ( right ) restricts parallel sampling to discrete windows, processing blocks sequentially from left to right.
Edit Distance ↓
Method
Size
OmniDoc
Fox
Specialized OCR
dots.ocr
3B
0.032
0.034
DeepSeek-OCR
3.4B
0.049
0.100
MinerU 2.0 VLM
0.9B
0.045
-
MonkeyOCR-pro
3B
0.058
0.084
Table 1: DODO results. OCR performance on the English subset of OmniDocBench and Fox-Pages. Bold numbers indicate the best value across each model type (MDM and AR).
Figure 5: Inference throughput comparison. DODO leverages parallel decoding, block-causal attention and KV-caching to achieve ≈104 tokens/sec, a 5× speedup over the autoregressive baseline.
Train Config
Max Len
Inf. Block
Edit Dist.
TPS
Vanilla
Oracle
-
0.100
8.77
Vanilla
8192
-
0.834
3.19
Vanilla
8192
32
0.951
7.47
Block
8192
32
0.067
29.5
Blk Causal
8192
32
0.069
103.7
Table 2: Impact of block structure. Vanilla MDM fails even with Oracle length; block-wise training is essential.
KV-Cache
Train
Test
Block
Edit Dist.
TPS
No KV-Cache
Bidir
Bidir
32
0.067
29.5
Bidir
Bidir
128
0.068
43.1
Bidir
Bidir
256
0.057
42.8
Bidir
Bidir
512
0.111
35.4
Bidir
Bidir
1024
0.192
26.5
Table 3: Block size and caching. Approx. KV-Cache collapses; block-causal training enables exact caching with 5× speedup.
Figure 6: Decoding Efficiency. DODO requires fewer than 0.1 steps per token, compressing inference by an order of magnitude vs. autoregressive models.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1 : Visualization of the attention structure. Full bidirectional attention allows prior blocks to attend to the current block (green hatched), meaning their internal representations dynamically adapt during the forward pass. Block-causal masking prevents prior blocks from attending to the current block. This ensures the history representations remain invariant, enabling exact KV-caching for faster inference.
Figure B2 : Edit Distance vs Speed for different sampling strategies. Confidence Thresholding (Green) achieves the optimal balance at p=0.99 , maintaining high accuracy while offering adaptive speedups.
Model
Pages
0–128
128–512
512–1024
1024–4096
4096+
Overall
Specialized OCR
MinerU
275
0.328
0.166
0.027
0.032
0.030
0.041
Autoregressive VLMs
Qwen 2.5 VL 7B
277
0.032
0.241
0.074
0.096
0.185
0.136
Qwen 2.5 VL 3B
276
0.032
0.156
0.062
0.159
0.363
0.237
Diffusion VLMs
Appendix
Table 4: Edit distance by document length (tokens). Lower is better.
Model
Academic
Book
News
Magazine
Textbook
Exam
PPT
Overall
Specialized OCR
MinerU
0.015
0.049
0.047
0.043
0.052
0.045
0.261
0.041
Autoregressive VLMs
Qwen 2.5 VL 7B
0.098
0.189
0.306
0.119
0.067
0.115
0.020
0.136
Qwen 2.5 VL 3B
0.180
0.216
0.629
0.189
0.157
0.138
0.019
0.237
Diffusion VLMs
Appendix
Table 5: Edit distance by document type. Lower is better.
Figure D3 : Qualitative Results. Each row displays a different document from OmniDocBench. Left: Input document image. Center: DODO’s generated transcript rendered to PDF. Right: Visualization of the decoding process (heatmap of token commitment order). DODO successfully recovers complex layouts, including multi-column text, tables, and mathematical formulas, while maintaining high parallel efficiency.