MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Hejun Dong; Junbo Niu; Bin Wang; Weijun Zeng; Wentao Zhang; Conghui He

cs.CVMar 23, 2026

MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Authors: Hejun Dong, Junbo Niu, Bin Wang, Weijun Zeng, Wentao Zhang, Conghui He

Organizations: Shanghai Artificial Intelligence Laboratory, OpenDataLab · Peking University

Abstract

Optical character recognition (OCR) has evolved from line-level transcription to structured document parsing, requiring models to recover long-form sequences containing layout, tables, and formulas. Despite recent advances in vision-language models, most existing systems rely on autoregressive decoding, which introduces sequential latency and amplifies error propagation in long documents. In this work, we revisit document OCR from an inverse rendering perspective, arguing that left-to-right causal generation is an artifact of serialization rather than an intrinsic property of the task. Motivated by this insight, we propose MinerU-Diffusion, a unified diffusion-based framework that replaces autoregressive sequential decoding with parallel diffusion denoising under visual conditioning. MinerU-Diffusion employs a block-wise diffusion decoder and an uncertainty-driven curriculum learning strategy to enable stable training and efficient long-sequence inference. Extensive experiments demonstrate that MinerU-Diffusion consistently improves robustness while achieving up to 3.2x faster decoding compared to autoregressive baselines. Evaluations on the proposed Semantic Shuffle benchmark further confirm its reduced dependence on linguistic priors and stronger visual OCR capability.

Figures & tables

Panel of Figure 1 — Figure 1: (a) The confidence threshold controls decoding parallelism in MinerU-Diffusion. Compared to MinerU2.5, our method achieves up to 3.26× speedup. (b) MinerU-Diffusion maintains a strong accuracy–efficiency trade-off, achieving 2.12× speedup with 99.9% and 3.01× speedup with 98.8% relative accuracy. (c) Diffusion decoding progressively reconstructs structured text from masked tokens under visual conditioning: black tokens are confirmed, red tokens are being updated, and yellow tokens remain masked, enabling parallel generation with global consistency, in contrast to autoregressive left-to-right decoding.

Figure 2: Overview of the document OCR inverse rendering process via different decoding methods.. The model maps a 2D document image to a 1D token sequence for decoding through autoregressive and diffusion-based methods.

Figure 3: Training of MinerU-Diffusion. Left: the target token sequence is randomly masked to form a partially observed input, and the model predicts only the masked positions under visual and prompt conditioning. Right: the structured block-attention mask used during training, where tokens attend bidirectionally within each block and causally to all preceding blocks, enabling parallel diffusion refinement within blocks while preserving coarse autoregressive structure across blocks.

Type	Methods	Params.	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\textit{TEDS}}$ $\uparrow$	Table ${}_{\textit{TEDS-S}}$ $\uparrow$	Reading Order $\downarrow$	GT Layout
Pipeline	Mineru2-pipeline [ 43 ]	-	75.51	0.209	76.55	70.90	79.11	0.225	$\times$
Pipeline	PP-StructureV3 [ 7 ]	-	86.73	0.073	85.79	81.68	89.48	0.073	$\times$
AR	Qwen2.5-VL-72B [ 40 ]	72B	87.02	0.094	88.27	82.15	86.22	0.102	$\times$
	Gemini-2.5 Pro [ 6 ]	-	88.03	0.075	85.82	85.71	90.29	0.097	$\times$
	MinerU2-VLM [ 43 ]	0.9B	85.56	0.078	80.95	83.54	87.66	0.086	$\times$
	Nanonets-OCR-s [ 25 ]	3B	85.59	0.093	85.90	80.14	85.57	0.108	$\times$

Table 1: Comprehensive evaluation of document parsing on OmniDocBench v1.5.

\uparrow

denotes higher is better,

\downarrow

denotes lower is better.

Type	Methods	Slides	Academic Papers	Book	Textbook	Exam Papers	Magazine	Newspaper	Notes	Financial Report	GT Layout
Pipeline	MinerU2-pipeline [ 43 ]	0.4244	0.0230	0.2628	0.1224	0.0822	0.3950	0.0736	0.2603	0.0411	$\times$
Pipeline	PP-StructureV3 [ 7 ]	0.0794	0.0236	0.0415	0.1107	0.0945	0.0722	0.0617	0.1236	0.0181	$\times$
AR	Qwen2.5-VL-72B [ 40 ]	0.0422	0.0801	0.0586	0.1146	0.0681	0.0964	0.2380	0.1232	0.0264	$\times$
	Gemini-2.5 Pro [ 6 ]	0.0326	0.0182	0.0694	0.1618	0.0937	0.0161	0.1347	0.1169	0.0169	$\times$
	MinerU2-VLM [ 43 ]	0.0745	0.0104	0.0357	0.1276	0.0698	0.0652	0.1831	0.0803	0.0236	$\times$
	Nanonets-OCR-s [ 25 ]	0.0551	0.0578	0.0606	0.0931	0.0834	0.0917	0.1965	0.1606	0.0395	$\times$

Table 2: Breakdown of document parsing on OmniDocBench v1.5 by page type, reported in text edit distance (

\downarrow

Type	Method	CC-OCR		OCRBench v2		UniMER-Test
Type	Method	TEDS $\uparrow$	TEDS-S $\uparrow$	TEDS $\uparrow$	TEDS-S $\uparrow$	CPE $\uparrow$	HWE $\uparrow$	SCE $\uparrow$	SPE $\uparrow$
AR	InternVL3.5-241B [ 45 ]	62.87	69.52	79.50	85.81	91.7	93.2	95.1	97.8
	Qwen2.5-VL-72B [ 40 ]	81.22	86.48	81.33	86.58	88.9	91.8	95.5	96.2
	GPT-4o [ 13 ]	66.98	79.04	70.51	79.55	82.7	85.9	87.8	96.7
	dots.ocr [ 16 ]	75.42	81.65	82.04	86.27	86.8	90.5	94.7	97.5
	MinerU2.5 [ 29 ]	79.76	85.16	87.13	90.62	96.6	94.4	96.4	98.4

Table 3: Comprehensive recognition results on CC-OCR [ 51 ] , OCRBench v2 [ 9 ] , and UniMER-Test [ 42 ] .

\uparrow

denotes higher is better.

Figure 4: Threshold sensitivity analysis of TPF, TPS, and accuracy. TPF denotes tokens per forward, and TPS refers to throughput measured on an NVIDIA H200 GPU with a batch size of 1.

Figure 5: Visualizing the accuracy-throughput trade-off of different models across different OCR tasks under the w/ GT Layout setting.

Type	Method	Inference		Score (w/ GT Layout)					Score (w/o GT Layout)
Type	Method	TPF	TPS	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\text{TEDS}}$ $\uparrow$	Table ${}_{\text{TEDS-S}}$ $\uparrow$	Overall $\uparrow$	Reading Order $\downarrow$
dLM	Static (step=6)	5.33	91.56	88.31	0.064	85.52	85.81	9120	82.73	0.077
	Dynamic ( $\tau=0.97$ )	5.18	98.32	93.34	0.028	91.65	91.17	94.92	89.04	0.057
	Static (step=32)	1.00	21.86	93.02	0.030	91.10	90.95	94.73	89.76	0.060
AR	MinerU2.5 [ 29 ]	1.00	51.46	93.44	0.025	91.98	90.84	95.10	90.67	0.044
AR	PaddleOCR-VL [ 7 ]	1.00	40.77	93.91	0.021	92.13	91.70	95.42	92.79	0.043

Table 4: Comparison of different inference strategies and model types in terms of efficiency (TPF, TPS) and document parsing performance.

Figure 6: Comparison of training dynamics across different curriculum strategies. The two-stage framework achieves smoother optimization and higher final accuracy compared to single-stage baselines.

Setting	Score (w/ GT Layout)					Score (w/o GT Layout)
Setting	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\text{TEDS}}$ $\uparrow$	Table ${}_{\text{TEDS-S}}$ $\uparrow$	Overall $\uparrow$	Reading Order $\downarrow$
Stage 1	92.89	0.032	91.58	90.28	94.93	86.13	0.056
Stage 2	89.33	0.038	88.50	83.29	88.93	35.71	0.472
Stage 1 + Stage 2	93.37	0.028	91.92	91.00	94.86	88.94	0.059

Table 5: Comparison of different training stages under GT and non-GT layout settings.

Figure 7: Semantic Shuffle benchmark results across distortion levels.

		Stage-0		Stage-1	Stage-2
		a	b
Vision	Max Resolution	$2048\times 28\times 28$	$4096\times 28\times 28$	$2048\times 28\times 28$	$2048\times 28\times 28$
Vision	#Tokens per Image	$4\sim 2048$	$4\sim 4096$	$4\sim 2048$	$4\sim 2048$
Data	Dataset	LLaVA-Pretrain	LLaVA-NeXT-Data	Layout&OCR	Layout&OCR
Data	#Samples	550K	739K	6.9M	630K
Model	Trainable	MLP Adaptor	All	All	All

Table 6: Training setup and hyperparameters in three training stages.

Figure 8: Training dynamics across the four stages in our training recipe: Stage 0a, Stage 0b, Stage 1, and Stage 2. For each stage, we report both the loss and the gradient norm.

Figure 9: Training curves on a table-only subset.

Figure 10: A very short table where over-provisioned Full-Attn repeats empty rows.

Figure 11: A medium-length table where matched Full-Attn remains stable but over-provisioned Full-Attn repeats.

Figure 12: A long table where short Full-Attn truncates and long Full-Attn repeats tail rows.

Panel of Figure 13 — Figure 13: Complete recognition examples. Left: original page image. Middle: predicted layout. Right: rendered parsed output.

Panel of Figure 14 — Figure 14: Additional complete recognition examples in the same format.

Panel of Figure 15 — Figure 15: Diffusion decoding examples for layout generation.

Panel of Figure 16 — Figure 16: Diffusion decoding examples for text recognition.

Panel of Figure 17 — Figure 17: Diffusion decoding examples for table recognition.

Panel of Figure 18 — Figure 18: Diffusion decoding examples for formula recognition.

Explore similar work

DODO: Discrete OCR Diffusion Models
Feb 18, 2026Sean Man, Gilad Deutch, Roy Ganz +4Optical Character Recognition Diffusion Language Model Decoding
Technion - Israel Institute of Technology, Haifa, Israel. · Amazon Web Services.
Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding
Sep 22, 2026Dohyun Kim, Sungjun Han, Hyungguk Kim +4Autoregressive Diffusion Speculative Decoding
Trillion Labs · Korea University · Seoul National University
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
Jul 6, 2026Gengluo Li, Xingyu Wan, Shangpin Peng +20Vision-Language Models Efficient VLM Inference
Institute of Information Engineering, Chinese Academy of Sciences · Large Language Model Department, Tencent · Nankai University

Feb 18, 2026J/K move · Enter open · S save

DODO: Discrete OCR Diffusion Models

Sean Man, Gilad Deutch, Roy Ganz, Roi Ronen+3

Technion - Israel Institute of Technology, Haifa, Israel. · Amazon Web Services.

Optical Character Recognition Diffusion Language Model Decoding Vision-Language Models Blockwise Diffusion Language Models

Open paper AskPDF arXiv

Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern Vision-Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via diffusion models. However, we show that existing masked diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.

Abstract

Explore similar work

DODO: Discrete OCR Diffusion Models

Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better