MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Hejun Dong; Junbo Niu; Bin Wang; Weijun Zeng; Wentao Zhang; Conghui He

cs.CVMar 23, 2026

MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding

Authors: Hejun Dong, Junbo Niu, Bin Wang, Weijun Zeng, Wentao Zhang, Conghui He

Organizations: Shanghai Artificial Intelligence Laboratory, OpenDataLab · Peking University

Abstract

Optical character recognition (OCR) has evolved from line-level transcription to structured document parsing, requiring models to recover long-form sequences containing layout, tables, and formulas. Despite recent advances in vision-language models, most existing systems rely on autoregressive decoding, which introduces sequential latency and amplifies error propagation in long documents. In this work, we revisit document OCR from an inverse rendering perspective, arguing that left-to-right causal generation is an artifact of serialization rather than an intrinsic property of the task. Motivated by this insight, we propose MinerU-Diffusion, a unified diffusion-based framework that replaces autoregressive sequential decoding with parallel diffusion denoising under visual conditioning. MinerU-Diffusion employs a block-wise diffusion decoder and an uncertainty-driven curriculum learning strategy to enable stable training and efficient long-sequence inference. Extensive experiments demonstrate that MinerU-Diffusion consistently improves robustness while achieving up to 3.2x faster decoding compared to autoregressive baselines. Evaluations on the proposed Semantic Shuffle benchmark further confirm its reduced dependence on linguistic priors and stronger visual OCR capability.

Figures & tables

Panel of Figure 1 — Figure 1: (a) The confidence threshold controls decoding parallelism in MinerU-Diffusion. Compared to MinerU2.5, our method achieves up to 3.26× speedup. (b) MinerU-Diffusion maintains a strong accuracy–efficiency trade-off, achieving 2.12× speedup with 99.9% and 3.01× speedup with 98.8% relative accuracy. (c) Diffusion decoding progressively reconstructs structured text from masked tokens under visual conditioning: black tokens are confirmed, red tokens are being updated, and yellow tokens remain masked, enabling parallel generation with global consistency, in contrast to autoregressive left-to-right decoding.

Figure 2: Overview of the document OCR inverse rendering process via different decoding methods.. The model maps a 2D document image to a 1D token sequence for decoding through autoregressive and diffusion-based methods.

Figure 3: Training of MinerU-Diffusion. Left: the target token sequence is randomly masked to form a partially observed input, and the model predicts only the masked positions under visual and prompt conditioning. Right: the structured block-attention mask used during training, where tokens attend bidirectionally within each block and causally to all preceding blocks, enabling parallel diffusion refinement within blocks while preserving coarse autoregressive structure across blocks.

Type	Methods	Params.	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\textit{TEDS}}$ $\uparrow$	Table ${}_{\textit{TEDS-S}}$ $\uparrow$	Reading Order $\downarrow$	GT Layout
Pipeline	Mineru2-pipeline [ 43 ]	-	75.51	0.209	76.55	70.90	79.11	0.225	$\times$
Pipeline	PP-StructureV3 [ 7 ]	-	86.73	0.073	85.79	81.68	89.48	0.073	$\times$
AR	Qwen2.5-VL-72B [ 40 ]	72B	87.02	0.094	88.27	82.15	86.22	0.102	$\times$
	Gemini-2.5 Pro [ 6 ]	-	88.03	0.075	85.82	85.71	90.29	0.097	$\times$
	MinerU2-VLM [ 43 ]	0.9B	85.56	0.078	80.95	83.54	87.66	0.086	$\times$
	Nanonets-OCR-s [ 25 ]	3B	85.59	0.093	85.90	80.14	85.57	0.108	$\times$

Table 1: Comprehensive evaluation of document parsing on OmniDocBench v1.5.

\uparrow

denotes higher is better,

\downarrow

denotes lower is better.

Type	Methods	Slides	Academic Papers	Book	Textbook	Exam Papers	Magazine	Newspaper	Notes	Financial Report	GT Layout
Pipeline	MinerU2-pipeline [ 43 ]	0.4244	0.0230	0.2628	0.1224	0.0822	0.3950	0.0736	0.2603	0.0411	$\times$
Pipeline	PP-StructureV3 [ 7 ]	0.0794	0.0236	0.0415	0.1107	0.0945	0.0722	0.0617	0.1236	0.0181	$\times$
AR	Qwen2.5-VL-72B [ 40 ]	0.0422	0.0801	0.0586	0.1146	0.0681	0.0964	0.2380	0.1232	0.0264	$\times$
	Gemini-2.5 Pro [ 6 ]	0.0326	0.0182	0.0694	0.1618	0.0937	0.0161	0.1347	0.1169	0.0169	$\times$
	MinerU2-VLM [ 43 ]	0.0745	0.0104	0.0357	0.1276	0.0698	0.0652	0.1831	0.0803	0.0236	$\times$
	Nanonets-OCR-s [ 25 ]	0.0551	0.0578	0.0606	0.0931	0.0834	0.0917	0.1965	0.1606	0.0395	$\times$

Table 2: Breakdown of document parsing on OmniDocBench v1.5 by page type, reported in text edit distance (

\downarrow

Type	Method	CC-OCR		OCRBench v2		UniMER-Test
Type	Method	TEDS $\uparrow$	TEDS-S $\uparrow$	TEDS $\uparrow$	TEDS-S $\uparrow$	CPE $\uparrow$	HWE $\uparrow$	SCE $\uparrow$	SPE $\uparrow$
AR	InternVL3.5-241B [ 45 ]	62.87	69.52	79.50	85.81	91.7	93.2	95.1	97.8
	Qwen2.5-VL-72B [ 40 ]	81.22	86.48	81.33	86.58	88.9	91.8	95.5	96.2
	GPT-4o [ 13 ]	66.98	79.04	70.51	79.55	82.7	85.9	87.8	96.7
	dots.ocr [ 16 ]	75.42	81.65	82.04	86.27	86.8	90.5	94.7	97.5
	MinerU2.5 [ 29 ]	79.76	85.16	87.13	90.62	96.6	94.4	96.4	98.4

Table 3: Comprehensive recognition results on CC-OCR [ 51 ] , OCRBench v2 [ 9 ] , and UniMER-Test [ 42 ] .

\uparrow

denotes higher is better.

Figure 4: Threshold sensitivity analysis of TPF, TPS, and accuracy. TPF denotes tokens per forward, and TPS refers to throughput measured on an NVIDIA H200 GPU with a batch size of 1.

Figure 5: Visualizing the accuracy-throughput trade-off of different models across different OCR tasks under the w/ GT Layout setting.

Type	Method	Inference		Score (w/ GT Layout)					Score (w/o GT Layout)
Type	Method	TPF	TPS	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\text{TEDS}}$ $\uparrow$	Table ${}_{\text{TEDS-S}}$ $\uparrow$	Overall $\uparrow$	Reading Order $\downarrow$
dLM	Static (step=6)	5.33	91.56	88.31	0.064	85.52	85.81	9120	82.73	0.077
	Dynamic ( $\tau=0.97$ )	5.18	98.32	93.34	0.028	91.65	91.17	94.92	89.04	0.057
	Static (step=32)	1.00	21.86	93.02	0.030	91.10	90.95	94.73	89.76	0.060
AR	MinerU2.5 [ 29 ]	1.00	51.46	93.44	0.025	91.98	90.84	95.10	90.67	0.044
AR	PaddleOCR-VL [ 7 ]	1.00	40.77	93.91	0.021	92.13	91.70	95.42	92.79	0.043

Table 4: Comparison of different inference strategies and model types in terms of efficiency (TPF, TPS) and document parsing performance.

Figure 6: Comparison of training dynamics across different curriculum strategies. The two-stage framework achieves smoother optimization and higher final accuracy compared to single-stage baselines.

Setting	Score (w/ GT Layout)					Score (w/o GT Layout)
Setting	Overall $\uparrow$	Text $\downarrow$	Formula $\uparrow$	Table ${}_{\text{TEDS}}$ $\uparrow$	Table ${}_{\text{TEDS-S}}$ $\uparrow$	Overall $\uparrow$	Reading Order $\downarrow$
Stage 1	92.89	0.032	91.58	90.28	94.93	86.13	0.056
Stage 2	89.33	0.038	88.50	83.29	88.93	35.71	0.472
Stage 1 + Stage 2	93.37	0.028	91.92	91.00	94.86	88.94	0.059

Table 5: Comparison of different training stages under GT and non-GT layout settings.

Figure 7: Semantic Shuffle benchmark results across distortion levels.

		Stage-0		Stage-1	Stage-2
		a	b
Vision	Max Resolution	$2048\times 28\times 28$	$4096\times 28\times 28$	$2048\times 28\times 28$	$2048\times 28\times 28$
Vision	#Tokens per Image	$4\sim 2048$	$4\sim 4096$	$4\sim 2048$	$4\sim 2048$
Data	Dataset	LLaVA-Pretrain	LLaVA-NeXT-Data	Layout&OCR	Layout&OCR
Data	#Samples	550K	739K	6.9M	630K
Model	Trainable	MLP Adaptor	All	All	All

Table 6: Training setup and hyperparameters in three training stages.

Figure 8: Training dynamics across the four stages in our training recipe: Stage 0a, Stage 0b, Stage 1, and Stage 2. For each stage, we report both the loss and the gradient norm.

Figure 9: Training curves on a table-only subset.

Figure 10: A very short table where over-provisioned Full-Attn repeats empty rows.

Figure 11: A medium-length table where matched Full-Attn remains stable but over-provisioned Full-Attn repeats.

Figure 12: A long table where short Full-Attn truncates and long Full-Attn repeats tail rows.

Panel of Figure 13 — Figure 13: Complete recognition examples. Left: original page image. Middle: predicted layout. Right: rendered parsed output.

Panel of Figure 14 — Figure 14: Additional complete recognition examples in the same format.

Panel of Figure 15 — Figure 15: Diffusion decoding examples for layout generation.

Panel of Figure 16 — Figure 16: Diffusion decoding examples for text recognition.

Panel of Figure 17 — Figure 17: Diffusion decoding examples for table recognition.

Panel of Figure 18 — Figure 18: Diffusion decoding examples for formula recognition.

Explore similar work

Feb 18, 2026

DODO: Discrete OCR Diffusion Models

Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern Vision-Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via diffusion models. However, we show that existing masked diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.

Sean Man, Gilad Deutch, Roy Ganz +4

Technion - Israel Institute of Technology, Haifa, Israel. · Amazon Web Services.

Optical Character Recognition Diffusion Language Model Decoding Vision-Language Models Blockwise Diffusion Language Models

Ask PDF

Sep 22, 2026

Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decoding

Autoregressive OCR vision-language models accurately convert document images into text and structured markup, but require one sequential decoding step per output token, limiting inference speed. Unlike open-ended text generation, OCR outputs are strongly grounded in the input image, making diffusion-based parallel generation promising. However, when several tokens are predicted in one diffusion step, each is predicted before the others are known. Committing them directly can therefore introduce errors. We therefore introduce GravityOCR, a parameter-shared AR-block-diffusion model jointly trained for parallel drafting and causal AR verification. Verifying drafts before commitment lets the model commit multiple output tokens per round without a separate drafting network. The causal AR path also enables GRPO with sequence- and structure-level OCR rewards, avoiding diffusion-trajectory likelihood estimation while updating the shared drafter parameters. On OmniDocBench v1.6, AR-path GRPO improves the Overall score from 94.92 to 95.16 without reducing diffusion drafting efficiency, while the final model remains close to the original GLM-OCR score of 95.48. In an SGLang serving deployment, GravityOCR commits an average of 9.7 output tokens per forward pass and achieves a

3.94\times

decode-only speedup on region crops and a

1.32\times

end-to-end page-processing speedup over AR decoding.

Dohyun Kim, Sungjun Han, Hyungguk Kim +4

Trillion Labs · Korea University · Seoul National University

Autoregressive Diffusion Speculative Decoding Diffusion Language Model Inference Optical Character Recognition Diffusion Model Inference Acceleration

Ask PDF

Jul 6, 2026

HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better

We present HunyuanOCR-1.5, a lightweight end-to-end OCR-specialized vision-language model. HunyuanOCR unifies document parsing, text spotting, information extraction, text-image translation, and multi-image document understanding within a single end-to-end VLM. Building upon the lightweight architecture of HunyuanOCR-1.0, HunyuanOCR-1.5 does not redesign the backbone, but systematically improves both efficiency and capability. For efficiency, we adapt DFlash to OCR decoding, significantly reducing the latency of long structured outputs such as dense documents, tables, and formulas while preserving output distribution. Powered by DFlash, HunyuanOCR-1.5 achieves a 6.37x Transformer inference speedup and a 2.14x speedup under vLLM, delivering the fastest inference among lightweight OCR VLMs. For capability, we propose Agentic Data Flow, an agent-driven data construction system that transforms model weaknesses into executable data requirements and autonomously performs material search, quality verification, and pipeline development. It substantially improves long-tail capabilities in ancient-script OCR, fine-grained chart and table parsing, multi-image text-centric QA, low-resource multilingual parsing, and document hallucination evaluation. HunyuanOCR-1.5 ranks among the top-tier end-to-end OCR solutions on OmniDocBench v1.6 while achieving new performance milestones across these long-tail tasks. Combined with an upgraded pretraining and post-training recipe, HunyuanOCR-1.5 further extends its capability in high-resolution, long-context, and multi-task scenarios. Experiments demonstrate faster inference, broader OCR capability coverage, and the deployment advantages of a lightweight end-to-end model. We will release the model weights and training code to support future research and real-world OCR applications.

Gengluo Li, Xingyu Wan, Shangpin Peng +20

Institute of Information Engineering, Chinese Academy of Sciences · Large Language Model Department, Tencent · Nankai University

Vision-Language Models Efficient VLM Inference Optical Character Recognition Document Image Understanding

Ask PDF