We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.
Figures & tables
Figure 1: Performance on OmniDocBench v1.5 across pipeline (left) and end-to-end (right) models. Qianfan-OCR (red) achieves 93.12, ranking first among all end-to-end models. The red dashed line indicates Qianfan-OCR’s score for cross-category comparison.
Figure 2: Performance on general OCR and document understanding benchmarks. Top row: OlmOCR Bench, OCRBench, OCRBenchv2 (en), and OCRBenchv2 (zh). Bottom row: CCOCR-multilan, and document understanding tasks (DocVQA, TextVQA, OCRVQA) where two-stage OCR+LLM pipelines (hatched bars) show significant degradation compared to end-to-end models.
Figure 3: Architectural comparison between traditional two-stage OCR pipeline and Qianfan-OCR’s end-to-end approach. (a) Traditional pipeline systems separate layout analysis and content recognition into independent stages, suffering from error propagation and irreversible loss of visual context. (b) Qianfan-OCR unifies all processing into a single vision-language model, accepting custom prompts for flexible task control and optionally generating intermediate layout reasoning via Layout-as-Thought ( ⟨ think ⟩ tokens).
Hyperparameter
Stage 1
Stage 2
Stage 3
Stage 4
Trainable modules
Adapter
All
All
All
Tokens
50B
2T
800B
–
Global batch size (samples)
1024
2048
2048
512
Peak learning rate
1e-3
2e-5
1e-5
1e-5
LR schedule
Cosine
Cosine
Cosine
Cosine
Warmup ratio
0.01
0.01
0.01
0.03
Table 1: Training hyperparameters for each stage. Stage 1 trains only the adapter with a higher learning rate for fast alignment. Stages 2–4 use full-parameter training with lower learning rates.
Stage 1
Stage 2
Stage 3
Stage 4
AVG
✓
–
–
✓
71.37
✓
–
OCR
✓
75.97
✓
–
OCR + General
✓
80.07
✓
General
–
✓
83.47
✓
General
OCR
✓
84.09
✓
General
OCR + General
✓
84.39
Table 2: Ablation study on multi-stage training effectiveness using Qianfan-VL-8B. “OCR” denotes domain-specific OCR data, “General” denotes general-purpose data, and “OCR + General” denotes a 1:1 mixture. Stage 4 is instruction tuning with a fixed set of alignment samples across all configurations. Average accuracy is computed over multiple OCR benchmarks.
Model
OlmOCR
ArXiv
Old scans
Tables
Old
Headers &
Multi
Long tiny
Base
Bench
math
scans
footers
column
text
Pipeline
PaddleOCR-VL
80.0
85.7
71.0
84.1
37.8
97.0
79.9
85.7
98.5
PaddleOCR-VL-1.5
79.1
84.6
63.8
81.8
36.9
95.5
80.9
90.5
98.9
MonkeyOCR
75.8
83.8
68.8
74.6
36.1
91.2
76.6
80.1
95.3
End-to-end Model
Table 3: Detailed performance on OlmOCR Bench across document categories. Models are categorized into Pipeline (two-stage) and End-to-end architectures. Best results in each section are in bold .
Model
Overall ↑
Text Edit↓
Formula CDM↑
Table TEDs↑
Table TEDss↑
R-order Edit↓
Pipeline
Marker-1.8.2
71.30
0.206
76.66
57.88
71.17
0.250
MinerU2-pp
71.51
0.209
76.55
70.90
79.11
0.225
Dolphin
74.67
0.125
67.85
68.70
77.77
0.124
Dolphin-1.5
83.21
0.092
80.78
78.06
84.10
0.080
PP-StructureV3
86.73
0.073
85.79
81.68
89.48
0.073
Table 4: Performance comparison on OmniDocBench. Models are categorized into Pipeline (two-stage) and End-to-end architectures. ↑ indicates higher is better, ↓ indicates lower is better. Best results in each section are in bold . Data source: https://github.com/opendatalab/OmniDocBench
Figure 4: Cumulative OmniDocBench v1.5 score with samples sorted by layout label entropy (descending). In the high-entropy region (left), enabling thinking provides a stable advantage. As lower-entropy samples are included, the gap narrows and eventually reverses, with the no-think mode achieving a higher total score overall.
Figure 5: Layout-as-Thought example on a math exam paper. Left: Original document image. Right: Visualization of bounding boxes generated during the thinking phase, with element types color-coded (e.g., text , image , paragraph_title , vision_footnote ).
Model
OCRBench
OCRBenchv2 (en/zh)
CCOCR-multilan
CCOCR-overall
Qianfan-OCR (Ours)
880
56.0 / 60.77
76.7
79.3
PaddleOCR-VL
549
18.15 / 40.86
45.5
29.1
MinerU2.5
331
17.09 / 40.03
43.2
24.4
Dotsocr
625
19.59 / 35.94
47.2
34.6
MonkeyOCR
655
21.78 / 38.91
43.8
35.2
PointsReader
620
23.40 / 40.39
43.5
35.0
Table 5: Performance on general OCR benchmarks. Qianfan-OCR achieves competitive scores while maintaining strong understanding capabilities (Table 6 ).
End-to-End Models
Two-Stage OCR+LLM Systems
Benchmark
Qianfan
Qwen3
Qwen3
PaddleOCR
MinerU2.5
Dotsocr
DeepSeek OCR
-OCR
-VL-4B
-VL-2B
+Qwen3-4B
+Qwen3-4B
+Qwen3-4B
+Qwen3-4B
OCRVQA_TESTCORE
66.8
64.7
59.3
20.4
21.2
26.5
19.3
TextVQA_VAL
80.0
81.8
79.9
21.4
16.1
28.0
40.0
DocVQA
92.8
94.9
92.7
59.8
62.0
67.1
55.9
CharXiv_DQ
94.0
81.8
69.7
0.0
0.0
0.0
0.0
Table 6: End-to-end models vs. two-stage OCR+LLM systems on document understanding benchmarks. Two-stage systems use Qwen3-4B as the downstream LLM. Best end-to-end results in bold .
Model
Overall
OCRBench
OCRBenchv2
OCRBenchv2
CCOCR
Nanonets
(Mean)
KIE
KIE (en)
KIE (zh)
KIE
KIE (F1)
Qianfan-OCR (Ours)
87.9
95.0
82.8
82.3
92.8
86.5
Qwen3-4B-VL
83.5
89.0
82.1
71.3
91.6
83.3
Qwen3-VL-235B-A22B
84.2
94.0
85.6
62.9
95.1
83.8
Gemini-3.1-Pro
79.2
96.0
87.8
63.4
72.5
76.1
Gemini-3-Pro
77.0
93.5
87.1
49.6
72.7
82.1
Table 7: Performance comparison on Key Information Extraction (KIE) benchmarks (normalized scores). Qianfan-OCR achieves the highest overall mean score of 87.9 across five public KIE benchmarks, outperforming both state-of-the-art commercial models and similarly-sized open-source models.
Model
Batch Size
# Query
PPS
PaddleOCR-VL †
–
–
1.224
MinerU 2.5 †
–
–
1.057
MonkeyOCR-pro-1.2B †
–
–
0.673
Dots OCR †
–
–
0.352
Qianfan-OCR (W16A16)
512
512
0.503
Qianfan-OCR (W8A8)
512
512
1.024
Table 8: Inference throughput comparison measured in pages per second (PPS) on OmniDocBench v1.5 with a single A100 GPU. † Results from PaddleOCR-VL technical report.
We introduce OvisOCR2, a 0.8B document parsing model. OvisOCR2 is designed as an end-to-end parser: given a document page image, it generates a Markdown representation in natural reading order, covering text, formulas, tables, and visual regions. We build a data engine that combines filtered real-document annotations with synthetic pages whose rendered images and Markdown targets are derived from the same HTML source. The training recipe includes supervised fine-tuning, reinforcement learning on a 4B branch with a multi-component reward design, on-policy distillation into the 0.8B model, and model fusion. On OmniDocBench v1.6, OvisOCR2 achieves a state-of-the-art overall score of 96.58, placing an end-to-end model at the top of this leaderboard previously dominated by pipeline methods and highlighting the potential of end-to-end document parsing. On PureDocBench, OvisOCR2 also achieves the highest Avg3 score of 75.06. Beyond these two public benchmarks, we evaluate OvisOCR2 on an in-house benchmark designed to cover a broader set of long-tail and challenging scenarios. OvisOCR2 obtains the best overall performance among the compared methods, providing further evidence of its generalization and robustness. OvisOCR2 is available at https://huggingface.co/ATH-MaaS/OvisOCR2.
Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extraction, leading to multi-stage error propagation. We fine-tune SmolDocling, a compact 256M-parameter vision-language model (VLM), to perform end-to-end key-value extraction directly from document images, jointly solving identification, localization, and association in a single pass without OCR preprocessing. We extend DocTags with specialized key, value, region, and link tags, enabling many-to-many relationships in a unified output sequence. To address data limitations, we design an augmentation pipeline combining synthetic form filling and graph-based crops that preserve complete key-value subgraphs. We further introduce a layout-aware evaluation framework extending text matching with spatial bounding box verification. On FUNSD, XFUND, and a large-scale private dataset, our model outperforms larger zero-shot VLM baselines under layout-aware evaluation, while being 27 times smaller than Qwen2.5-VL (7B) and over 5 times faster at inference. The model weights will be released publicly after publication.
A. Said Gurbuz, Ahmed Nassar, Christoph Auer +8
IBM Research Zurich · ETH Zurich · IBM Research Zurich, Rüschlikon, Switzerland +1
Compact OCR-specific vision-language models achieve strong document parsing performance, but often rely on costly supervision and focus primarily on visual-text reconstruction. We introduce Xiaomi-OCR-0, a unified 0.8B model for document parsing and OCR-centric understanding. We build an approximately 170M-sample OCR-centric corpus using an automated data engine that combines expert consensus, render-based verification, and targeted synthesis. Starting from Qwen3.5-0.8B, our progressive training recipe combines Q-Mask-based text anchoring, continued pretraining, and mixed-task reinforcement learning (Mix-RL). Xiaomi-OCR-0 achieves 95.24 on Real5-OmniDocBench, 96.83 on OmniDocBench v1.6, and 87.94 on Wild-OmniDocBench, while reaching an average score of 83.2 across five OCR-oriented VQA benchmarks. Ablations further show that, with sufficient parsing training, OCR-centric understanding supervision provides additional gains for document parsing. Homepage: https://huggingface.co/spaces/SeerRay-Lab/Xiaomi-OCR-0.