cs.CVMar 11, 2026
SaveQianfan-OCR: A Unified End-to-End Model for Document Intelligence
Abstract
We present Qianfan-OCR, a 4B-parameter end-to-end vision-language model that unifies document parsing, layout analysis, and document understanding within a single architecture. It performs direct image-to-Markdown conversion and supports diverse prompt-driven tasks including table extraction, chart understanding, document QA, and key information extraction. To address the loss of explicit layout analysis in end-to-end OCR, we propose Layout-as-Thought, an optional thinking phase triggered by special think tokens that generates structured layout representations -- bounding boxes, element types, and reading order -- before producing final outputs, recovering layout grounding capabilities while improving accuracy on complex layouts. Qianfan-OCR ranks first among end-to-end models on OmniDocBench v1.5 (93.12) and OlmOCR Bench (79.8), achieves competitive results on OCRBench, CCOCR, DocVQA, and ChartQA against general VLMs of comparable scale, and attains the highest average score on public key information extraction benchmarks, surpassing Gemini-3.1-Pro, Seed-2.0, and Qwen3-VL-235B. The model is publicly accessible via the Baidu AI Cloud Qianfan platform.
Figures & tables
Figure 1: Performance on OmniDocBench v1.5 across pipeline (left) and end-to-end (right) models. Qianfan-OCR (red) achieves 93.12, ranking first among all end-to-end models. The red dashed line indicates Qianfan-OCR’s score for cross-category comparison.
Figure 2: Performance on general OCR and document understanding benchmarks. Top row: OlmOCR Bench, OCRBench, OCRBenchv2 (en), and OCRBenchv2 (zh). Bottom row: CCOCR-multilan, and document understanding tasks (DocVQA, TextVQA, OCRVQA) where two-stage OCR+LLM pipelines (hatched bars) show significant degradation compared to end-to-end models.
Figure 3: Architectural comparison between traditional two-stage OCR pipeline and Qianfan-OCR’s end-to-end approach. (a) Traditional pipeline systems separate layout analysis and content recognition into independent stages, suffering from error propagation and irreversible loss of visual context. (b) Qianfan-OCR unifies all processing into a single vision-language model, accepting custom prompts for flexible task control and optionally generating intermediate layout reasoning via Layout-as-Thought ( think tokens).
| Hyperparameter | Stage 1 | Stage 2 | Stage 3 | Stage 4 |
| Trainable modules | Adapter | All | All | All |
| Tokens | 50B | 2T | 800B | – |
| Global batch size (samples) | 1024 | 2048 | 2048 | 512 |
| Peak learning rate | 1e-3 | 2e-5 | 1e-5 | 1e-5 |
| LR schedule | Cosine | Cosine | Cosine | Cosine |
| Warmup ratio | 0.01 | 0.01 | 0.01 | 0.03 |
Table 1: Training hyperparameters for each stage. Stage 1 trains only the adapter with a higher learning rate for fast alignment. Stages 2–4 use full-parameter training with lower learning rates.
| Stage 1 | Stage 2 | Stage 3 | Stage 4 | AVG |
| ✓ | – | – | ✓ | 71.37 |
| ✓ | – | OCR | ✓ | 75.97 |
| ✓ | – | OCR + General | ✓ | 80.07 |
| ✓ | General | – | ✓ | 83.47 |
| ✓ | General | OCR | ✓ | 84.09 |
| ✓ | General | OCR + General | ✓ | 84.39 |
Table 2: Ablation study on multi-stage training effectiveness using Qianfan-VL-8B. “OCR” denotes domain-specific OCR data, “General” denotes general-purpose data, and “OCR + General” denotes a 1:1 mixture. Stage 4 is instruction tuning with a fixed set of alignment samples across all configurations. Average accuracy is computed over multiple OCR benchmarks.
| Model | OlmOCR | ArXiv | Old scans | Tables | Old | Headers & | Multi | Long tiny | Base |
| Bench | math | scans | footers | column | text | ||||
| Pipeline | |||||||||
| PaddleOCR-VL | 80.0 | 85.7 | 71.0 | 84.1 | 37.8 | 97.0 | 79.9 | 85.7 | 98.5 |
| PaddleOCR-VL-1.5 | 79.1 | 84.6 | 63.8 | 81.8 | 36.9 | 95.5 | 80.9 | 90.5 | 98.9 |
| MonkeyOCR | 75.8 | 83.8 | 68.8 | 74.6 | 36.1 | 91.2 | 76.6 | 80.1 | 95.3 |
| End-to-end Model | |||||||||
Table 3: Detailed performance on OlmOCR Bench across document categories. Models are categorized into Pipeline (two-stage) and End-to-end architectures. Best results in each section are in bold .
| Model | Overall | Text | Formula | Table | Table | R-order |
| Pipeline | ||||||
| Marker-1.8.2 | 71.30 | 0.206 | 76.66 | 57.88 | 71.17 | 0.250 |
| MinerU2-pp | 71.51 | 0.209 | 76.55 | 70.90 | 79.11 | 0.225 |
| Dolphin | 74.67 | 0.125 | 67.85 | 68.70 | 77.77 | 0.124 |
| Dolphin-1.5 | 83.21 | 0.092 | 80.78 | 78.06 | 84.10 | 0.080 |
| PP-StructureV3 | 86.73 | 0.073 | 85.79 | 81.68 | 89.48 | 0.073 |
Table 4: Performance comparison on OmniDocBench. Models are categorized into Pipeline (two-stage) and End-to-end architectures. indicates higher is better, indicates lower is better. Best results in each section are in bold . Data source: https://github.com/opendatalab/OmniDocBench
Figure 4: Cumulative OmniDocBench v1.5 score with samples sorted by layout label entropy (descending). In the high-entropy region (left), enabling thinking provides a stable advantage. As lower-entropy samples are included, the gap narrows and eventually reverses, with the no-think mode achieving a higher total score overall.
Figure 5: Layout-as-Thought example on a math exam paper. Left: Original document image. Right: Visualization of bounding boxes generated during the thinking phase, with element types color-coded (e.g., text , image , paragraph_title , vision_footnote ).
| Model | OCRBench | OCRBenchv2 (en/zh) | CCOCR-multilan | CCOCR-overall |
| Qianfan-OCR (Ours) | 880 | 56.0 / 60.77 | 76.7 | 79.3 |
| PaddleOCR-VL | 549 | 18.15 / 40.86 | 45.5 | 29.1 |
| MinerU2.5 | 331 | 17.09 / 40.03 | 43.2 | 24.4 |
| Dotsocr | 625 | 19.59 / 35.94 | 47.2 | 34.6 |
| MonkeyOCR | 655 | 21.78 / 38.91 | 43.8 | 35.2 |
| PointsReader | 620 | 23.40 / 40.39 | 43.5 | 35.0 |
Table 5: Performance on general OCR benchmarks. Qianfan-OCR achieves competitive scores while maintaining strong understanding capabilities (Table 6 ).
| End-to-End Models | Two-Stage OCR+LLM Systems | ||||||
| Benchmark | Qianfan | Qwen3 | Qwen3 | PaddleOCR | MinerU2.5 | Dotsocr | DeepSeek OCR |
| -OCR | -VL-4B | -VL-2B | +Qwen3-4B | +Qwen3-4B | +Qwen3-4B | +Qwen3-4B | |
| OCRVQA_TESTCORE | 66.8 | 64.7 | 59.3 | 20.4 | 21.2 | 26.5 | 19.3 |
| TextVQA_VAL | 80.0 | 81.8 | 79.9 | 21.4 | 16.1 | 28.0 | 40.0 |
| DocVQA | 92.8 | 94.9 | 92.7 | 59.8 | 62.0 | 67.1 | 55.9 |
| CharXiv_DQ | 94.0 | 81.8 | 69.7 | 0.0 | 0.0 | 0.0 | 0.0 |
Table 6: End-to-end models vs. two-stage OCR+LLM systems on document understanding benchmarks. Two-stage systems use Qwen3-4B as the downstream LLM. Best end-to-end results in bold .
| Model | Overall | OCRBench | OCRBenchv2 | OCRBenchv2 | CCOCR | Nanonets |
| (Mean) | KIE | KIE (en) | KIE (zh) | KIE | KIE (F1) | |
| Qianfan-OCR (Ours) | 87.9 | 95.0 | 82.8 | 82.3 | 92.8 | 86.5 |
| Qwen3-4B-VL | 83.5 | 89.0 | 82.1 | 71.3 | 91.6 | 83.3 |
| Qwen3-VL-235B-A22B | 84.2 | 94.0 | 85.6 | 62.9 | 95.1 | 83.8 |
| Gemini-3.1-Pro | 79.2 | 96.0 | 87.8 | 63.4 | 72.5 | 76.1 |
| Gemini-3-Pro | 77.0 | 93.5 | 87.1 | 49.6 | 72.7 | 82.1 |
Table 7: Performance comparison on Key Information Extraction (KIE) benchmarks (normalized scores). Qianfan-OCR achieves the highest overall mean score of 87.9 across five public KIE benchmarks, outperforming both state-of-the-art commercial models and similarly-sized open-source models.
| Model | Batch Size | # Query | PPS |
| PaddleOCR-VL † | – | – | 1.224 |
| MinerU 2.5 † | – | – | 1.057 |
| MonkeyOCR-pro-1.2B † | – | – | 0.673 |
| Dots OCR † | – | – | 0.352 |
| Qianfan-OCR (W16A16) | 512 | 512 | 0.503 |
| Qianfan-OCR (W8A8) | 512 | 512 | 1.024 |
Table 8: Inference throughput comparison measured in pages per second (PPS) on OmniDocBench v1.5 with a single A100 GPU. † Results from PaddleOCR-VL technical report.