We present Multimodal OCR (MOCR), a document parsing paradigm that jointly parses text and graphics into unified textual representations. Unlike conventional OCR systems that focus on text recognition and leave graphical regions as cropped pixels, our method, termed dots.mocr, treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets, enabling systems to parse documents while preserving semantic relationships across elements. It offers several advantages: (1) it reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction; (2) it supports end-to-end training over heterogeneous document elements, allowing models to exploit semantic relations between textual and visual components; and (3) it converts previously discarded graphics into reusable code-level supervision, unlocking multimodal supervision embedded in existing documents. To make this paradigm practical at scale, we build a comprehensive data engine from PDFs, rendered webpages, and native SVG assets, and train a compact 3B-parameter model through staged pretraining and supervised fine-tuning. We evaluate dots.mocr from two perspectives: document parsing and structured graphics parsing. On document parsing benchmarks, it ranks second only to Gemini 3 Pro on our OCR Arena Elo leaderboard, surpasses existing open-source document parsing systems, and sets a new state of the art of 83.9 on olmOCR Bench. On structured graphics parsing, our model achieves higher reconstruction quality than Gemini 3 Pro across image-to-SVG benchmarks, demonstrating strong performance on charts, UI layouts, scientific figures, and chemical diagrams. These results show a scalable path toward building large-scale image-to-code corpora for multimodal pretraining. Code and models are publicly available at https://github.com/rednote-hilab/dots.mocr.
Figures & tables
Figure 1: Overview of MOCR. Given a document image, MOCR parses anything on the page into unified, ordered textual representations, capturing both textual and visual structures and enabling faithful reconstruction of the original document.
Figure 2: Overall performance comparison results. (a) reports metrics for image parsing tasks within documents. dots.mocr-svg are further enhanced with additional task-specific training for graphics parsing. (b) reports metrics for general document parsing tasks (olmOCR-Bench, OmniDocBench 1.5, and XDocParse).
Figure 3: Comparison between traditional text-only OCR and MOCR paradigms. Traditional OCR treats graphics as pixels and often discards them, while MOCR parses graphics into structured code (e.g. SVG), enabling faithful reconstruction and broader downstream applications.
Model Type
Inference Type
Model Name
Task
Layout-Det
Parsing
Spotting
VQA
IE
Graphics
Cascade Pipeline
Multi-Step
PaddleOCR-V5 [ 5 ]
✗
✗
✓
✗
✗
✗
Marker-1.8.2
✓
✓
✗
✗
✗
✗
PP-StructureV3 [ 5 ]
✓
✓
✗
✗
✗
✗
Specialized VLMs (Modular)
Multi-step
MonkeyOCR-pro-3B [ 16 ]
✓
✓
✗
✗
✗
✗
MinerU2.5 [ 25 ]
✓
✓
✗
✗
✗
✗
Table 1: Comparison of different VLMs and OCR systems across multiple tasks. ✓ indicates Supported (good), △ indicates Supported but Underperforming, and ✗ indicates Not Supported.
Models
olmOCR-Bench
OmniDocBench1.5
XDocParse
Average
Gemini 3 Pro [ 28 ]
1180.4
1128.0
1323.7
1210.7
dots.mocr
1104.4
1059.0
1210.7
1124.7
dots.ocr [ 15 ]
1041.1
1027.2
1190.3
1086.2
HunyuanOCR [ 33 ]
997.6
1003.9
951.1
984.2
PaddleOCR-VL-1.5 [ 4 ]
897.3
997.9
866.4
920.5
GLM-OCR [ 6 ]
884.2
972.6
820.7
892.5
Table 2: Elo comparison on olmOCR-Bench [ 29 ] , OmniDocBench(v1.5) [ 26 ] , and XDocParse [ 15 ] . All models are evaluated under the same Elo protocol with Gemini 3 Flash as judge, on benchmark-provided document images.
Model
ArXiv
Old scans math
Tables
Old scans
Headers & footers
Multi column
Long tiny text
Base
Overall
MinerU 2.5.4 [ 25 ]
76.6
54.6
84.9
33.7
96.6
78.2
83.5
93.7
75.2 ± 1.1
MonkeyOCR-pro-3B [ 16 ]
83.8
68.8
74.6
36.1
91.2
76.6
80.1
95.3
75.8 ± 1.0
dots.ocr [ 15 ]
82.1
64.2
88.3
40.9
94.1
82.4
81.2
99.5
79.1 ± 1.0
DeepSeek-OCR [ 39 ]
77.2
73.6
80.2
33.3
96.1
66.4
79.4
99.8
75.7 ± 1.0
Nanonets-OCR2-3B [ 21 ]
75.4
46.1
86.8
40.9
32.1
81.9
93.0
99.6
69.5 ± 1.1
PaddleOCR-VL [ 3 ]
85.7
71.0
84.1
37.8
97.0
79.9
85.7
98.5
80.0 ± 1.0
Table 3: Performance comparison on olmOCR-Bench.
Methods
UniSVG Low-Level
UniSVG High-Level
UniSVG Score
ChartMimic
Design2Code
GenExam
SciGen
ChemDraw
OCRVerse [ 47 ]
0.632
0.852
0.763
0.799
-
-
-
0.881
Gemini 3 Pro [ 28 ]
0.563
0.850
0.735
0.788
0.760
0.756
0.783
0.839
dots.mocr
0.850
0.923
0.894
0.772
0.801
0.664
0.660
0.790
dots.mocr-svg
0.860
0.931
0.902
0.905
0.834
0.800
0.797
0.901
Table 4: ISVGEN reconstruction scores on UniSVG (low-level, high-level, overall) and downstream visual-language benchmarks (higher is better).
Model
CharXiv Descriptive
CharXiv Reasoning
OCR Reasoning
InfoVQA
DocVQA
ChartQA
OCRBench
AI2D
CountBenchQA
RefCOCO
Qwen3-VL-2B-Instruct [ 1 ]
62.3
26.8
-
72.4
93.3
-
85.8
76.9
88.4
-
Qwen3-VL-4B-Instruct [ 1 ]
76.2
39.7
-
80.3
95.3
-
88.1
84.1
84.9
-
dots.mocr
77.4
55.3
22.85
73.76
91.85
83.2
86.0
82.16
94.46
80.03
Table 5: Performance comparison on CharXiv, OCR reasoning, and a suite of document and vision-language benchmarks.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Model Type
Methods
Size
OmniDocBench(v1.5) TextEdit ↓
OmniDocBench(v1.5) Read OrderEdit ↓
pdf-parse-bench
GeneralVLMs
Gemini 2.5 Pro
-
0.075
0.097
9.06
Qwen3-VL-235B-A22B-Instruct [ 1 ]
235B
0.069
0.068
9.71
Gemini 3 Pro [ 28 ]
-
0.066
0.079
9.68
SpecializedVLMs
Mistral OCR
-
0.164
0.144
8.84
DeepSeek-OCR [ 39 ]
3B
0.073
0.086
8.26
MonkeyOCR-3B [ 16 ]
3B
0.075
0.129
9.27
Appendix
Table 6: Comparison of general-purpose and specialized VLMs on OmniDocBench(v1.5) and pdf-parse-bench. Lower is better for edit metrics.
Figure 4: Qualitative layout analysis results of dots.mocr on heterogeneous documents. The model identifies and partitions structural elements such as titles, paragraphs, multi-column regions, dense tables, mathematical formulas, scanned text, and handwritten content, demonstrating robust document-level structural understanding across diverse formats and languages.
Figure 5: Parsing results of dots.mocr on full-length web screenshots and real-world scene images. The model preserves global reading order and structured components in long webpage layouts, while accurately recognizing and organizing scene text in complex real-world environments, highlighting generalization beyond conventional OCR benchmarks.
Figure 6: SVG parsing outputs and rendered reconstructions produced by dots.mocr-svg for icon images. The model converts raster icons into concise and executable SVG code, capturing geometric primitives, hierarchical grouping, and spatial relationships for scalable vector-level reconstruction.
Figure 7: Rendered SVG reconstruction results of dots.mocr-svg for diverse statistical charts, including bar, line, scatter, and composite visualizations. The model recovers chart structure, axes, legends, data encodings, and semantic grouping through executable SVG representations, enabling editable and reusable vector outputs.
Figure 8: Rendered SVG reconstruction results of dots.mocr-svg for complex cross-disciplinary illustrations in science and design. The model captures intricate geometric structures, semantic components, and hierarchical organization, demonstrating its ability to generalize beyond standard charts to domain-specific structured graphics.
Figure 9: General-purpose visual question answering results of dots.mocr. While retaining strong document parsing and SVG reconstruction capabilities, the model produces coherent and context-aware responses across documents, charts, UI screenshots, and complex illustrations, indicating competitive performance in broader vision–language tasks.
Figure 10: Prompt used in OCR Arena
Figure 11: Example judging results in OCR Arena, showing document images, OCR outputs from two models, and the explanations produced by the VLM judge.