The expansion of retrieval-augmented generation (RAG) into multimodal domains has intensified the challenge for processing complex visual documents, such as financial reports. While page-level chunking and retrieval is a natural starting point, it creates a critical bottleneck: delivering entire pages to the generator introduces excessive extraneous context. This not only overloads the generator's attention mechanism but also dilutes the most salient evidence. Moreover, compressing these information-rich pages into a limited visual token budget further increases the risk of hallucinations. To address this, we introduce AgenticOCR, a dynamic parsing paradigm that transforms optical character recognition (OCR) from a static, full-text process into a query-driven, on-demand extraction system. By autonomously analyzing document layout in a "thinking with images" manner, AgenticOCR identifies and selectively recognizes regions of interest. This approach performs on-demand decompression of visual tokens precisely where needed, effectively decoupling retrieval granularity from rigid page-level chunking. AgenticOCR has the potential to serve as the "third building block" of the visual document RAG stack, operating alongside and enhancing standard Embedding and Reranking modules. Experimental results demonstrate that AgenticOCR improves both the efficiency and accuracy of visual RAG systems, achieving expert-level performance in long document understanding. Code and models are available at https://github.com/OpenDataLab/AgenticOCR.
Figures & tables
Figure 1: Overview of AgenticOCR-based RAG. AgenticOCR performs on-demand decompression of visual information precisely where it is needed by utilizing operations such as zoom and rotate.
Figure 2: The training and inference of AgenticOCR. Built upon the image_zoom_and_ocr_tool , trajectory distillation is first performed to initialize the SFT policy. The model is subsequently optimized through GRPO and finally deployed through an integration protocol within visual RAG pipelines.
Figure 3: Distribution of positive trajectories in SFT dataset.
Figure 4: RL curves for reward and its standard deviation. The trend demonstrates that the agent effectively learns to use tools strategically to solve tasks.
Model
MMLongBench-Doc
FinRAGBench-V
TXT
LAY
CHA
TAB
FIG
UNA
ALL
TXT
TAB
CHA
ALL
Vanilla VLMs
GPT-4o †
46.3
46.0
45.3
50.0
44.1
20.2
42.8
-
-
-
37.2
Claude-4-Sonnet
50.4
49.4
50.5
57.3
43.9
59.0
53.4
36.6
20.2
51.9
33.8
Gemini-2.5-Flash
44.0
53.2
46.0
43.9
48.2
56.7
49.6
49.0
41.6
41.0
43.0
Gemini-2.5-Pro
52.1
62.1
55.5
55.3
54.0
59.9
58.1
62.2
55.3
50.4
54.9
Table 1: Main Results on MMLongBench-Doc and FinRAGBench-V. We report the accuracy of five types of evidence sources including pure text (TXT), layout (LAY), chart (CHA), table (TAB), and figure (FIG), and on unanswerable (UNA) samples. Bold and underlined indicate the best and the second score per column. † denotes results reported in original papers, hence some results are unavailable. ∗ Denotes results surpassing human experts (their performance on MMLongBench-Doc is 65.8).
Model
FinRAGBench-V (subset w. bbox)
MVToolbench (w.o. rotation)
FinRAG (Neg)
Pageacc
Recallmin
Precmin
F1min
RecallEM
Pageacc
Recallmin
Precmin
F1min
RecallEM
Pageacc
Qwen3-VL-235B-A22B
86.4
75.6
79.8
76.5
43.4
91.3
57.0
56.7
56.8
7.0
92.9
Gemini-3-Pro-Preview
94.6
83.1
82.2
81.3
41.3
98.6
70.3
67.8
68.6
31.8
96.8
AgenticOCR-4B
97.3
83.7
83.8
81.8
50.1
99.1
78.3
74.6
75.9
27.7
90.0
AgenticOCR-8B
96.8
83.3
85.6
82.8
44.0
99.4
75.9
74.8
75.2
25.5
93.1
Table 2: Performance of AgenticOCR on FinRAGBench-V (subset with bounding boxes), MVToolbench (without rotation), and hard negative samples constructed from FinRAGBench-V. We report page-level accuracy and element-level metrics for positive samples, and page-level accuracy for negative samples.
Figure 5: An example of the AgenticOCR Model retrieval and evidence extraction workflow. The figure illustrates the input document, the key region identified by the model, the visualization of structured HTML table output, and the final evidence crop image, demonstrating the system’s ability to perform on-demand decompression of visual information through zoom and OCR operations.
Table 3: Performance of the visual retrieval pipeline on MMLongBench-Doc and FinRAGBench-V datasets. We report accuracy, average input tokens, and output tokens for different input configurations (Page, Page+OCR, Evidence, Evidence+OCR) using two generators: Gemini-2.5-Pro and Qwen3-VL-32B-Thinking. For Gemini, the Evidence and Evidence+OCR settings incur high token costs due to its fixed per-image token allocation policy, whereas Qwen3-VL allows finer control, demonstrating the efficiency potential of AgenticOCR.
Statistics
Dataset Name
MMLongBench-Doc
FinRAGBench-V
Documents
135
301
- Average/Medium pages
47.5 / 28
76.1 / 57.0
- Average/Medium words
8,393 / 5,743
36,026 / 16,329
Total question
1,082 (-8=1,074)
1,394
- Single-page question
494 (45.7%)
1,218 (87.4%)
Table 4: Statistics of benchmarks
Figure 6: Another example of the AgenticOCR Model retrieval and evidence extraction workflow from MVToolbench. Compared with Figure 5 , the AgenticOCR model invokes image_zoom_and_ocr_tool in "text" mode, which returns plain textual content. The figure presents the input document, the model-identified key region, and the final evidence crop corresponding to the extracted sentence. AgenticOCR successfully completes the task in a real-world scenario, demonstrating its capability to accurately localize and retrieve evidence through on-demand zoom and OCR operations.
Figure 7: An example where a generative model (Gemini-2.5-Pro) answers a question based on the evidence groups provided by the AgenticOCR Module. The model derives the correct result through simple calculations using the key table information and even performs basic cross-validation across different evidence groups.
Figure 8: A hard case in our Module. Since AgenticOCR Model is limited to single-page processing, it selected an incorrect key region (the table in Group 2 / Page 16 contains two cafés). Although the generative model demonstrates relatively strong robustness to distractors, it misidentified the café name due to the low-resolution page screenshot, resulting in an incorrect final answer. This example highlights the current limitations of our framework and points to directions for future improvement.