Document understanding with multimodal large language models (MLLMs) requires not only accurate answers but also explicit, evidence-grounded reasoning, especially in high-stakes scenarios. However, current document MLLMs still fall short of forming a complete, human-like reasoning process, because even when they improve both layout encoding and CoT-style prompting, the interaction between the two is typically learned implicitly and remains loosely coupled rather than being enforced as a systematic mechanism. So we propose DocCogito, a unified framework that integrates global layout perception with structured, region-grounded reasoning. DocCogito introduces a lightweight layout tower that distills page structure into learnable global layout prior tokens, and a deterministic Visual-Semantic Chain (VSC)-a concise structured representation less ambiguous than free-form natural-language CoT-to supervise fine-grained intermediate reasoning aligned with evidence regions. Training follows a progressive recipe, including layout perception pretraining, VSC-guided cold start, rejection sampling, and GRPO. To further strengthen the internal coupling between layout priors and VSC execution, we augment standard rewards with a fine-grained region-confidence signal that encourages reasoning traces to stay aligned with corresponding evidence regions. Extensive experiments on six benchmarks (DocVQA, WTQ, ChartQA, TextVQA, OCRBench, and InfoVQA) demonstrate strong generalization, achieving state-of-the-art results on four benchmarks.
Figures & tables
Figure 1: Overview of reasoning pipelines in document understanding models. (a) Traditional text-only CoT. (b) DocLayLLM with OCR-based text/boxes and multiple CoT templates. (c) LayoutLLM with OCR-based text/boxes and staged reasoning (question analysis → area concentration → answer formation). (d) Ours, an OCR-free approach that integrates global layout perception with structured, region-grounded reasoning. VSC means Visual-Semantic Chain.
Figure 2: Overview of our methods. (a) The model architecture, where a lightweight layout tower extracts global layout cues and injects a [LAYOUT] token into the LLM. (b) The two-stage training recipe, consisting of layout perception pretraining and multi-stage GRPO.
Operation
Description
Select
Choose a semantic region ( e.g . , header, table, cell) using structural or key cues.
Table 1: Operations in the Visual-Semantic Chain (VSC).
Model
Size(B)
DocVQA
InfoVQA
ChartQA
TextVQA Val
WTQ
OCRBench
Donut [ 13 ]
0.26
67.5
11.6
41.8
–
18.8
–
DocFormerv2 [ 1 ]
0.75
87.8
48.8
–
65.6
48.3
–
Mini-Monkey [ 11 ]
2
87.4
60.1
76.5
75.7
–
806
HRVDA [ 17 ]
7.1
72.1
43.5
67.6
73.3
31.2
–
DocKylin [ 40 ]
7.1
77.3
46.6
–
–
32.4
–
Park et al. [ 27 ]
7.2
72.7
45.9
63.3
59.2
34.5
–
Table 2: Comparison with representative document-understanding models on multiple benchmarks. We report model size (in billions of parameters) and performance on six benchmarks, following the officially designated evaluation metrics. “Val” denotes the validation set. Light-gray cells indicate benchmarks whose data type is not included in our training mixture (doc/text/inf), i.e., out-of-domain evaluation for our models.
Figure 5Table 6
Figure 5: Qualitative examples of our VSC-style CoT across diverse document types.
Visual long-document understanding is critical for enterprise, legal, and scientific applications, yet the best performing open recipes have not explored reasoning, a capability which has driven leaps in math and code performance. We introduce a synthetic data pipeline for reasoning in long-document understanding that generates thinking traces by scoring each page for question relevance, extracting textual evidence and ordering it from most to least relevant. We apply SFT to the resulting traces within <think> tags, gated by a <cot> control token, and the resulting reasoning capability is internalized via low-strength model merging. We study Qwen3 VL 32B and Mistral Small 3.1 24B. With Qwen3 VL, we achieve 58.3 on MMLongBenchDoc, surpassing the 7× larger Qwen3 VL 235B A22B (57.0). With Mistral, we show that synthetic reasoning outperforms distillation from the Thinking version's traces by 3.8 points on MMLBD-C, and internalized reasoning exhibits 12.4× fewer mean output tokens compared to explicit reasoning. We release our pipeline for reproducibility and further exploration.
Existing Multimodal Large Language Models (MLLMs) suffer from significant performance degradation on the long document understanding task as document length increases. This stems from two fundamental challenges: 1) a low Signal-to-Noise Ratio (SNR), with crucial evidence buried in irrelevant pages; and 2) supervision scarcity, as datasets offering only final short answers provide a weak learning signal. In this paper, we address these challenges by proposing a paradigm that requires the model to execute a structured Analysis, Localization and Reasoning workflow. To instill this capability, we design a two-stage training framework: we first perform Supervised Fine-Tuning on high-quality data generated via an efficient knowledge distillation strategy. Subsequently, we employ an Evidence-aware Group Relative Policy Optimization which jointly optimizes for both evidence localization and answer accuracy. Additionally, we introduce a Evidence-Guided Resolution Allocation strategy to mitigate memory constraints of training on multi-pages documents. Extensive experiments demonstrate that DocSeeker achieves superior performance on both in-domain and out-of-domain tasks. We show it robustly generalizes from short-page training to ultra-long documents and is naturally synergistic with visual Retrieval-Augmented Generation systems, serving as a solid foundation for their implementation.
Hao Yan, Yuliang Liu, Xingchen Liu +5
Huazhong University of Science and Technology · Huawei Inc.
Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a benchmark for integrated chart--context reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%. Reasoning-enhanced models consistently show improved results, but performance degrades as reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document reasoning.
Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park +5
University of Wisconsin-Madison · Massachusetts Institute of Technology · Stanford University +2