DocCogito: Aligning Layout Cognition and Step-Level Grounded Reasoning for Document Understanding
Organizations: Fudan University, Shanghai, China
Abstract
Document understanding with multimodal large language models (MLLMs) requires not only accurate answers but also explicit, evidence-grounded reasoning, especially in high-stakes scenarios. However, current document MLLMs still fall short of forming a complete, human-like reasoning process, because even when they improve both layout encoding and CoT-style prompting, the interaction between the two is typically learned implicitly and remains loosely coupled rather than being enforced as a systematic mechanism. So we propose DocCogito, a unified framework that integrates global layout perception with structured, region-grounded reasoning. DocCogito introduces a lightweight layout tower that distills page structure into learnable global layout prior tokens, and a deterministic Visual-Semantic Chain (VSC)-a concise structured representation less ambiguous than free-form natural-language CoT-to supervise fine-grained intermediate reasoning aligned with evidence regions. Training follows a progressive recipe, including layout perception pretraining, VSC-guided cold start, rejection sampling, and GRPO. To further strengthen the internal coupling between layout priors and VSC execution, we augment standard rewards with a fine-grained region-confidence signal that encourages reasoning traces to stay aligned with corresponding evidence regions. Extensive experiments on six benchmarks (DocVQA, WTQ, ChartQA, TextVQA, OCRBench, and InfoVQA) demonstrate strong generalization, achieving state-of-the-art results on four benchmarks.
Figures & tables
| Operation | Description |
|---|---|
| Select | Choose a semantic region ( e.g . , header, table, cell) using structural or key cues. |
| Read | Extract textual content from the selected region. |
| Filter | Filter candidates using predicates ( e.g . , key contains “Revenue”, year==“2024”). |
| Compare | Apply comparison metrics ( eq, max, min ) to identify the intended value. |
| Aggregate | Combine multiple values ( e.g . , summation or string concatenation). |
| Model | Size(B) | DocVQA | InfoVQA | ChartQA | TextVQA | WTQ | OCRBench |
| Donut [ 13 ] | 0.26 | 67.5 | 11.6 | 41.8 | – | 18.8 | – |
| DocFormerv2 [ 1 ] | 0.75 | 87.8 | 48.8 | – | 65.6 | 48.3 | – |
| Mini-Monkey [ 11 ] | 2 | 87.4 | 60.1 | 76.5 | 75.7 | – | 806 |
| HRVDA [ 17 ] | 7.1 | 72.1 | 43.5 | 67.6 | 73.3 | 31.2 | – |
| DocKylin [ 40 ] | 7.1 | 77.3 | 46.6 | – | – | 32.4 | – |
| Park et al. [ 27 ] | 7.2 | 72.7 | 45.9 | 63.3 | 59.2 | 34.5 | – |
Document Understanding
Document understanding enables machines to interpret text, layout, tables, and images to extract information and answer questions about documents.
See all 93 papers