SAGE: From Direct Answering to Evidence-Grounded Inference for Chinese Ancient Document Understanding

Yuchuan Wu; Xuan Luo; Yinglian Zhu; Meng Fang; Xiangyang Xue; Bin Li

cs.CLAug 25, 2026

SAGE: From Direct Answering to Evidence-Grounded Inference for Chinese Ancient Document Understanding

Authors: Yuchuan Wu, Xuan Luo, Yinglian Zhu, Meng Fang, Xiangyang Xue, Bin Li

Organizations: Fudan University · University of Liverpool

Abstract

Chinese ancient document understanding demands complex visual, linguistic, and historical reasoning. Current Large Vision-Language Models (LVLMs) typically rely on an opaque, single-pass generation paradigm, often producing overconfident and weakly grounded responses. To address this, we propose SAGE, an evidence-grounded multi-agent framework that reformulates Chinese ancient document understanding as evidence-grounded inference rather than direct answer generation. SAGE coordinates specialized agents for task-aware planning, tool-mediated evidence acquisition, claim-level verification, and bounded replanning under a constrained shared-state runtime. This design supports bounded evidence seeking, answer revision, and abstention when grounding is insufficient. Experiments on the AncientDoc benchmark show that SAGE consistently outperforms matched direct-answering baselines across three LVLM backbones. Remarkably, SAGE with Qwen3.5-9B surpasses much larger monolithic LVLMs on most evaluated metrics, highlighting the importance of structured, evidence-grounded inference beyond model scaling.

Figures & tables

Figure 1: Direct LVLM answering vs. SAGE: Evidence-Grounded Inference.

Figure 2: Overview of SAGE . SAGE reformulates Chinese ancient document understanding from direct answering into evidence-grounded inference by coordinating planning, tool-augmented execution, and verification under a constrained shared-state runtime. The runtime maintains an evidence-grounded state, exposes a constrained action interface, tracks budget, and records execution traces.

Task	Evidence Need	Planned Strategy
OCR	Visual page evidence	Read page text and check recognition validity.
Translation	Page / normalized text	Read and normalize text, then generate a page-grounded vernacular translation.
Knowledge QA	Concept / historical evidence	Extract key concepts, retrieve source-constrained evidence, and verify factual claims.
Reasoning QA	Page-grounded evidence	Use recognized page text and question constraints to perform document-grounded reasoning.
Linguistic QA	Text + style evidence	Use page text with variant, rhetoric, style, or genre evidence for linguistic analysis.

Table 1: Task-aware planning strategies in SAGE . The planner selects evidence requirements and action strategies before answer synthesis.

Model / Backbone	Paradigm	Translation		Reasoning QA		Knowledge QA		Linguistic QA
Model / Backbone	Paradigm	CHRF++	BS-F1	CHRF++	BS-F1	CHRF++	BS-F1	CHRF++	BS-F1
DeepSeek-VL2 Wu et al. (2024)	Direct	0.49	50.27	3.42	59.09	3.77	59.83	3.70	60.05
LLaVA-OneVision-72B Li et al. (2024)	Direct	0.44	48.83	5.96	67.20	5.56	66.31	2.51	57.57
InternVL3-78B Zhu et al. (2025)	Direct	4.45	62.40	4.95	65.99	5.25	65.79	2.38	57.78
Qwen2.5-VL-72B Team (2025b)	Direct	9.77	69.87	9.04	71.40	7.82	69.15	3.65	59.34
GPT-4o Hurst et al. (2024)	Direct	3.02	58.86	7.99	70.52	8.02	70.01	4.16	64.58

Table 2: Main results on AncientDoc. OCR is excluded because it primarily depends on external page-reading tools. We report CHRF++ and BERTScore F1 for four understanding tasks. Direct baselines use task-specific prompts with known task types, whereas gray rows denote SAGE runs without task-type labels. Best and second-best results are in bold and underline.

Figure 3: Trace-based behavior analysis of SAGE . (a) Task-Adaptive Runtime Actions : action patterns across task types. Each cell reports the percentage of examples invoking the corresponding action, while the last column reports average final verifier confidence. (b) Verification-Guided Replanning : analysis on Knowledge QA examples where replanning is triggered. After replanning, both trace-derived support rate and verifier confidence increase.

Variant	Reasoning QA		Knowledge QA
Variant	CHRF++	BS-F1	CHRF++	BS-F1
Direct LVLM	6.88	68.56	6.8	66.83
w/o Planner	9.27	67.06	9.34	67.8
w/o Page Reader	9.15	68.65	11.58	70.79
w/o Retrieval	-	-	11.78	70.89
w/o Verifier	9.13	68.98	11.23	70.08

Table 3: Ablation study on reasoning-based QA and knowledge-based QA. We report CHRF++ and BERTScore F1 (BS-F1). Direct LVLM denotes single-pass answering with the same backbone. Other variants remove one replaceable component from SAGE while keeping the backbone, constrained runtime, execution agent, and budget settings fixed. Retrieval is ablated only on Knowledge QA, where the local evidence pool is used.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Tool	Input	Output
Page Reader	Document image	Page-level OCR text and source metadata
Char Normalizer	Raw OCR text	Normalized text and character-level changes
Term Extractor	Question and page text	Candidate terms and retrieval queries
Local Evidence	Question, terms, and page context	Page evidence and local KB snippets

Appendix

Table 4: Evidence-acquisition tool interface used by SAGE. Answer synthesis and verification are handled by the agent modules rather than listed as tools in this table.

Figure 4: Qualitative trace of a Knowledge QA example. The verifier identifies unsupported institutional claims in the initial candidate answer, triggers replanning, and leads to a revised answer that is better grounded in the page evidence and local retrieval results.

Document Understanding

Document understanding enables machines to interpret text, layout, tables, and images to extract information and answer questions about documents.

See all 93 papers

LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding
Oct 10, 2026Zeyu Shan, Yongxin Shi, Shihang Wu +4Document Understanding Historical Document Analysis
School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510641, China · College of Literature and Journalism, Sichuan University, Chengdu, 610065, China
SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA
Apr 10, 2025Sher Badshah, Ali Emami, Hassan SajjadLLM Evaluation Language Model Generation Evaluation
Dalhousie University · Emory University
DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding
Apr 14, 2026Hao Yan, Yuliang Liu, Xingchen Liu +5Document Understanding Long-Document VQA
Huazhong University of Science and Technology · Huawei Inc.

Oct 10, 2026J/K move · Enter open · S save

LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding

Zeyu Shan, Yongxin Shi, Shihang Wu, Peirong Zhang+3

School of Electronic and Information Engineering, South China University of Technology, Guangzhou, 510641, China · College of Literature and Journalism, Sichuan University, Chengdu, 610065, China

Document Understanding Historical Document Analysis Document QA Long-Context QA

Open paper AskPDF arXiv

Chinese historical documents are invaluable cultural treasures, but their complexity poses significant comprehension challenges for non-specialists. Despite the success of Large Language Models (LLMs) and Large Vision-Language Models (LVLMs) in modern document tasks, their capabilities in Long (full-book) Historical Document understanding remain largely unexplored. To bridge this gap, we introduce LongHisDoc, the first benchmark tailored for long historical document understanding. LongHisDoc comprises 1,012 expert-annotated QA pairs derived from 101 PDF documents, with an average of 85.1 pages and 42,776.4 tokens per document. It spans 10 categories and features four QA types: single-page, cross-page, multi-hop, and unanswerable. Answering these questions requires integrating multimodal evidence from text, layout, and visual elements. Our evaluation reveals that the top-performing LVLM and LLM achieve only 62.9% and 69.9% accuracy, respectively, highlighting a substantial performance gap. Furthermore, we assess various modal retrievers and the end-to-end RAG performance to provide references for Retrieval-Augmented Generation (RAG) systems. We believe that LongHisDoc serves as a comprehensive benchmark to catalyze future research in full-book historical document understanding. The benchmark and evaluation code are available at https://huggingface.co/datasets/qweq12433454/LongHisDoc and https://github.com/SCUT-DLVCLab/LongHisDoc.

Abstract

Document Understanding

Explore similar work

LongHisDoc: A Comprehensive Benchmark for Chinese Long Historical Document Understanding

SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA

DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understanding