DocSage: An Information Structuring Agent for Multi-Doc Multi-Entity Question Answering

Teng Lin; Yizhang Zhu; Zhengxuan Zhang; Yuyu Luo; Nan Tang

cs.AIMar 12, 2026

DocSage: An Information Structuring Agent for Multi-Doc Multi-Entity Question Answering

Authors: Teng Lin, Yizhang Zhu, Zhengxuan Zhang, Yuyu Luo, Nan Tang

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology

Abstract

Multi-document Multi-entity Question Answering inherently demands models to track implicit logic between multiple entities across scattered documents. However, existing Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) frameworks suffer from critical limitations: standard RAG's vector similarity-based coarse-grained retrieval often omits critical facts, graph-based RAG fails to efficiently integrate fragmented complex relationship networks, and both lack schema awareness, leading to inadequate cross-document evidence chain construction and inaccurate entity relationship deduction. To address these challenges, we propose DocSage, an end-to-end agentic framework that integrates dynamic schema discovery, structured information extraction, and schema-aware relational reasoning with error guarantees. DocSage operates through three core modules: (1) A schema discovery module dynamically infers query-specific minimal joinable schemas to capture essential entities and relationships; (2) An extraction module transforms unstructured text into semantically coherent relational tables, enhanced by error-aware correction mechanisms to reduce extraction errors; (3) A reasoning module performs multi-hop relational reasoning over structured tables, leveraging schema awareness to efficiently align cross-document entities and aggregate evidence. This agentic design offers three key advantages: precise fact localization via SQL-powered indexing, natural support for cross-document entity joins through relational tables, and mitigated LLM attention diffusion via structured representation. Evaluations on two MDMEQA benchmarks demonstrate that DocSage significantly outperforms state-of-the-art long-context LLMs and RAG systems, achieving more than 27% accuracy improvements respectively.

Figures & tables

Figure 1: The overview of DocSage framework, which consists of three core modules: Interactive Schema Discovery, Logic-Aware Structured Extraction, and Schema-Guided Relational Reasoning.

Method	Accuracy
Method	Comparison	Statistics	Relationship	Overall
All sets
GPT-4o	0.262	0.353	0.407	0.338
GPT-4o + RAG	0.696	0.579	0.593	0.620
GraphRAG	0.618	0.558	0.593	0.586
StructRAG	0.678	0.588	0.573	0.612

Table 1: Experimental results for MEBench.

Method	Spotlight Locating		Comparison		Clustering		Chain of Reason		Overall
All sets
GPT-4o	76.79	0.65	50.98	0.29	45.04	0.10	57.46	0.27	54.17	0.26
GPT-4o + RAG	64.04	0.44	41.85	0.26	35.37	0.03	41.62	0.19	43.05	0.18
GraphRAG	22.49	0.00	22.91	0.01	37.52	0.03	45.34	0.23	33.44	0.07
StructRAG	68.07	0.40	63.36	0.36	60.71	0.14	53.27	0.18	60.56	0.23
DocSage (Ours)	85.06	0.84	73.28	0.43	64.43	0.42	73.67	0.57	68.29	0.53

Table 2: Experimental results for Loong.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Comparison	Statistics	Relationship	Overall
DocSage (Full)	0.934	0.908	0.812	0.892
w/o Structured Extraction	0.752	0.687	0.635	0.691
w/o Schema-Guided Reasoning	0.821	0.794	0.703	0.773
w/o Schema Discovery	0.845	0.775	0.724	0.781
w/o CLEAR (Logic Checking)	0.901	0.862	0.783	0.849
w/ Passive Schema Discovery	0.916	0.879	0.795	0.863

Appendix

Table 3: Results of the ablation study on the MEBench benchmark.

Method	Spotlight Locating	Comparison	Clustering	Chain of Reason	Overall
DocSage (Full)	85.06	73.28	64.43	73.67	68.29
w/o Schema Discovery	73.14	58.92	51.87	60.45	59.10
w/o Structured Extraction	78.35	64.11	55.23	66.88	63.64
w/o Schema-Guided Reasoning	80.02	67.45	58.91	68.52	65.73
w/o CLEAR (Logic Checking)	82.57	70.33	61.05	70.89	66.71
w/ Passive Schema Discovery	83.41	71.16	62.78	72.05	67.35

Appendix

Table 4: Ablation Study Results for DocSage on Loong Benchmark

Multi-Document QA

See all 20 papers

Aug 21, 2026

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

Question answering (QA) over long, connected documents remains challenging because relevant evidence may span multiple entities and their relationships. Existing retrieval-augmented generation (RAG) methods typically index documents as raw chunks and retrieve them through embedding similarity. Their performance degrades when chunk boundaries separate entities from supporting evidence or when a question requires multi-hop reasoning across the corpus. We propose EnSI-RAG (Entity-Structure-Indexed Retrieval-Augmented Generation), a framework that constructs a query-independent, entity-centered index. Each record (e, t, k, v) represents an entity e, its type t, a semantic category k in {property, relation, aspect}, and a value v, while retaining links to the original source passages. At query time, these records serve as retrieval handles, and an LLM synthesizes the retrieved passages into the final answer. This design separates evidence localization from answer synthesis while preserving traceable source evidence. Across Loong and Oolong, EnSI-RAG achieves an average accuracy of 78.24. Relative to the published baseline scores used as references, this is 6.62 points higher, suggesting its effectiveness across these settings. The code is available at https://github.com/RamonMeng/EnSI-RAG.

Xuanyu Meng, Jiashuo Sun, Jash Rajesh Parekh +1

University of Illinois Urbana-Champaign Urbana, Illinois, USA

Document QA Retrieval-Augmented Generation Multi-Hop QA

Ask PDF

Aug 12, 2026

SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges

While retrieval-augmented generation (RAG) has proven effective at giving LLMs access to external knowledge, mainstream dense-retrieval implementations remain inherently limited in handling structured constraints and multi-hop reasoning. Graph-based methods address this by constructing knowledge graphs offline, but they often fragment semantics, incur high maintenance, and complicate incremental updates. We propose SAG (SQL-Retrieval Augmented Generation), a structured retrieval architecture that organizes documents into an event-entity index without building a global knowledge graph. SAG represents each chunk as a semantically complete event paired with its entities, forming a latent hyperedge that preserves n-ary relations without decomposing them into triples. At query time, SAG treats shared entities as join keys to connect related chunks. This dynamically yields a query-scoped neighborhood of events, and yet every piece of evidence remains the original chunk throughout. Experiments on HotpotQA, 2WikiMultiHopQA, and MuSiQue show that SAG achieves the best retrieval and end-to-end QA performance on every benchmark, with gains that widen as reasoning-chain complexity increases. On MuSiQue, where multi-hop evidence chaining is most demanding, SAG reaches 80.36% Recall@5, outperforming the strongest baseline by 11.52 points. This work paves the way for knowledge infrastructure that enables LLM agents to retrieve and reason over continually growing organizational knowledge.

Yuchao Wu, Junqin Li, XingCheng Liang +4

Zleap AI

Multi-Hop QA Retrieval-Augmented Generation Graph-Based Retrieval

Ask PDF

Mar 9, 2026

SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation

Answering complex, real-world queries often requires synthesizing facts scattered across vast document corpora. In these settings, standard retrieval-augmented generation (RAG) pipelines suffer from incomplete evidence coverage, while long-context large language models (LLMs) struggle to reason reliably over massive inputs. We introduce SPD-RAG, a hierarchical multi-agent framework for exhaustive cross-document question answering that decomposes the problem along the document axis. Each document is processed by a dedicated document-level agent operating only on its own content, enabling focused retrieval, while a coordinator dispatches tasks to relevant agents and aggregates their partial answers. Agent outputs are synthesized by merging partial answers through a token-bounded synthesis layer (which supports recursive map-reduce for massive corpora). This document-level specialization with centralized fusion improves scalability and answer quality in heterogeneous multidocument settings while yielding a modular, extensible retrieval pipeline. On the LOONG benchmark (EMNLP 2024) for long-context multi-document QA, SPD-RAG achieves an Avg Score of 58.1 (GPT-5 evaluation), outperforming Normal RAG (33.0) and Agentic RAG (32.8) while using only 38% of the API cost of a full-context baseline (68.0).

Yagiz Can Akay, Muhammed Yusuf Kartal, Esra Alparslan +2

TOBB University of Economics and Technology · OSTIM Technical University

Multi-Document QA Retrieval-Augmented Generation Agentic RAG Document QA Multi-Agent LLM Systems

Ask PDF

Abstract

Multi-Document QA

Explore similar work

EnSI-RAG: Entity-Structure-Indexed Retrieval-Augmented Generation for Long-Document Question Answering

SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges

SPD-RAG: Sub-Agent Per Document Retrieval-Augmented Generation