TaSR-RAG: Taxonomy-guided Structured Reasoning for Retrieval-Augmented Generation

Jiashuo Sun; Yixuan Xie; Jimeng Shi; Shaowen Wang; Jiawei Han

cs.CLMar 10, 2026

TaSR-RAG: Taxonomy-guided Structured Reasoning for Retrieval-Augmented Generation

Authors: Jiashuo Sun, Yixuan Xie, Jimeng Shi, Shaowen Wang, Jiawei Han

Organizations: University of Illinois Urbana-Champaign

Abstract

Retrieval-Augmented Generation (RAG) helps large language models (LLMs) answer knowledge-intensive and time-sensitive questions by conditioning generation on external evidence. However, most RAG systems still retrieve unstructured chunks and rely on one-shot generation, which often yields redundant context, low information density, and brittle multi-hop reasoning. While structured RAG pipelines can improve grounding, they typically require costly and error-prone graph construction or impose rigid entity-centric structures that do not align with the query's reasoning chain. We propose TaSR-RAG, a taxonomy-guided structured reasoning framework for evidence selection. We represent both queries and documents as relational triples, and constrain entity semantics with a lightweight two-level taxonomy to balance generalization and precision. Given a complex question, TaSR-RAG decomposes it into an ordered sequence of triple sub-queries with explicit latent variables, then performs step-wise evidence selection via hybrid triple matching that combines semantic similarity over raw triples with structural consistency over typed triples. By maintaining an explicit entity binding table across steps, TaSR-RAG resolves intermediate variables and reduces entity conflation without explicit graph construction or exhaustive search. Experiments on multiple multi-hop question answering benchmarks show that TaSR-RAG consistently outperforms strong RAG and structured-RAG baselines by up to 14%, while producing clearer evidence attribution and more faithful reasoning traces.

Figures & tables

Figure 1. Overview of TaSR-RAG . We illustrate Document-level triple extraction and taxonomy-guided entity typing, query decomposition with latent variables, hybrid triple matching for document reranking, and sequential entity binding.

Methods	General QA						Multi-Hop QA								Avg. EM	Avg. F1
	NQ		TriviaQA		PopQA		HotpotQA		2WikimQA		Musique		Bamboogle
	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1	EM	F1
Qwen2.5-72B-Instruct
Direct Inference	19.9	28.3	45.4	53.1	19.7	24.4	20.6	29.0	15.3	19.6	5.5	10.3	17.6	27.8	20.6	27.5
CoT	23.8	33.8	51.6	60.4	24.4	30.3	26.4	37.2	20.5	26.2	9.9	18.6	19.6	31.0	25.2	33.9
RAG	34.2	48.6	53.4	62.5	33.2	41.2	32.8	46.2	22.7	29.1	10.6	19.9	21.6	34.1	29.7	40.2

Table 1. Performance comparison (EM / F1) of different methods on Qwen2.5-7B-Instruct and Qwen2.5-72B-Instruct. The best results are in bold and the second best are underlined .

Method	HotpotQA	2WikiMQA	Bamboogle	Avg.
RAG	32.8	22.7	21.6	25.7
Doc-level Structure	44.0	34.0	40.8	39.6
Query-level Structure	42.5	31.5	43.2	39.1
Full	46.2	35.6	45.1	42.3

Table 2. Ablation to disentangle the effects of Document-level and Query-level structuredization under the Qwen2.5-72B-Instruct setting. Doc-level Structure applies triple extraction and taxonomy typing to documents while keeping the original query unchanged. Query-level Structure enables query decomposition and typed sub-queries but disables document-level structural representations.

Method	HotpotQA	2WikiMQA	Bamboogle	Avg.
No Typing	32.8	22.7	21.6	25.7
Top-level Taxonomy	43.0	34.5	33.6	37.0
Top-two level Taxonomy	46.2	35.6	45.1	42.3
Top-three level Taxonomy	42.0	34.1	36.0	37.4

Table 3. Effect of taxonomy granularity on multi-hop QA performance under the Qwen2.5-72B-Instruct setting.

Matching Strategy	HotpotQA	2WikiMQA	Bamboogle	Avg.
Semantic-only Matching	43.5	23.5	43.2	36.7
Structural-only Matching	44.0	33.5	44.8	40.8
Hybrid Matching (Full)	46.2	35.6	45.1	42.3

Table 4. Ablation on matching strategies under the Qwen2.5-72B setting. Semantic-only matching relies solely on triple embedding similarity, structural-only matching uses typed triple consistency, and hybrid matching combines both signals.

Panel of Figure 2 — Figure 2. Error distribution in our manual analysis. The left figure illustrates Matching Errors and right figure illustrates Generation Errors.

Panel of Figure 3 — Figure 3. Performance–efficiency trade-off under offline preprocessing and inference-time execution. TaSR-RAG (ours) achieves the best overall balance, avoiding the high offline cost of graph-based methods while maintaining low inference-time overhead.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4. Performance by reasoning depth (hop count). We report results for hop counts with non-empty support. While RAG performance degrades as reasoning depth increases, TaSR-RAG maintains strong performance and exhibits increasingly larger gains in deeper reasoning regimes, with particularly pronounced improvements at 3-hop and beyond.

Panel of Figure 5 — Figure 5. Performance comparison between TaSR-RAG and standard RAG across different retrieval budgets (Top-K) on three multi-hop QA datasets. TaSR-RAG consistently outperforms RAG across all settings, with particularly strong gains on 2WikimQA (over 100% relative improvement at Top-10) and substantial improvements on HotpotQA and Bamboogle.

L1	L2 categories
PERSON	Scientist , Engineer , Academic , Politician , Businessperson , Athlete , Actor , Musician , Writer , Journalist , Inventor , MilitaryPerson
ORGANIZATION	Company , University , ResearchInstitute , GovernmentAgency , Nonprofit , InternationalOrganization , MilitaryUnit , SportsTeam , PoliticalParty , MediaOutlet , Hospital , School
LOCATION	Country , StateOrProvince , City , Region , Continent , River , Lake , Mountain , Island , SeaOrOcean , Desert , Park
FACILITY	Building , Bridge , Airport , Station , Port , Museum , Stadium , Campus , Laboratory , PowerPlant
EVENT	War , Election , Tournament , Conference , Festival , Disaster , Protest , LaunchEvent , MergerEvent , Trial
WORK	Book , Film , TVSeries , Song , Album , VideoGame , SoftwareProject , ResearchPaper , LawOrPolicy , Dataset

Appendix

Table 5. Two-level entity taxonomy used for type constraints (L1 and L2).

Split	Dataset	#Examples	Task Type
Eval	NQ	3,610	Single-hop
	TriviaQA	11,313	Single-hop
	PopQA	14,267	Single-hop
	HotpotQA	7,405	Multi-hop
	2WikiMultiHopQA	12,576	Multi-hop
	Musique	2,417	Multi-hop

Appendix

Table 6. Dataset statistics for evaluation.

Question: Which database does Science Activity Planner use, and which company developed it?
Step	Sub-query / operation	Retrieved/selected content	Binding update
0	Candidate retrieval	Dense retrieval returns an evidence pool $\mathcal{D}_{\text{cand}}$ containing Doc[1], Doc[3], Doc[6].	—
1	Document triples + typing	Extract triples $\mathcal{R}_{d}$ and typed triples $\tilde{\mathcal{R}}_{d}$ for each $d\in\mathcal{D}_{\text{cand}}$ .	—
2	Query decomposition + typing	$s_{1}=(\text{Science Activity Planner},\,\texttt{uses},\,?x)$ ; $s_{2}=(?x,\,\texttt{developed\_by},\,?y)$ . $\tilde{s}_{1}=(\textsc{Work/System},\,\texttt{uses},\,\textsc{Work/Database})$ ; $\tilde{s}_{2}=(\textsc{Work/Database},\,\texttt{developed\_by},\,\textsc{Organization/Company})$ .	—
3	Initialize bindings	Initialize entity binding table $\mathcal{B}\leftarrow\emptyset$ .	$\mathcal{B}_{0}=\emptyset$
4	Hop 1: context selection	Compute HybridMatch scores $S(d)$ for $d\in\mathcal{D}_{\text{cand}}$ w.r.t. $(s_{1},\tilde{s}_{1})$ , then filter by $S(d)\geq\theta$ and rank. In this pool, Doc[1] is kept (relation- and type-consistent), while Doc[3] is filtered (option list; weak match). Select $\mathcal{D}_{1}=\{\text{Doc[1]}\}$ .	—

Appendix

Table 7. Case study aligned with Figure 1 and our method. TaSR-RAG performs step-wise evidence selection via hybrid (semantic + structural) triple matching and maintains explicit bindings (

\mathcal{B}_{1},\mathcal{B}_{2}

) to carry resolved entities across hops.

Structured RAG

See all 19 papers

EfficientGraph-RAG: Structured Retrieval-State Management for Cross-Task Retrieval-Augmented Generation
May 25, 2026Miaohe Niu, Lianlei Shan, Zhengtao Yu +2Retrieval-Augmented Generation Graph Retrieval-Augmented Generation
School of Computer Science and Engineering, Northeastern University, China · 2Tsinghua University, China · 3Kunming University of Science and Technology, China +1
SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges
Aug 12, 2026Yuchao Wu, Junqin Li, XingCheng Liang +4Multi-Hop QA Retrieval-Augmented Generation
Zleap AI
GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning
Aug 23, 2026Jun Chen, Yongchao Liu, Pengyu Qiu +6Multi-Hop QA Agentic RAG
1Shenzhen University, Shenzhen, China · 2Great Bay University, Dongguan, China · 3Ant Group, Hangzhou, China +1

May 25, 2026J/K move · Enter open · S save

EfficientGraph-RAG: Structured Retrieval-State Management for Cross-Task Retrieval-Augmented Generation

Miaohe Niu, Lianlei Shan, Zhengtao Yu, Jingbo Zhu+1

School of Computer Science and Engineering, Northeastern University, China · 2Tsinghua University, China · 3Kunming University of Science and Technology, China+1

Retrieval-Augmented Generation Graph Retrieval-Augmented Generation Agentic RAG State Tracking Structured RAG

Open paper AskPDF arXiv

Retrieval-augmented generation (RAG) has become the standard way to ground large language models in external knowledge, but many systems still organize evidence as flat chunks and retrieve it through largely unstructured search. This weak structure becomes a bottleneck for complex retrieval: the system must decide where to search, how to move from coarse topics to entity-relation evidence, which evidence has been verified, and which intermediate artifacts can be reused. We define these intermediate variables as a retrieval state and study RAG as structured state management. EfficientGraph-RAG makes this state explicit through three coupled mechanisms: TAM defines a typed hierarchical state space over evidence, MARS updates and verifies the state through role-specialized agents, and SMP stores reusable state under hierarchy-aware access control. Using one shared framework configuration, EfficientGraph-RAG ranks first on the reported answer-quality metrics averaged over the three evaluated LongBench retrieval-style subsets, matches the strongest agentic baseline on HotpotQA EM while reducing large-model token usage by

3.51\times

, and provides a low-token DocVQA result among retrieval-organizing cross-modal methods. Component analysis shows role-specific mechanisms: MARS is the main answer-quality driver, TAM supplies the typed traversal state and Adaptive Routing signal, and SMP enables corpus-dependent reuse, with cross-query cache hit rates ranging from 3.77% to 23.18%.

Abstract

Structured RAG

Explore similar work

EfficientGraph-RAG: Structured Retrieval-State Management for Cross-Task Retrieval-Augmented Generation

SAG: SQL-Retrieval Augmented Generation with Query-Time Dynamic Hyperedges

GTA-RAG: Graph-Trajectory-Augmented Reinforcement Learning for Multi-Turn Retrieval-Augmented Reasoning