Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residuals

Junaid Farooq

cs.AIAug 21, 2026

Semantic Compression Trees: Multi-Resolution Knowledge Retrieval via Hierarchical Semantic Residuals

Authors: Junaid Farooq

Organizations: National Institute of Technology Srinagar

Abstract

Retrieval-augmented generation relies mostly on flat, fixed-granularity indexes: documents are cut into uniform chunks and retrieved by similarity, discarding the hierarchical structure of the source. We introduce Semantic Compression Trees (SCT), a hierarchical index in which each node stores only its semantic residual -- the information it adds beyond its parent -- and retrieval proceeds by progressive descent from the root, so that per-query cost is governed by tree depth rather than collection size. We evaluate on QASPER (50 papers, 173 questions) under two protocols differing only in whether the benchmark supplies the relevant document, with bootstrap confidence intervals and paired significance tests throughout. The results are mixed and we report them as such. When the document is given, SCT with a zero-LLM extractive compressor matches dense retrieval on answer quality (0.274 vs. 0.277 F1, $p = 0.37$ ) using 30% fewer context tokens and no LLM calls to build the index, and residual storage beats storing full summaries at each node (0.274 vs. 0.205, $p < 0.001$ ). Increasing the collection fifty-fold multiplies flat retrieval's per-query scoring work by 48.9x and SCT's by 6.4x. Progressive descent itself is not supported. Retrieving the same residuals without the tree performs identically when the document is given ( $p = 0.27$ ), and descent is substantially worse when the system must select the document (0.122 vs. 0.165, $p < 0.001$ ). Routing accuracy localises the cause: descent selects the correct paper 20.2% of the time against 39.3% for flat retrieval, because that choice is made from the root residual, the most compressed node in the tree. We conclude that the residual representation is worth keeping and top-down routing is not.

Figures & tables

System	Multi-res.	Non-redund.	Source-agn.	Formal	LLM-free
DPR/ColBERT	✗	N/A	Partial	✗	✓
RAPTOR	✓	✗	Docs only	✗	✗
PageIndex	Partial	✗	PDF only	✗	✗
GraphRAG	✗	N/A	Text only	✗	✗
HippoRAG	✗	N/A	Text only	Partial	✗
SCT (ours)	✓	✓ †	✓	✓	✓ ∗

Table 1: Comparison with related approaches.

System	Ans. F1	95% CI	$p$	Ev. Rec.	Ev. Tok. Rec.	Ctx. tok.
BM25	0.262	[0.233, 0.290]	0.024	0.794	0.946	3,165
VectorRAG (top-5)	0.277	[0.248, 0.306]	—	0.866	0.969	3,120
VectorRAG (top-1)	0.232	[0.203, 0.262]	$<$ 0.001	0.470	0.759	646
RAPTOR	0.272	[0.243, 0.301]	0.207	0.794	0.955	3,058
SCT -Extractive (ours)	0.274	[0.244, 0.306]	0.366	0.720	0.895	2,178
SCT -LLM (ours)	0.233	[0.205, 0.261]	$<$ 0.001	0.009	0.590	802

Table 2: Main results, single-document protocol (

n=172

questions, 50 QASPER papers). Answer F1 is token F1 of the generated answer against the best-matching annotator reference. Brackets give bootstrap 95% confidence intervals;

p

is a paired bootstrap test against VectorRAG. Context tokens are the generator’s input, reported so quality is read against cost; nodes are index nodes scored per query.

System	Ans. F1	95% CI	$p$	Ev. Rec.	Ev. Tok. Rec.	Ctx. tok.
BM25	0.154	[0.129, 0.179]	0.138	0.339	0.750	3,214
VectorRAG (top-5)	0.165	[0.139, 0.192]	—	0.390	0.767	3,206
VectorRAG (top-1)	0.143	[0.118, 0.168]	$<$ 0.001	0.205	0.516	631
RAPTOR	0.156	[0.131, 0.181]	0.048	0.365	0.760	3,425
SCT -Extractive (ours)	0.122	[0.100, 0.145]	$<$ 0.001	0.173	0.575	2,187
SCT -LLM (ours)	0.137	[0.114, 0.162]	0.010	0.006	0.472	1,042

Table 3: Main results, corpus protocol (

n=173

questions, 50 QASPER papers). Answer F1 is token F1 of the generated answer against the best-matching annotator reference. Brackets give bootstrap 95% confidence intervals;

p

is a paired bootstrap test against VectorRAG. Context tokens are the generator’s input, reported so quality is read against cost; nodes are index nodes scored per query.

System	Relev.	Compl.	Correct.	Faith.
BM25	0.760	0.903	0.711	0.994
VectorRAG (top-5)	0.794	0.923	0.780	0.996
VectorRAG (top-1)	0.664	0.690	0.597	0.996
RAPTOR	0.805	0.933	0.743	0.974
SCT -Extractive (ours)	0.785	0.847	0.688	0.997
SCT -LLM (ours)	0.759	0.817	0.674	0.994

Table 4: LLM-as-judge scores, single-document protocol (

n=172

), rubric-scored 1–5 and normalised to

[0,1]

. Judge dimensions are scored independently, one call each.

Variant	Ans. F1	95% CI	$\Delta$	Ev. Tok.	Ctx. tok.	Nodes
Component ablations
SCT -Extractive (ours)	0.274	[0.244, 0.306]	—	0.895	2,178	13
SCT -FullSummary	0.205	[0.179, 0.233]	-0.069	0.584	437	12
SCT -Flat	0.272	[0.244, 0.302]	-0.002	0.939	2,149	17
SCT -EarlyStop	0.273	[0.243, 0.304]	-0.002	0.895	2,098	13
Retrieval depth bound

Table 5: Ablations and depth sweep, single-document protocol.

\Delta

is the change in Answer F1 against the full system. Nodes visited counts index nodes scored per query.

Single-document protocol
Comparison (A vs. B)	$\Delta$	$p$	95% CI of $\Delta$
Tree vs. no tree (identical residuals)	+0.0017	0.420	[-0.0138, +0.0168]
Residuals vs. full summaries	+0.0688	$<$ 0.001	[+0.0411, +0.0983]
Extractive vs. LLM compression	+0.0411	$<$ 0.001	[+0.0207, +0.0618]
Depth $=5$ vs. depth $=1$	+0.0026	0.161	[-0.0025, +0.0075]
Corpus protocol

Table 6: Direct paired comparisons between SCT variants on Answer F1.

\Delta

is A minus B, so a positive value favours A;

p

is a paired bootstrap test and the interval is a bootstrap 95% CI on

\Delta

itself. Tested pairwise because the claims are pairwise: two systems that are each indistinguishable from a common reference may still differ from one another.

Protocol	System	LLM calls	Units indexed	Build (s)	Retr. (ms)
Single-doc	BM25	0	386	0.1	0.0
	VectorRAG (top-5)	0	386	0.1	0.4
	VectorRAG (top-1)	0	386	0.1	0.2
	RAPTOR	131	517	0.2	0.2
	SCT -Extractive (ours)	0	814	0.3	1.3
	SCT -LLM (ours)	0	814	3.1	1.4

Table 7: Index construction cost over 50 papers, and per-query retrieval time. LLM calls are the model-independent construction cost. Build times are wall clock at the stated concurrency. Retrieval time is reported for scale only and must not be compared across systems: SCT scores residual embeddings in pure Python while the flat baselines use a vectorised dot product, so the difference measures the implementations. Table 8 gives the implementation-independent comparison.

System	1 document	50 documents	Growth
BM25	8	386	48.9 $\times$
VectorRAG (top-5)	8	386	48.9 $\times$
VectorRAG (top-1)	8	386	48.9 $\times$
RAPTOR	11	485	46.0 $\times$
SCT -Extractive (ours)	13	81	6.4 $\times$
SCT -LLM (ours)	13	80	6.2 $\times$

Table 8: Index nodes scored per query, as the indexed collection grows from one document to fifty. Reported instead of latency because SCT scores residual embeddings in pure Python while the flat baselines use a vectorised dot product; a wall-clock comparison would measure the implementations. Growth is the ratio between the two columns.

System	Ctx.–Ans. overlap F1	Ctx. tokens	Answer F1
BM25	0.010	3,165	0.262
VectorRAG (top-5)	0.011	3,120	0.277
VectorRAG (top-1)	0.038	646	0.232
RAPTOR	0.011	3,058	0.272
SCT -Extractive (ours)	0.023	2,178	0.274
SCT -LLM (ours)	0.034	802	0.233

Table 9: Context–answer overlap F1, a diagnostic rather than a quality measure, shown against context length. Overlap F1 divides by the length of the retrieved context, so it rewards returning less text whether or not that text answers the question. Answer F1 is repeated for contrast.

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 474 papers

Oct 9, 2026

PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real

High-precision retrieval over external knowledge bases is essential for LLM agents to operate effectively. However, existing Information Retrieval (IR) research primarily optimizes query-side retrievers (e.g., ColBERT, BGE) and generators (e.g., Self-RAG, GraphRAG), while reducing corpus-side representation to chunking. This leaves index-side quality an underexplored lever. We propose Pre-Retrieval Index Modeling & Enhancement (PRIME), an index-side optimization framework built on two complementary tracks: an LLM-driven Semantic Restructuring (SR) track that transforms long documents into structured, information-dense sections, and a deterministic Metadata Synthesis (MS) and Hierarchical Retrieval (HR) track that extracts semantic anchors and heading structure for coarse-to-fine section retrieval. Our evaluations show that deterministic metadata matches structure-aware chunking's retrieval quality while cutting embedding computation by 45.8

\times

and index vector count by 22.4

\times

. On top of these gains, Semantic Restructuring (SR) further improves Answer-F1 by 13.7% in full deployment, with characterized quality-fidelity-cost trade-offs. Across QASPER, BRIGHT, and NFCorpus, we further characterize three qualitative regimes that determine whether index-side optimization helps or degrades retrieval, while showing that these index-side effects are neutralized under a strong reranker.

Cheuk Hei Chung, Li Liu

The Hong Kong University of Science and Technology (Guangzhou)

Information Retrieval

Ask PDF

Sep 30, 2026

Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA

Retrieval-augmented generation grounds language models in external context, but for long documents flat top-

k

retrieval can cluster on a single region and miss complementary evidence. RAPTOR-style summary trees address this by recursively clustering chunks and using a language model to summarize each cluster at indexing time, then ranking summary nodes alongside raw chunks at query time. We show the main benefit of summary trees in long-document QA can come from navigation rather than the generated summary content. We introduce NavTree, a leaves-only retriever that builds a deterministic balanced segment tree over chunks (zero language-model calls at indexing) and uses the tree purely as a navigation scaffold: a hybrid lexical-and-dense frontier walk, anchored on top retrieved leaves, descends from the root and emits only leaf chunks to the reader. On a matched-cost evaluation against flat retrievers and an extractive re-implementation of RAPTOR, NavTree is the strongest matched-cost hierarchical retriever in our evaluated grid and ties the strongest flat baseline. On long-document multi-hop QA, it is the only hierarchical method that significantly beats BM25 on a class-vs-class basis, corroborated by a reader-free retrieval-recall check. A matched-reader replication of the published abstractive RAPTOR variant, given strong cluster summaries, still loses to NavTree at every multi-chunk budget, at zero indexing cost. The ranking carries across stronger and open-weight readers, a stronger encoder, and a full factorial that isolates leaves-only emission as the structural lever.

Priyank Jayraj, Poonam Goyal, Navneet Goyal

Birla Institute of Technology and Science, Pilani Pilani, Rajasthan, India

Long-Context Retrieval Information Retrieval Retrieval-Augmented Generation Hybrid Retrieval

Ask PDF

Oct 25, 2026

Rethinking Retrieval in the Long-Context Era: Training-Free Semantic Trees for Adaptive Context Construction

Long-context language models can increasingly process entire documents, making full-context prompting a practical alternative to passage retrieval. However, greater capacity determines how much information a model can receive, but not which information best supports a query. This distinction motivates rethinking retrieval: when the full document already fits, can selecting and organizing its evidence improve answering? The challenge is to retain supporting facts and relations while keeping the context focused on the question. To reconcile these goals, we introduce \ours, a training-free framework that uses semantic abstraction to guide selection while retaining original text for answering. \ours constructs a reusable semantic hierarchy from the full document in a single generation, linking node descriptions to source spans. Given a query, structural--semantic scoring and fine-grained lexical attribution nominate shared source nodes; their union with ancestor context preserves structural relations and complete local text. Across four frontier answer models, \ours improves average QA-F1 by 1.72 points over full-context prompting on three LongBench tasks, surpassing the evaluated baselines on the four-model average while reducing answer-context length by 58.55%. Our code is available at the repository.

Zhe Xu, Tianhao Liao, Zhenqing Ling +3

Sun Yat-sen University · Alibaba Group

Long-Context Retrieval Information Retrieval LLM Context Compression Question Answering

Ask PDF

Abstract

Information Retrieval

Explore similar work

PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real

Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA

Rethinking Retrieval in the Long-Context Era: Training-Free Semantic Trees for Adaptive Context Construction