Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understanding

Yuqing Li; Jiangnan Li; Zheng Lin; Ziyan Zhou; Junjie Wu; Jie Zhou; Mo Yu

cs.CLDec 19, 2025

Mindscape-Aware Retrieval Augmented Generation for Improved Long Context Understanding

Authors: Yuqing Li, Jiangnan Li, Zheng Lin, Ziyan Zhou, Junjie Wu, Jie Zhou, Mo Yu

Organizations: Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · WeChat AI, Tencent · Hong Kong University of Science and Technology

Abstract

Humans understand long and complex texts by relying on a holistic semantic representation of the content. This global view helps organize prior knowledge, interpret new information, and integrate evidence dispersed across a document, as revealed by the Mindscape-Aware Capability of humans in psychology. Current Retrieval-Augmented Generation (RAG) systems lack such guidance and therefore struggle with long-context tasks. In this paper, we propose Mindscape-Aware RAG (MiA-RAG), the first framework to formulate mindscape-aware retrieval and generation as a unified conditioning paradigm for LLM-based RAG. MiA-RAG builds a mindscape through hierarchical summarization and conditions both retrieval and generation on this global semantic representation. This enables the retriever to form enriched query embeddings and the generator to reason over retrieved evidence within a coherent global context. We evaluate MiA-RAG across diverse long-context and bilingual benchmarks for evidence-based understanding and global sense-making. It consistently surpasses baselines, and further analysis shows that it aligns local details with a coherent global representation, enabling more human-like long-context retrieval and reasoning.

Figures & tables

Figure 1: Average model ranks across five long-context benchmarks under 3/5/10-chunk settings.

Figure 2: Overview of the framework of MiA-RAG.

Input	# Books	Avg. Chunks	EM	F1
Silver Chunks	20	2.3	31.29	52.91
Oracle (Full Book)	20	69.2	30.04	50.79

Table 1: Validation of silver chunk annotations.

Model	Retriever		Generator		NarrativeQA	$\infty$ Bench	Det.QA-Zh	Det.QA-En	NoCha	Avg.
	Emb. Model	+S	Gen. Model	+S	F1	Acc	Acc	Acc	Pair Acc	Avg.
Summary-Only	-	-	Qwen2.5-72B	✓	39.24	72.05	73.67	61.33	31.75	55.61
Vanilla-RAG	Qwen3-Emb-8B	✗	Qwen2.5-72B	✗	41.13/45.51/49.06	75.55/80.79/86.90	63.67/70.83/78.00	55.50/61.33/71.17	33.33/38.10/41.27	59.48
HippoRAG-v2	Qwen3-Emb-8B	✗	Qwen2.5-72B	✗	40.06/45.20/47.16	79.48/84.28/85.15	70.33/73.83/77.17	64.83/68.00/68.50	33.33/36.51/47.62	61.43
MemoRAG	BGE-M3	✗	Qwen2.5-72B	✗	40.72/43.63/44.86	75.11/76.42/78.17	67.50/68.83/71.17	61.17/63.50/64.33	46.03 / 49.21 /49.21	59.99
Raptor	Qwen3-Emb-8B	✗	Qwen2.5-72B	✗	39.48/42.76/46.94	75.98/79.91/83.84	62.00/68.67/74.67	54.50/62.00/69.67	34.92/34.92/46.03	58.42

Table 2: Overall results on Long-story QA and Reasoning benchmarks under top-3/5/10 retrieval. “+S” indicates whether the summary is incorporated at that stage. Darker gray rows represent deeper mindscape involvement; bold marks the best per scale. The final column reports the average over all metrics per row. Notably, MemoRAG Qian et al. (2025) uses an additional 7B memory model to enrich retrieval, as detailed in Appx. C.2 .

Model	R@3	R@5	R@10	Avg.
Out-of-box
Qwen3-Embedding-8B	28.6	39.1	55.6	41.1
NV-Embed-v2-7B	17.5	24.7	40.7	27.6
GTE-Qwen2-7B-Instruct	21.0	30.4	48.3	33.2
Voyage-Context-3 †	36.1	46.8	63.3	48.7
Trained

Table 3: Retrieval performance on DetectiveQA-ZH. † denotes results reported in SitEmb Wu et al. (2025) .

Method	NarrativeQA			DetectiveQA-ZH			DetectiveQA-EN
Method	3	5	10	3	5	10	3	5	10
MiA-Emb-8B	62.68	75.92	88.09	46.75	59.17	72.50	42.08	54.17	69.75
w/o Summary	55.62	67.19	83.65	37.92	48.75	66.50	34.00	45.75	61.25
SitEmb-8B	59.98	70.70	82.68	42.50	54.50	69.30	36.75	49.25	63.83
Qwen3-Embedding-8B	41.81	54.51	71.13	28.58	39.08	55.58	24.17	34.17	49.25

Table 4: Retrieval performance measured by Recall (%).

Method	Narra.QA	$\infty$ Bench	DetectiveQA	NoCha
MiA-Gen-14B	53.19	82.39	76.03	52.91
w/o Summary	50.49	75.39	71.03	44.97
w/o Claim.	51.22	85.44	75.58	44.44
w/o QA	46.40	81.08	72.81	46.56

Table 5: Ablation study of MiA-Gen-14B . Reported scores are averaged over the 3/5/10-chunk settings.

Summary Generator	Recall@3/5/10 (%)	F1-Score (%)
GPT-4o (Ours)	62.68/75.92/88.09	52.48/53.52/53.56
Qwen2.5-32B-Instruct	61.66/74.60/88.06	50.20/51.80/53.37
Qwen2.5-14B-Instruct	59.74/73.54/87.68	51.45/51.81/52.61
Qwen2.5-7B-Instruct	58.62/72.61/86.17	49.79/51.55/51.48

Table 6: Impact of summary quality on NarrativeQA.

Dimension	(A) MiA-Emb vs SFT-Emb			(B) MiA-Emb vs Vanilla
Dimension	A1	A2	Win	A1	A2	Win
Comprehensiveness	87.74	12.26	MiA-Emb	88.39	11.61	MiA-Emb
Diversity	68.39	31.61	MiA-Emb	63.23	36.77	MiA-Emb
Empowerment	73.87	26.13	MiA-Emb	71.94	28.06	MiA-Emb
Overall Winner	81.29	18.71	MiA-Emb	78.39	21.61	MiA-Emb

Table 7: Pairwise comparison of MiA-based methods vs baselines across evaluation dimensions. Values are percentages. We use Qwen2.5-72B as the generator.

Figure 3: Impact of retriever scale on average results across benchmarks (DetectiveQA-ZH/EN,

\infty

Bench, NoCha, NarrativeQA) with a Qwen2.5-72B generator.

Figure 4: Impact of generator model scale on average results over 5 benchmarks, with a MiA-Emb-8B retriever.

Figure 5: Comparison of projection angles for MiA-Emb and Qwen3-Emb. Lower angles indicate better alignment of queries with the book’s semantic subspace.

Figure 6: Layer-wise comparison of silver-chunk retrieval accuracy and attention allocation proportion.

Figure 7: Attention pattern of MiA-Emb: the last token attends to preceding summary tokens, with red regions indicating tokens that receive high attention.

Figure 8: Layer-wise Mindscape-Coherent Evidence Alignment (MCEA) scores for generator.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Queries	Avg. Tokens	Metrics
NarrativeQA	556	83k	F1, EM, Recall
$\infty$ Bench-EN.MC	229	184k	Accuracy
DetectiveQA	1,200	118k	Accuracy, Recall
Nocha	126	139k	Pairwise Acc.

Appendix

Table 8: Summary of the evaluation datasets.

Setting	MiA-Emb	MiA-Gen
Precision	bfloat16	bfloat16
Batch Size	4	2
Steps	2000	2000
warmup ratio	0.1	0.05
Learning Rate	$1\times 10^{-4}$	$1\times 10^{-5}$
LoRA Rank	128	–

Appendix

Table 9: Training configurations.‘-’ denotes not used.

Method	Retr.	Gen.	Total	Acc
	(ms/q)	(ms/q)	(ms/q)
MiA-RAG (3)	100.23	337.44	437.67	80.79
Vanilla-RAG (10)	10.48	614.45	624.93	77.29
Vanilla-RAG (3)	10.48	307.01	317.49	72.49

Appendix

Table 10: End-to-end latency comparison on

\infty

Bench measured on A100 GPUs. 3-chunk MiA-RAG is 1.43

\times

faster than the 10-chunk vanilla baseline while achieving higher accuracy.

Emb. Model	Gen. Model		EM	F1	Tokens
Emb. Model	Model	+Summ	EM	F1	Tokens
Qwen3-Emb-8B	Qwen2.5-14B	✗	17.7	34.8	12k
MiA-Emb-8B	GPT4o-2405	✗	21.9	38.9	12k
MiA-Emb-8B	Qwen2.5-14B	✗	18.2	36.7	12k
MiA-Emb-8B	Qwen2.5-14B	✓	20.4	39.11	13k
MiA-Emb-8B	MiA-Gen-14B	✓	28.9	48.7	4k

Appendix

Table 11: Results on the NarrativeQA subset in the Helmet benchmark Yen et al. (2024) , evaluated under RAG (

k

=3 or 10) and full context settings.

{\dagger}

denotes results copied from Helmet.

Figure 9: Impact of retriever scale on retrieval performance (Recall@K) on DetectiveQA and NarrativeQA. DetectiveQA scores are averaged over its ZH and EN subsets. SFT-Emb denotes the baseline trained with the identical supervision as MiA-Emb but without access to mindscape summaries.

Model	Retriever		Generator		NarrativeQA	$\infty$ Bench	Det.QA-Zh	Det.QA-En	NoCha	Avg.
	Emb. Model	+S	Gen. Model	+S	F1	Acc	Acc	Acc	Pair Acc	Avg.
Summary-Only	–		Qwen2.5-72B		39.24	72.05	73.67	61.33	31.75	55.61
Vanilla	Qwen3-0.6B		Qwen2.5-72B		37.98/44.11/47.56	72.05/79.48/82.53	64.33/71.00/78.50	54.67/59.83/67.67	31.75/31.75/42.86	57.74
MiA (Emb-Only)	MiA-Emb-0.6B		Qwen2.5-72B		45.13/47.74/49.61	78.23/ 83.00/87.40	72.83/ 80.50/81.50	64.67/70.50/72.12	31.75/33.33/49.21	63.17
MiA	MiA-Emb-0.6B		Qwen2.5-72B		47.92/51.99/52.24	79.04 /80.35/86.03	77.67 /79.50/81.33	69.50/71.67/74.67	42.86/42.86/50.79	65.89
Vanilla	Qwen3-4B		Qwen2.5-72B		36.90/42.02/46.97	75.11/77.73/82.97	64.33/71.00/79.33	54.67/59.00/68.17	31.75/38.10/41.27	57.95

Appendix

Table 12: Results of MiA-Emb framework on long-story QA tasks across different embedding model scales. All generators are fixed to Qwen2.5-72B and evaluated on top-3/5/10 retrieved chunks.

Figure 10: Scaling results comparing MiA-Gen with the vanilla Qwen2.5-Instruct baseline. To ensure a fair comparison with identical retrieval quality, we provide all generators with the same context: the top-

k

chunks retrieved by our MiA-Emb-8B retriever.

Method	NarrativeQA			DetectiveQA-ZH			DetectiveQA-EN			Avg
Method	3	5	10	3	5	10	3	5	10	Avg
Vanilla	41.81	54.51	71.13	28.58	39.08	55.58	24.17	34.17	49.25	44.25
+ Summary	26.24	36.26	53.54	25.83	36.50	49.25	22.58	29.42	42.42	35.78
+ Residual	41.58	54.65	71.29	33.92	43.50	59.58	30.50	37.58	54.42	47.45
MiA-Emb	62.68	75.92	88.09	46.75	59.17	72.50	42.08	54.17	69.75	63.46
- Residual	63.13	76.19	87.47	47.00	58.75	73.83	40.33	54.00	69.83	63.39

Appendix

Table 13: Effect of summary concatenation and residual connection.

Panel of Figure 11 — Figure 11: Prompt templates used in our two-step hierarchical summarization process.

Panel of Figure 12 — Figure 12: Prompt used to filter silver chunks.

Panel of Figure 13 — Figure 13: Prompt used to filter silver nodes.

Panel of Figure 14 — Figure 14: Prompt for sensemaking question generation.

Panel of Figure 15 — Figure 15: Prompt for pairwise evaluation.

Panel of Figure 16 — Figure 16: The query format of [INST]_emb

Panel of Figure 17 — Figure 17: Instruction format of ${\color[rgb]{0.5,0.5,0.5}\texttt{[INST]}_{\texttt{gen}}}$ across tasks.

Panel of Figure 18 — Figure 18: Prompt for sense-making answer generation based on retrieved context.

Panel of Figure 19 — Figure 19: Concise QA prompt design for NarrativeQA.

Panel of Figure 20 — Figure 20: Prompt for Infinity Benchmark.

Panel of Figure 21 — Figure 21: Q&A prompt for NoCha Dataset.

Panel of Figure 22 — Figure 22: Q&A prompt for DetectiveQA Dataset.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 821 papers

UNREAL: Unifying Retrieval and Long-Context with a Single Model
Oct 6, 2026Edan Kinderman, Elad Hoffer, Yochai Blau +4Retrieval-Augmented Generation Knowledge Augmentation for Language Models
NVIDIA · Technion
Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment
Mar 7, 2026Junming Liu, Yuqi Li, Shiping Wen +2Retrieval-Augmented Generation Multimodal RAG
Tongji University, Shanghai, China · The City University of New York, New York, USA · University of Technology Sydney, Sydney, Australia +2
HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning
Jun 3, 2026Zeju Li, Ziyang Zheng, Yizhou Zhou +1Agent Memory Multi-Agent LLM Systems
The Chinese University of Hong Kong · Independent Researcher

Oct 6, 2026J/K move · Enter open · S save

UNREAL: Unifying Retrieval and Long-Context with a Single Model

Edan Kinderman, Elad Hoffer, Yochai Blau, Brian Chmiel+3

NVIDIA · Technion

Retrieval-Augmented Generation Knowledge Augmentation for Language Models LLM Inference Acceleration Information Retrieval Long-Context Inference Long-Context Reasoning in Language Models

Open paper AskPDF arXiv

Long-context inference and Retrieval-Augmented Generation (RAG) handle evidence selection at vastly different scales, from a single long prompt to an entire corpus. We ask whether a single model-internal mechanism can select evidence across this range. We introduce UNifying REtrieval And Long-Context with a Single Model (UNREAL), a model-native evidence selection framework to span corpus retrieval and long-context inference. UNREAL encodes chunks and derives retrieval queries directly from the frozen LLM's internal representations. It adds fewer than 500K trainable parameters and leaves the backbone unchanged. On a 3B-token, 21M-chunk Wikipedia index, all four dense and hybrid UNREAL backbones outperform state-of-the-art retriever-reranker systems. The best model raises recall from 49.1% to 73.2% on HotpotQA, from 31.7% to 60.1% on 2WikiMultiHopQA, and from 8.8% to 14.4% on MuSiQue. Applied to long-context tasks, the same selection mechanism removes distractors before generation, raising NoLiMa accuracy from 1.0% to 24.83% at its maximum context length of 128K tokens, and LV-Eval's F1 score from 49.97% to 54.66% at 256K. UNREAL also reduces FLOPs and time-to-first-token relative to full-context inference from roughly 32K tokens onward, with larger gains as context grows. Together, these results establish model-internal evidence selection as a common foundation for corpus retrieval and evidence-sparse long-context inference.

Abstract

Retrieval-Augmented Generation

Explore similar work

UNREAL: Unifying Retrieval and Long-Context with a Single Model

Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment

HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning