GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation

Yifan Wang; Mingxuan Jiang; Zhihao Sun; Yixin Cao; Yicun Liu; Keyang Chen; Guangnan Ye; Hongfeng Chai

cs.AIMar 2, 2026

GAM-RAG: Gain-Adaptive Memory for Evolving Retrieval in Retrieval-Augmented Generation

Authors: Yifan Wang, Mingxuan Jiang, Zhihao Sun, Yixin Cao, Yicun Liu, Keyang Chen, Guangnan Ye, Hongfeng Chai

Organizations: 1Fudan University.

Abstract

Retrieval-Augmented Generation (RAG) grounds large language models with external evidence, but many implementations rely on pre-built indices that remain static after construction. Related queries therefore repeat similar multi-hop traversal, increasing latency and compute. Motivated by schema-based learning in cognitive neuroscience, we propose GAM-RAG, a training-free framework that accumulates retrieval experience from recurring or related queries and updates retrieval memory over time. GAM-RAG builds a lightweight, relation-free hierarchical index whose links capture potential co-occurrence rather than fixed semantic relations. During inference, successful retrieval episodes provide sentence-level feedback, updating sentence memories so evidence useful for similar reasoning types becomes easier to activate later. To balance stability and adaptability under noisy feedback, we introduce an uncertainty-aware, Kalman-inspired gain rule that jointly updates memory states and perplexity-based uncertainty estimates. It applies fast updates for reliable novel signals and conservative refinement for stable or noisy memories. We provide a theoretical analysis of the update dynamics, and empirically show that GAM-RAG improves average performance by 3.95% over the strongest baseline and by 8.19% with 5-turn memory, while reducing inference cost by 61%. Our code and datasets are available at: https://anonymous.4open.science/r/GAM_RAG-2EF6.

Figures & tables

Figure 1: Left : Traditional graph-based RAG relies on a static, stateless graph index; related queries repeatedly traverse similar paths, resulting in redundant reasoning and limited adaptability. Right : Memory-plastic RAG distills retrieval feedback and performs memory updates, so future retrieval becomes more efficient and better aligned with evolving query needs.

Figure 2: Overview of our method. GAM-RAG introduce a training-free, gain-adaptive sentence memory to retrieval. A query activates entity nodes and runs iterative graph propagation to discover multi-hop evidence. After each episode, a Kalman-inspired, uncertainty-aware gain update adjusts both memory states and their perplexities to keep updates stable under noisy signals.

Figure 3: Kalman gain dynamics: under different observation-noise levels $R_{i}$ , illustrating fast warm-up and damped refinement.

Methods	2Wiki		HotpotQA		MuSiQue		TimeQA		Medical
Methods	GPT-Acc	Contain-Acc.	GPT-Acc	Contain-Acc.	GPT-Acc	Contain-Acc.	GPT-Acc	Contain-Acc.	GPT-Acc	F1.
Standard LLM
Direct Answering	30.20	33.70	40.00	37.60	14.90	13.20	30.18	31.85	42.10	32.16
Chain-of-Thought (CoT)	31.40	34.50	39.80	38.20	15.80	13.60	30.62	33.67	42.52	34.50
Vanilla RAG
GritLM / GritLM-7B	42.10	44.80	55.20	51.30	28.50	25.80	41.81	42.68	58.83	54.21

Table 1: GPT-Acc and Contain-Acc. on five QA benchmarks. The best result is in bold and the second-best is underlined. w/k-turn memorization reports multi-round evaluation on the same dataset; the superscript ↑ percentages denote improvements relative to 0-turn memorization.

Methods	Indexing		Same Query		Similar Query		Different Query
Methods	Tokens(M)	Time(ks)	GPT-Acc	Time(s)	GPT-Acc	Time(s)	GPT-Acc	Time(s)
GFM-RAG	4.16	3.57	59.80	4.16	57.40	4.07	56.30	4.23
HippoRAG2	5.36	3.48	55.70	4.82	54.10	4.75	54.10	5.10
LinearRAG	0	0.54	61.70	1.32	60.30	1.45	60.60	1.52
PoG	4.97	3.57	54.10	20.43	51.90	20.10	51.10	21.32
DyG-RAG	9.54	7.75	40.70	9.86	40.20	8.45	39.80	9.71

Table 2: Efficiency and robustness analysis on 2Wiki. We report offline indexing cost (token budget and wall-clock time) and online performance/latency under three query scenarios: Same Query, Similar Query, and Different Query, including the effect of multi-turn memorization for memory-based methods.

Figure 4: Ablation under long-term memorization. Performance trends over 10 turns on five datasets for GAM-RAG.

Figure 5: Sentence-memory trajectories. Blue points denote queries, gray points are inactive sentences, and colored points are activated sentences; lighter-to-darker indicates earlier-to-later memorization turns. A sentence’s displacement across turns reflects the magnitude of its memory update. S1 denotes a supportive sentence, whereas S2 denotes a non-supportive sentence.

Figure 6: Search space analysis of GAM-RAG by varying the per-iteration activation for sentences ( $K_{S}$ ) and entities ( $K_{E}$ ).

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Setting	Corpus	Eval. questions
HotpotQA ( Yang et al., 2018 )	Multi-hop QA	Same as HippoRAG	1,000 (val)
2Wiki ( Ho et al., 2020 )	Multi-hop QA	Same as HippoRAG	1,000 (val)
MuSiQue ( Trivedi et al., 2022 )	Multi-hop QA	Same as HippoRAG	1,000 (val)
TimeQA ( Chen et al., 2021 )	Time-sensitive QA	Provided corpus	3,078 (test-hard)
Medical ( Xiao et al., 2025 )	Domain QA	GraphRAG-Bench	4,076 (all)

Appendix

Table 3: Dataset summary and evaluation splits used in our experiments.

Methods	2Wiki	HotpotQA	MuSiQue	TimeQA	Medical
Methods	GPT-Acc	GPT-Acc	GPT-Acc	GPT-Acc	GPT-Acc
GritLM-7B ( Muennighoff et al., 2024 )	64.00	75.10	43.10	49.11	65.24
NV-Embed-v2 ( Lee et al., 2024 )	64.10	74.60	43.20	48.75	64.98
all-mpnet-base-v2 ( Song et al., 2020 )	64.20	74.90	43.30	49.45	65.47
all-MiniLM-L6-v2 ( Wang et al., 2020 )	63.70	73.70	42.80	48.11	65.22
bge-large-en-v1.5 ( Xiao et al., 2024 )	64.10	74.30	43.10	48.73	64.28

Appendix

Table 4: Effect of different sentence embedding models on vanilla dense-retrieval RAG, evaluated by GPT-Acc across five QA benchmarks.

Retrieval-Augmented Generation

Retrieval-augmented generation connects language models to external knowledge so answers can use current, domain-specific evidence.

See all 792 papers

Latent Abstraction for Retrieval-Augmented Generation
Apr 20, 2026Ha Lan N. T, Minh-Anh Nguyen, Dung D. LeRetrieval-Augmented Generation Adaptive RAG
Center for AI Research, VinUniversity, Vietnam
Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation
Apr 9, 2026Zhengyi Zhao, Shubo Zhang, Zezhong Wang +7Hallucination in RAG Retrieval-Augmented Generation
The Chinese University of Hong Kong · Ministry of Education Key Laboratory of High Confidence Software Technologies, CUHK · University of International Relations +2
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
Aug 3, 2026Shicheng Xu, Liang Pang, Liyi Chen +7Retrieval-Augmented Generation Continual Learning for LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences · Xiaohongshu Inc.

Apr 20, 2026J/K move · Enter open · S save

Latent Abstraction for Retrieval-Augmented Generation

Ha Lan N. T, Minh-Anh Nguyen, Dung D. Le

Center for AI Research, VinUniversity, Vietnam

Retrieval-Augmented Generation Adaptive RAG Adaptive Retrieval Dense Retrieval

Open paper AskPDF arXiv

Retrieval-Augmented Generation (RAG) has become a standard approach for enhancing large language models (LLMs) with external knowledge, mitigating hallucinations, and improving factuality. However, existing systems rely on generating natural language queries at each hop and maintaining a strict architectural separation between retriever and generator, preventing them from leveraging the full representational capacity of the LLM. We propose LAnR (Latent Abstraction for RAG), a unified framework in which a single LLM jointly performs encoding, retrieval, and generation entirely within its own latent space. Rather than generating textual queries, LAnR produces dense retrieval vectors from the hidden states of a designated [PRED] token and uses them to match against encoded document representations from the same model. Furthermore, LAnR adaptively decides when sufficient evidence has been retrieved using a lightweight MLP control head over those same hidden states, eliminating both the separate retriever and explicit token-level stopping reasoning. This design is motivated by our empirical observation that answer token entropy reliably signals retrieval sufficiency. Extensive experiments on six QA benchmarks spanning single-hop and multi-hop settings demonstrate that LAnR outperforms existing RAG methods, while achieving improved inference efficiency through reduced number of retrieval calls and tighter model integration.

Abstract

Retrieval-Augmented Generation

Explore similar work

Latent Abstraction for Retrieval-Augmented Generation

Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation

RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection