Towards Autonomous Memory Agents

Xinle Wu; Rui Zhang; Mustafa Anis Hussain; Yao Lu

cs.AIFeb 25, 2026

Towards Autonomous Memory Agents

Authors: Xinle Wu, Rui Zhang, Mustafa Anis Hussain, Yao Lu

Organizations: National University of Singapore, Singapore

Abstract

Recent memory agents improve LLMs by extracting experiences and conversation history into an external storage. This enables low-overhead context assembly and online memory update without expensive LLM training. However, existing solutions remain passive and reactive; memory growth is bounded by information that happens to be available, while memory agents seldom seek external inputs in uncertainties. We propose autonomous memory agents that actively acquire, validate, and curate knowledge at a minimum cost. U-Mem materializes this idea via (i) a cost-aware knowledge-extraction cascade that escalates from cheap self/teacher signals to tool-verified research and, only when needed, expert feedback, and (ii) semantic-aware Thompson sampling to balance exploration and exploitation over memories and mitigate cold-start bias. On both verifiable and non-verifiable benchmarks, U-Mem consistently beats prior memory baselines and can surpass RL-based optimization, improving HotpotQA (Qwen2.5-7B) by 14.6 points and AIME25 (Gemini-2.5-flash) by 7.33 points.

Figures & tables

Figure 1. Passive vs. autonomous memory agents.

Dataset	Domain	Train Set	Test Set
AIME	Math Reasoning	943	30
HotpotQA	Multi-hop QA	2000	500
AdvancedIF	Instruction Following	1316	329
HelpSteer3 (STEM)	Technical Helpfulness	2000	500

Table 1. Datasets used in our experiments and their statistics.

	Qwen2.5-7B				Qwen2.5-14B
Method	Verifiable		Non-verifiable		Verifiable		Non-verifiable
	AIME25	HotpotQA	AdvancedIF	STEM	AIME25	HotpotQA	AdvancedIF	STEM
No Mem.	$6.67\pm 3.33$	$37.80\pm 2.17$	$45.13\pm 1.72$	$56.16\pm 2.72$	$12.00\pm 3.80$	$60.00\pm 2.74$	$49.53\pm 3.22$	$62.56\pm 3.01$
ReasoningBank	$9.33\pm 1.49$	$46.40\pm 1.54$	$44.98\pm 0.57$	$55.98\pm 2.99$	$12.67\pm 2.79$	$62.40\pm 1.97$	$49.01\pm 2.72$	$62.79\pm 2.09$
ReMe	$8.00\pm 1.83$	$44.20\pm 1.87$	$47.05\pm 1.32$	$58.16\pm 1.02$	$12.00\pm 1.83$	$61.80\pm 2.43$	$50.67\pm 2.22$	$63.98\pm 1.88$
MemRL	$7.33\pm 1.49$	$47.80\pm 2.16$	$46.77\pm 1.65$	$57.37\pm 1.78$	$10.00\pm 3.33$	$64.40\pm 2.17$	$49.73\pm 1.86$	$64.03\pm 2.14$

Table 2. Main results on verifiable and non-verifiable tasks.

Method	Accuracy (%)	Expert Calls
Extraction Variants
Only Teacher (Level 1)	$49.40\pm 1.45$	0%
Only Human Expert (Level 3)	$52.80\pm 0.97$	100%
Retrieval & Update Variants
w/o SA-CTS	$47.80\pm 1.48$	–
w/ MemRL’s Retrieval	$50.40\pm 0.96$	–

Table 3. Component Ablations on HotpotQA (Qwen2.5-7B).

Study A: Effectiveness of Learning Source
Setting	Accuracy (%)
Success Trajectories Only	$50.80\pm 1.12$
Failure Trajectories Only (Contrastive)	$51.60\pm 0.45$
U-Mem (Hybrid)	$\boldsymbol{52.40\pm 0.89}$
Study B: Effect of Retrieved Memory Number ( $k$ )
$k=1$	$50.20\pm 1.10$

Table 4. Ablation studies on HotpotQA (Qwen2.5-7B): Impact of Learning Sources and Retrieval Hyperparameter

k

	Gemini-2.5-flash				DeepSeek-chat
Method	Verifiable		Non-verifiable		Verifiable		Non-verifiable
	AIME25	HotpotQA	AdvancedIF	STEM	AIME25	HotpotQA	AdvancedIF	STEM
No Mem.	$46.67\pm 4.08$	$62.40\pm 2.08$	$58.62\pm 2.41$	$64.92\pm 3.41$	$46.67\pm 2.36$	$58.90\pm 2.38$	$54.82\pm 2.49$	$61.82\pm 3.24$
ReasoningBank	$52.00\pm 3.80$	$68.40\pm 2.77$	$59.67\pm 2.66$	$64.99\pm 2.10$	$48.67\pm 3.80$	$60.80\pm 2.14$	$58.74\pm 1.71$	$62.22\pm 1.48$
ReMe	$50.00\pm 2.36$	$69.20\pm 1.08$	$60.96\pm 1.83$	$65.02\pm 2.84$	$48.00\pm 1.83$	$59.80\pm 2.06$	$58.04\pm 1.91$	$63.88\pm 1.73$
MemRL	$50.67\pm 2.79$	$70.40\pm 1.59$	$59.83\pm 2.37$	$65.32\pm 2.55$	$48.00\pm 1.83$	$61.00\pm 1.83$	$58.67\pm 2.04$	$63.12\pm 0.95$

Table 5. Performance comparison on stronger foundation models across four benchmarks.

Figure 3. Scaling trend of U-Mem on HotpotQA.

Figure 4. Correlation between task similarity and memory benefit. Each point denotes a task; the x-axis is AMCS (task similarity), and the y-axis is U-Mem’s performance gain over the base model. Pearson correlation: r = 0.888..

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5. Comparison of average tokens usage between U-MEM, ReasoningBank, ReMe, MemRL and No Mem.

Figure 6. Comparison of training time (GPU hours) between U-MEM and RL (GRPO). U-MEM achieves comparable or better performance with significantly reduced time costs.

Panel of Figure 7 — Figure 7. AIME Success Memory Extraction

Panel of Figure 8 — Figure 8. AIME Failure Memory Extraction (Cascade)

Panel of Figure 9 — Figure 9. Non-verifiable Pairwise Memory Extraction.

Panel of Figure 10 — Figure 10. AIME solving Prompt with Retrieved Memory

Panel of Figure 11 — Figure 11. AdvancedIF solving Prompt with Retrieved Memory

Panel of Figure 12 — Figure 12. Memory Update.

Memory-Augmented Language Model Agents

Memory-augmented language model agents use persistent facts, interaction histories, and skills to support personalization and long-horizon tasks.

See all 428 papers

MemTrain: Self-Supervised Context Memory Training
Jun 2, 2026Ziheng Li, Xingrun Xing, Haoqing Wang +2LLM Agent Memory Persistent Memory for Language Models
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Samsung Research, Beijing, China
ElasticMem: Latent Memory as a Learnable Resource for LLM Agents
May 29, 2026Tao Feng, Chongrui Ye, Tianyang Luo +5LLM Agent Memory Persistent Memory for Language Models
University of Illinois Urbana-Champaign · Nanyang Technological University
AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents
Jun 19, 2026Xingyu Chen, Rui Wang, Zhaopeng Tu +1LLM Agent Memory Persistent Memory for Language Models
Shanghai Jiao Tong University, Tencent · Shanghai Jiao Tong University · Tencent

Jun 2, 2026J/K move · Enter open · S save

MemTrain: Self-Supervised Context Memory Training

Ziheng Li, Xingrun Xing, Haoqing Wang, Zhi-Hong Deng+1

State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Samsung Research, Beijing, China

LLM Agent Memory Persistent Memory for Language Models Long-Horizon LLM Agents Memory-Augmented Language Model Agents Memory Management for LLM Agents

Open paper AskPDF arXiv

Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks. However, collecting high-quality annotated problems for memory-intensive scenarios is costly, and the resulting training data often lack sufficient diversity to cover general memory behaviors. In this work, we propose MemTrain, a self-supervised training framework for generally enhancing the context-memory capability of LLM agents for more effective downstream post-training. MemTrain introduces two coupled proxy tasks over unlabeled Wikipedia corpora: (1) an end-to-end masked reconstruction objective, which requires the model to recover masked entities after multiple rounds of memory updates, thereby encouraging memory maintenance from the final outcome perspective; and (2) an intermediate memory recall objective, which requires the model to reconstruct masked historical information using intermediate memory states, encouraging faithful compression and memory completeness throughout the interaction process. The two objectives are jointly optimized using GRPO. Extensive experiments on long-text QA and search-based QA benchmarks demonstrate that MemTrain consistently improves downstream memory-intensive reasoning performance across different models, achieving gains of up to 17.67 points over direct task-specific post-training.

Abstract

Memory-Augmented Language Model Agents

Explore similar work

MemTrain: Self-Supervised Context Memory Training

ElasticMem: Latent Memory as a Learnable Resource for LLM Agents

AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents