Remembering Has a Price: Learning Memory Worth under Resource Constraints

Guannan Lai; Zi-Yang Bai; Jun-Peng Jiang; Han-Jia Ye

cs.AIOct 19, 2026

Remembering Has a Price: Learning Memory Worth under Resource Constraints

Authors: Guannan Lai, Zi-Yang Bai, Jun-Peng Jiang, Han-Jia Ye

Organizations: School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University

Abstract

Lifelong agents continually accumulate and reuse experience to improve after deployment, yet in practice they often operate under tight generation, interaction, and memory budgets. Under such stringent joint constraints, we find that existing lifelong-memory methods can become ineffective, with several sophisticated approaches offering little advantage over simple retrieval or even no-memory agents. This reveals a more fundamental problem: resource scarcity makes each memory decision more consequential. When only a small amount of past experience can be retained or reused and opportunities for reasoning and recovery are limited, retrieving a relevant but ineffective memory can consume scarce context and hinder downstream execution. The key challenge is therefore not simply to retrieve relevant experience, but to determine which experiences are actually worth retaining and reusing. To address this challenge, we propose TokenWorth, a lightweight online memory framework that learns query-dependent memory worth from sparse execution feedback and applies it consistently to memory retrieval and bounded memory management. Across three LifelongAgentBench environments and both local and API-based deployments, TokenWorth achieves the highest task success rate while using the fewest output tokens among the evaluated methods. These results suggest that effective lifelong adaptation under resource constraints depends not on remembering more, but on learning what is worth remembering. Our code is publicly available at https://github.com/AIGNLAI/TokenWorth.

Figures & tables

Panel of Figure 1 — Figure 1: Left: Representative local results with Qwen2.5-14B-Instruct under the evaluated stringent joint resource constraints; TokenWorth remains effective where several memory baselines do not. Right: Resource-constrained lifelong agents operate under bounded generation, interaction, and memory budgets.

Figure 2: Overview of TokenWorth . Online execution feedback guides query-dependent memory-worth learning for retrieval and bounded memory-bank management.

	Local: Qwen2.5-14B-Instruct			API: DeepSeek-V4-Flash
Method	DB	OS	KG	DB	OS	KG
Standard	25.2	50.2	30.7	26.2	67.2	12.7
RAG	33.0	54.6	11.3	29.4	66.6	31.3
MemRL	28.0	52.2	4.7	30.4	66.2	32.0
Mem0	21.2	50.6	30.7	32.2	68.8	15.3
MemP	23.4	46.8	27.3	28.0	69.8	21.3

Table 1: Task success rate (%) on LifelongAgentBench under stringent joint resource constraints. Best results are shown in bold and second-best results are underlined .

Panel of Figure 3 — Figure 3: Efficiency and component analysis of TokenWorth. (a) Relative output-token usage on LifelongAgentBench with respect to the no-memory Standard agent; diamonds denote the most token-efficient competing method in each setting. (b) Core ablation on DB using the local backbone; the plotted vertical axis is truncated, so the absolute percentages should be read alongside the numeric values discussed in the text.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	DB	OS	KG
Generation budget $O_{\max}$	128	512	128
Interaction budget $J_{\max}$	3	5	15
Memory capacity $B_{\max}$	64	64	64
Retrieved memories $K_{\max}$	1	1	1
Injection budget $L_{\max}$	900 chars	800 chars	4096 chars
Candidate pool size $K_{c}$	12	12	12

Appendix

Table A1 : Environment-specific resource and adaptation configurations for LifelongAgentBench.

	Local: Qwen2.5-14B-Instruct		API: DeepSeek-V4-Flash
Method	BigCode	ALFWorld	BigCode
Standard	41.7	59.1	49.9
RAG	42.6	73.2	50.9
MemRL	42.9	69.7	50.5
Mem0	41.7	64.1	50.2
MemP	42.3	66.4	49.6

Appendix

Table A2 : Task success rate (%) on additional generalization benchmarks. Best results are bold ; second-best results are underlined (including ties).

Parameter	Default	Tested value	SR (%)	$\Delta$ (pp)
RFM influence $\eta$	0.15	0.05	62.4	$-2.6$
		0.10	63.8	$-1.2$
		0.20	64.2	$-0.8$
		0.30	64.0	$-1.0$
Base regularization $\lambda_{w}$	0.10	0.03	64.8	$-0.2$
		0.30	62.6	$-2.4$

Appendix

Table A3 : Hyperparameter sensitivity on DB using Qwen2.5-14B-Instruct. The default configuration achieves

65.0\%

task success.

\Delta

denotes the change in percentage points relative to the default.

Memory Management for LLM Agents

See all 58 papers

LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents
Sep 12, 2026Siddharth Sharma, Nilesh Prasad Pandey, Onat Gungor +1Inference-Time Compute Allocation Memory-Augmented Language Model Agents
University of California, San Diego, CA, USA · West Virginia University, Morgantown, WV, USA
MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents
Oct 18, 2026Yiming Zong, Shiyuan Piao, Jiashuo Jiang +1Memory Management for LLM Agents LLM Context Management
Department of Industrial Engineering & Decision Analysis, HKUST
When to Forget: A Memory Governance Primitive
Apr 13, 2026Baris SimsekAgent Memory Agent Memory Management

Sep 12, 2026J/K move · Enter open · S save

LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents

Siddharth Sharma, Nilesh Prasad Pandey, Onat Gungor, Tajana Rosing

University of California, San Diego, CA, USA · West Virginia University, Morgantown, WV, USA

Inference-Time Compute Allocation Memory-Augmented Language Model Agents Cost-Aware Inference Replay-Based Continual Learning

Open paper AskPDF arXiv

As LLM agents become integrated into increasingly complex workflows, they must continually acquire new capabilities while retaining competence on previously learned tasks. Lifelong agents address this through experience replay, injecting past interactions into the prompt to leverage prior experience during inference. However, replay is not free: every replayed trajectory competes with retrieval, reasoning, tool use, and verification for the same limited prompt and compute budget, making effective resource allocation essential. Existing approaches allocate these resources using fixed replay policies, regardless of whether replay is beneficial for the current task. We identify this as inference-time memory allocation, a distinct problem class for lifelong agents, and introduce LIMBO: the first online framework to our knowledge that treats memory as a controllable inference-time resource and jointly optimizes memory strategy and inference budget for each incoming task. Unlike prior approaches that fix the replay policy or require model weights, teacher supervision, or offline retraining, LIMBO learns this allocation online in a single pass, explicitly balancing task performance and inference cost without modifying the underlying agent. Across three LLM backbones on LifelongAgentBench, LIMBO achieves better cost-accuracy tradeoffs than state-of-the-art memory-augmented baselines and nearly matches all strongest such baselines at up to ~83% lower inference cost (~53% on average). LIMBO adapts its policy across models and environments without retraining, demonstrating that effective allocation can be learned online rather than manually specified.

Abstract

Memory Management for LLM Agents

Explore similar work

LIMBO: Lifelong Inference-Time Memory and Budget Optimization for LLM Agents

MemPrice: Online Memory Allocation via Adaptive Pricing for Long-Horizon Agents

When to Forget: A Memory Governance Primitive