VitaBench: Memory Lifecycles Beyond Retrieval

Muquan Yu; Dihan Hu

cs.LGOct 10, 2026

VitaBench: Memory Lifecycles Beyond Retrieval

Authors: Muquan Yu, Dihan Hu

Organizations: University of Hong Kong · Chinese University of Hong Kong · Cornell University

Abstract

Long-term memory benchmarks for language agents primarily evaluate whether information from earlier interactions can be retrieved and used when relevant. Persistent memory systems face an additional problem before future use is known: deciding whether a memory should remain exact, be compressed, consolidated, or deleted. Existing benchmarks capture long-range retrieval, temporal reasoning, evidence reuse, and incremental interaction, but do not directly evaluate these earlier representation decisions. We formalize memory lifecycle evaluation and introduce VitaBench, a benchmark that makes lifecycle decisions measurable through repeated memory reviews and full-horizon counterfactual evaluation. VitaBench contains a controlled Diagnostic tier that isolates future-use patterns and a Naturalistic tier that composes them into generated customer-service-style interaction streams. We characterize lifecycle decision structure with three diagnostics: lifecycle headroom, measuring the advantage of adaptive choices over the best fixed strategy; horizon reversal, measuring when short-horizon decisions change under delayed consequences; and optimal-action diversity, measuring whether different memory states favor different lifecycle operations. Across both tiers, VitaBench exhibits substantial lifecycle structure: lifecycle headroom is 1.364 in Diagnostic and 1.405 in Naturalistic, and 60.2% and 56.5% of states, respectively, change their preferred action between short and full horizons. A learned delayed lifecycle policy further improves over the best fixed strategy on VitaBench-Naturalistic, reaching 101.5 versus 83.4 joint utility. These results show that retrieving past information and deciding how it should persist are distinct evaluation problems, and that lifecycle management requires evaluation of the delayed consequences of representation decisions.

Figures & tables

Figure 1: VitaBench evaluates memory lifecycle decisions. At each review point, a policy must choose how a memory should persist before its future use is known. VitaBench branches from the same pre-decision memory state, applies each candidate lifecycle action, and replays the same future interaction stream for every branch. The resulting full-horizon utilities identify the preferred lifecycle action and quantify the benchmark’s lifecycle decision structure.

Property	LongMemEval	LoCoMo	MemoryAgentBench	VitaBench (ours)
Long-range memory use	yes	yes	yes	yes
Evidence reuse	yes	yes	yes	yes
Updates / conflicts	yes	partial	yes	yes
Incremental interaction	no	no	yes	yes
Repeated lifecycle decisions	no	no	no	yes
Counterfactual lifecycle evaluation	no	no	no	yes

Table 1: Structural comparison of representative long-memory benchmark interfaces with respect to lifecycle-relevant capabilities.

Figure 2: VitaBench exhibits heterogeneous and horizon-dependent lifecycle decisions. (a) Diagnostic and (b) Naturalistic oracle-optimal action shares across

H=5,10,25,120

; (c) Naturalistic stream-level

H=5\rightarrow 120

reversal rates, with the dashed line marking the 56.5% mean.

Tier	$H_{\rm life}$	$R_{H}$	KEEP	COMPRESS	CONSOLIDATE	DELETE
Diagnostic	1.364	0.602	0.429	0.153	0.172	0.246
Naturalistic	1.405	0.565	0.388	0.214	0.111	0.287

Table 2: VitaBench structural diagnostics on held-out streams. Action shares are fractions of states for which each action is full-horizon oracle-optimal.

	Diagnostic			Naturalistic
Policy	Joint utility $\uparrow$	Storage $\downarrow$	Regret $\downarrow$	Joint utility $\uparrow$	Storage $\downarrow$	Regret $\downarrow$
always Keep-Exact	52.4	9,722	1.50	29.3	7,028	1.41
always Compress	170.1	629	1.40	83.4	413	2.53
always Consolidate	125.4	4,418	1.50	40.5	2,776	2.61
myopic ( $H=1$ )	55.5	878	1.90	10.6	613	2.78
one-shot delayed	171.4	3,026	1.50	64.1	5,668	1.32

Table 3: Policy evaluation on held-out VitaBench tiers. Entries report joint utility, mean persistent storage in tokens per episode, and full-horizon policy regret.

Figure 3: Repeated VitaMem improves the storage–utility trade-off over fixed and myopic policies. Results are shown on held-out VitaBench tiers: (a) Diagnostic and (b) Naturalistic. The star marks repeated VitaMem , and dashed lines indicate oracle joint utility.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Policy	Joint utility $\uparrow$	Storage $\downarrow$	Regret $\downarrow$
recency delete	$-57.6$	2,986	2.5
always Keep-Exact	52.4	9,722	1.5
myopic ( $H=1$ )	55.5	878	1.9
always Consolidate	125.4	4,418	1.5
tuned exact-aware heuristic	130.3	6,506	1.1
always Compress	170.1	629	1.4

Appendix

Table 4: Full held-out policy evaluation on VitaBench -Diagnostic. Storage is mean persistent tokens per episode; regret is the mean full-horizon oracle-minus-selected-action value per review point over the common held-out decision-point set.

Policy	Joint utility $\uparrow$	Storage $\downarrow$	Regret $\downarrow$
always Keep-Exact	29.3	7,028	1.41
always Compress	83.4	413	2.53
always Consolidate	40.5	2,776	2.61
always Delete	$-2.6$	88	4.42
recency heuristic	$-5.3$	219	3.36
tuned heuristic	78.0	4,782	1.24

Appendix

Table 5: Full held-out policy evaluation on VitaBench -Naturalistic over six evaluation streams. Storage is mean persistent tokens per episode; regret is averaged over the common held-out decision-point set.

Training target	Joint utility $\uparrow$
full-horizon counterfactual	208.2
MC10 reduced branching	209.2
MC3 sensitive-state branching	216.3
immediate ( $H=1$ )	55.5

Appendix

Table 6: Delayed-credit ablation configuration on held-out Diagnostic streams; these values are separate from the canonical joint-aware repeated-policy result in Table 3 .

Benchmark	Long-range use	Action scored	Same-state alternatives	Shared-future CF	Full-horizon value
LongMemEval	yes	no	no	no	no
LoCoMo	yes	no	no	no	no
MemoryAgentBench	yes	no	no	no	no
MemOps	yes	yes	no	no	no
VitaBench	yes	yes	yes	yes	yes

Appendix

Table 7: Comparison of memory-benchmark evaluation interfaces. “Action scored” means that an earlier memory-management decision is itself an evaluation target. “Same-state alternatives” means that multiple candidate actions are evaluated from the same pre-decision state. “Shared-future CF” means that these alternatives are evaluated under the same subsequent interaction stream.

Memory Agent Benchmarks

Memory agent benchmarks evaluate how AI agents store, retrieve, update, and use information across interactions and changing tasks.

See all 164 papers

Jul 14, 2026

MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

Long-term memory has become a foundational capability for LLM-based agents that accompany users across extended, multi-session interactions. Existing benchmarks, however, evaluate such memory almost exclusively through downstream question answering, scoring only the correctness of a final answer. This black-box formulation conflates the heterogeneous causes of memory failure, such as missing the introduction of a relevant fact, binding an operation to the wrong target, or relying on stale values after a correction. As a result, it can credit correct answers despite their reliance on inconsistent or unsafe memory states. In this paper, we argue that, in dynamic long-horizon interactions, memory is not a static collection of facts but a lifecycle of explicit operations, including remembering, forgetting, updating, reflecting, and their compositions. We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations and represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence. A controllable generation pipeline embeds these operations into long, task-oriented conversations and produces gold operation traces together with six categories of operation-level probes, evaluated under both adjacent-evidence and long-context settings. Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals, revealing that current systems remain far from uniformly reliable. For instance, session-level retrieval outperforms turn-level retrieval, and long-context models remain notably weak at reconstructing ordered memory-state trajectories. These results move long-term memory evaluation from final-answer scoring toward interpretable, operation-level diagnosis.

Xixuan Hao, Zeyu Zhang, Zehao Lin +6

MemTensor (Shanghai) Technology Co., Ltd. · The Hong Kong University of Science and Technology (Guangzhou) · Renmin University of China

Conversational Memory Persistent Memory for Language Models Long-Horizon Agent Evaluation Memory Agent Benchmarks Long-Term Conversational Memory Agent Memory Management

Ask PDF

Sep 11, 2026

LifeFuse-Mem: Lifecycle-Aware State Fusion Against Temporary Overwriting for Long-Term Memory

Long-running LLM agents require memory mechanisms that maintain coherent internal states across interactions. We study a lifecycle-labeled memory setting in which write episodes provide lifecycle metadata during training, and phase-aware readout is used during evaluation. This setting reflects the need to distinguish information that should remain influential across future interactions from information that should affect only the current context. A mismatch between these lifecycles can cause temporary information to overwrite durable knowledge, leading to behavioral drift in persistent agents. Within this setting, we introduce LifeFuse-Mem, a lifecycle-aware neural memory framework that separates information according to its temporal commitment. LifeFuse-Mem uses dedicated memory components and lifecycle-aware updates to allow stable and transient knowledge to evolve locally without converting temporary context into durable state. On the controlled anti-overwrite benchmark, LifeFuse-Mem improves acquisition-controlled retention and reduces temporary overwrite; on two public long-memory benchmarks, it remains broadly competitive. These results suggest that explicit lifecycle signals can help diagnose and mitigate overwrite in compact online memory.

Hanyu Zhao, Yuqian Feng, Zhenyu Song +4

Beijing Academy of Artificial Intelligence · University of Chinese Academy of Sciences · Institute of Software, Chinese Academy of Sciences +1

LLM Agent Memory Persistent Memory for Language Models Agent Memory Management

Ask PDF

Sep 21, 2026

DolphinBench: Mapping the Pareto Frontier of Agent Memory

Agents today often take real-world actions that depend on long-term memory and context recall over time. However, most current memory benchmarks are built for a conversational question-answer format, where the question itself signals that some fact must be retrieved, and often which one. Moreover, benchmarks rarely require anything beyond accuracy from submissions, allowing memory systems to make unreasonable cost/time tradeoffs to achieve higher scores. We present DolphinBench, a benchmark that evaluates memory directly through an agent's task completion. DolphinBench includes three knowledge-work personas with roughly 500k tokens of user messages per persona and evaluates agents on tasks that depend on information from that history. We verify all 200 tasks per persona by running an agent with and without the relevant history, requiring success with it and failure without it. Finally, we require all evaluations to report total cost and latency alongside accuracy, which enables us to evaluate agent memory systems holistically. No existing memory benchmark combines all three. The dataset and evaluation code are available at https://dolphinbench.ai.

Soumil Rathi, Deshraj Yadav, Taranjeet Singh

Mem0

Agent Memory Persistent Memory for Language Models Long-Horizon Agent Evaluation Memory Agent Benchmarks

Ask PDF

Abstract

Memory Agent Benchmarks

Explore similar work

MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations

LifeFuse-Mem: Lifecycle-Aware State Fusion Against Temporary Overwriting for Long-Term Memory

DolphinBench: Mapping the Pareto Frontier of Agent Memory