Autonomous agents powered by foundation models can reason, plan, and use tools, but their fixed weights and context limits hinder sustained learning from experience over long-term deployment. While external memory systems attempt to bridge this gap, they mostly act as passive retrieval caches, evaluated primarily by whether they can recall stored facts or past interactions. Whether memory can drive genuine continual learning remains an open question. In this paper, we introduce Memory-Centric Continual Learning (MCCL), arguing that memory should be judged by whether it improves future actions, not merely what it retrieves. MCCL has three main parts. First, a three-axis taxonomy organizes memory by where state is stored (substrates), what form it takes (assets), and how it is managed (policies). This defines a two-loop lifecycle that updates memory state in an inner loop while refining management policies in an outer loop based on task feedback. Second, an evaluation protocol measures retention under interference, adaptation to change, and task transfer to isolate genuine behavioral gains from simple context expansion. Third, a memory scaling framework models how agent performance scales with experience, storage capacity, and compute budgets across non-parametric and parametric memory. Together, these components reframe agent memory from passive lookup to continuous learning, offering a principled route toward self-improving autonomous agents.
Figures & tables
Figure 1: Memory scaling across two substrates. Theoretical illustrations: left , joint allocation of storage and reading resources sustains improvement beyond fixed-capacity limits; center , greater experience requires more parametric memory capacity; right , load-induced interference emerges at substrate-dependent thresholds. Curves are synthetic illustrations; see Section 4.3 and Figures 7 and 11 .
Figure 2: Memory-centric continual learning as a coupled dynamical system. A task stream is executed by the agent and produces outcome feedback. Experience is organized as episodic, semantic, and procedural assets that persist across contextual, activation, external, and parametric substrates. An inner memory-state loop forms, consolidates, reactivates, and revises these assets, while prescribed, deliberative, or learned policies govern the corresponding operations. An outer policy-adaptation loop uses task feedback to update memory management.
Figure 3: A unified taxonomy of agent memory for continual learning. Memory substrates specify where experience persists, memory assets describe what reusable representations are constructed from experience, and memory policies determine how memory is written, retrieved, consolidated, revised, and forgotten. Representative related works are organized by subclass under each of the three dimensions.
Figure 4: The memory lifecycle as two coupled processes in memory-centric continual learning. The memory-state loop governs how experience enters persistent memory, is organized and reused, and is revised or forgotten as conditions change. The control loop uses behavioral outcomes to influence future memory decisions.
Figure 5: Lifecycle–control landscape of selected representative agent memory systems. The horizontal axis shows the memory operations emphasized by each system. The vertical axis shows how decisions are controlled: through prescribed rules, deliberative runtime LLM reasoning, or learned control. Darker regions indicate the main design focus and lighter regions indicate supporting operations. Colors distinguish design families. Positions describe design choices, not performance.
Metric Layer
Demonstrated Capability
Unaddressed Failure Mode
Content Fidelity
Accurate and lossless storage of past factual observations
Does not guarantee that stored evidence will be retrieved or utilized in future decisions
Retrieval Access
High recall, precision, and reachability of relevant memories
Does not prevent retrieved evidence from being ignored, misinterpreted, or misapplied
Single-Task Outcome
Immediate performance boost on a specific evaluation probe
Does not establish retention under delay and interference, adaptive revision, or transfer
Table 1: Diagnostic scope of evaluation metric layers in agent memory systems. While component-level and snapshot metrics diagnose internal storage fidelity and retrieval reachability, establishing continual learning requires behavioral retention, adaptive revision, and cross-task transfer.
Benchmark Family
Evaluation Paradigm
Supported Learning Claim
LoCoMo Maharana et al. [2024] , LongMemEval Wu et al. [2025a] , PerLTQA Du et al. [2024] , MemoryBank Zhong et al. [2024]
Static dialogue history with single-turn question answering
Long-context factual recall, passive profile extraction, and needle retrieval
MemBench Tan et al. [2025a]
Multi-scenario synthetic memory evaluation
Access efficiency, query latency, and retrieval accuracy across storage scales
MemoryAgentBench Hu et al. [2026b]
Incremental multi-turn ingestion with evolving facts
Dynamic knowledge assimilation, working-memory updates, and explicit conflict resolution
LifelongAgentBench Zheng et al. [2025] , CL-Bench Asawa et al. [2026] , AgentCL Shu et al. [2026]
Sequential interactive environments with task feedback
Cumulative experience accumulation, behavioral retention, and stateful margin ( ΔPt ) over reset baselines
Table 2: Categorization of representative agent memory benchmarks by evaluation paradigm. Benchmarks are organized by the extent to which their experimental design supports genuine learning-from-experience claims rather than static information retrieval.
Figure 6: Matched comparison framework for isolating the contribution of agent memory. The stateful condition retains accumulated experience, whereas the reset condition removes experience-related state on a matched task. ΔPt=Pstateful(t)−Preset(t) is the additional benefit associated with accumulated state. No-memory, full-history, static RAG, and matched-compute baselines progressively rule out alternative explanations.
Figure 7: Theoretical illustration of joint scaling and optimal resource allocation. (a) Fixed-capacity error curves and their optimized lower envelope. (b) Returns at equal total budgets. (c) Optimal capacity and reading budgets. (d) Equal-error contours and the optimal path. For normalized capacity b=Bˉ/B0 and reading r=R/R0 , curves use ϵex(b,r)=0.6b−0.55+0.4r−0.75 and C=b+r , with other resources non-binding. Filled markers sample this function; hollow diamonds mark its calculated optima.
Figure 8: Measured performance across capacity, reading, and experience settings. (a–b) Stored text and Raw RAG performance on LoCoMo and LongMemEval. (c) Paired gains from increasing LoCoMo reading from 512 to 2,048 tokens; error bars in (a–c) are cluster-bootstrap 95% intervals. (d) Stateful–reset score differences on fixed IdeaBench questions at successive checkpoints, without periodic consolidation; shading spans the min–max range across three random training-dialogue orderings. Continuous horizontal axes report text-token volumes; (c) indexes retained-history fractions, and capacity labels in (d) denote upper limits.
50 fixed target queries evaluated across sequential training checkpoints (3 random task orderings)
Stateful–reset performance margin ( ΔPt ) without consolidation
LoCoMo Query Economics
Cohort of 9 conversations, 180 questions; comparing direct read, search, and search + summarize
F1 score, serialized context tokens, and end-to-end query latency
Table 3: Experimental comparisons in this section. Repeated predictions across budgets are aggregated within the stated sampling units. LoCoMo scores use categories 1–4.
Figure 9: Theoretical illustration of experience structure, capacity requirements, and adaptation. Curves use preset response and recovery functions. (a) Performance as task demand expands with experience, at fixed capacity limits Bˉ . (b) Capacity needed to maintain a target under different experience structures. (c) Recovery after environmental changes, marked by dashed lines; m is the maintenance rate.
Figure 10: Theoretical illustration of reuse and resource-optimal policy choice. (a) Query-cost–performance Pareto frontiers; lighter points denote dominated configurations. (b) Amortization at a common performance target P=0.75 . (c) The best policy at each reuse count and average-cost budget; gray denotes infeasibility, and dashed lines are performance contours. Preset functions specify policy performance and costs in common arbitrary units, with offline consolidation and no ongoing maintenance.
Reader
F1 (%) ↑
95% interval
Input (tokens)
Latency (s) ↓
Read all
4.63
[3.28, 6.02]
729.78
0.154
Search
4.29
[3.15, 5.50]
225.68
0.154
Search + summarize
3.91
[2.81, 5.15]
43.52
0.367
Table 4: Three reading policies over Mem0 memory banks on nine LoCoMo conversations and 180 questions, with three timing repetitions. Memory input includes serialization and formatting. Median latency covers retrieval, query-time summarization, and answering, excluding offline construction. F1 intervals use 10,000 conversation-cluster bootstrap resamples.
Figure 11: Theoretical illustration of the parametric memory scaling relation. (a) Excess loss versus memory capacity at different experience volumes; interference dominates until the load ratio falls below ρ⋆ , after which the capacity term follows a power law. (b) Excess loss versus accumulated experience at fixed capacity for backbone editing (low ρ⋆ ) and a dedicated memory module (high ρ⋆ ); dotted lines mark the critical loads. (c) Capacity required to maintain a target performance for independent facts ( ν=1 ) and shared structure ( ν<1 ). Curves sample the illustrative instance of Equations ( 4.7 )–( 4.9 ) described in the text.
Figure 12: Reported capacity and load effects in published parametric-memory studies. (a) Memory+ value count at a fixed 1.3B configuration Berges et al. [2025] ; dotted lines show the dense model without memory layers. (b) Memorization accuracy versus LoRA rank for three write methods on Qwen3-8B Xu et al. [2026b] . (c) Reported exact match versus stored key–value pairs with eight fixed memory vectors and different numbers of write steps Kuratov et al. [2026] . (d) Reported sequential or batch update horizons across parametric-memory studies Meng et al. [2022] , Meng et al. [2023] , Gupta et al. [2024] , Das et al. [2024] , Wang et al. [2024d] , Wang et al. [2024e] ; MEMORYLLM evaluates recent-context retention across 650,000 context updates under random slot replacement rather than simultaneous retention of all past edits. All values are taken from the original papers without fitting.
Figure 13: Reported effects of the foundation model and write timing on parametric memory. (a) Average domain perplexity (biomedicine, finance, and law) of Qwen2.5 models from 0.5B to 72B, with and without the same 0.5B Memory Decoder Cao et al. [2025] ; arrows mark the reduction at the smallest and largest models, and the dotted line marks the 72B model alone. (b–c) F1 and exact match of TMEM on Qwen3-4B as the context threshold that triggers a parametric write varies, on LoCoMo and LongMemEval-S Ren et al. [2026] ; hollow markers denote the observed maxima. A larger threshold means less frequent writes, not more write compute. All values are taken from the original papers without fitting.
Figure 14: Memory-centric continual learning across application domains. Each row shows how accumulated experience updates memory to support personalized service, digital automation, embodied exploration, or multi-agent collaboration. Return arrows mark experience flowing back into memory across tasks and deployments.
Figure 15: Four open challenges in memory-centric continual learning: (1) credit assignment across prospective, retrospective, and long-term horizons; (2) the stability–plasticity dilemma driven by memory interference; (3) a bridge connecting episodic experience with parametric weights; and (4) a lifecycle pipeline of deployment risks and governance rules. Colors distinguish the four challenges, and the bottom bar highlights their core questions.
Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these models remain fundamentally limited in continuous operation, experience accumulation, and personalization, capabilities that are central to adaptive intelligence. While continual learning research has long targeted these goals, its historical focus on in-weight learning (IWL), i.e., updating a single model's parameters to absorb new knowledge, has rendered catastrophic forgetting a persistent challenge. Our position is that combining the strengths of In-Weight Learning (IWL) and the newly emerged capabilities of In-Context Learning (ICL) through the design of modular memory is the missing piece for continual adaptation at scale. We outline a conceptual framework for modular memory-centric architectures that leverage ICL for rapid adaptation and knowledge accumulation, and IWL for stable updates to model capabilities, charting a practical roadmap toward continually learning agents.
Vaggelis Dorovatas, Malte Schwerin, Andrew D. Bagdanov +21
Toyota Motor Europe · University of Bremen · University of Florence +17
Memory-augmented LLM agents offer an appealing shortcut to continual learning: rather than updating model parameters, they accumulate experience in external memory, seemingly sidestepping the stability-plasticity dilemma of parametric learning. We show that this challenge does not disappear but resurfaces at the memory level. Under a limited context window, old and new experiences compete during retrieval, relocating the continual-learning bottleneck from parameter updates to memory access. To study this phenomenon, we introduce a (k,v) framework that disentangles two fundamental design axes of external memory: how experience is represented and how it is organized for retrieval. Across sequential-task experiments in ALFWorld and BabyAI, we find that abstract procedural memories transfer more reliably than detailed trajectories, while negative transfer disproportionately harms the hard cases. Moreover, finer-grained memory organization is not universally beneficial: designs that yield strong forward transfer can simultaneously induce severe forgetting. Together, these results reveal that external memory does not resolve the continual-learning problem; it reshapes it into a problem of memory representation and retrieval design.
Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on textual reasoning, multimodal perception, and open-world interaction demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and show that the stability--plasticity trade-off can be explicitly adjusted.
Borui Kang, Jinrui Gu, Junhan Lv +3
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia