Are Self-Evolving Agents Continual Learners?

Xiaoyin Chen; Yizheng Huang; Yinglun Xu; Ziyun Zhao; Emmett McQuinn; Julie Liu; Hope Yao

cs.AIOct 19, 2026

Are Self-Evolving Agents Continual Learners?

Authors: Xiaoyin Chen, Yizheng Huang, Yinglun Xu, Ziyun Zhao, Emmett McQuinn, Julie Liu, Hope Yao

Organizations: Mila – Quebec AI Institute · Université de Montréal · Work done as a student researcher at Google DeepMind · Google DeepMind

Abstract

Large language model agents are often deployed in environments where new knowledge, tools, and workflows keep arriving. Such an agent is therefore expected to adapt continually, learning new tasks as they arrive without forgetting earlier ones. This resembles the continual learning problem, but at a pace that conventional gradient updates cannot sustain. Nonparametric self-evolving agents are a natural alternative: they adapt by rewriting their own harness, such as prompts, memory, and skills, from task feedback while the model weights stay frozen. We study whether such agents are effective continual learners across four drifting task streams that shift in topic, workflow, or domain (built from BIRD, BFCL, and ScienceWorld). Evaluating three representative methods spanning experience retrieval and self-modification (ReasoningBank, GEPA, and Meta-Harness), we find that although self-evolution improves aggregate performance on most streams, higher plasticity consistently comes at the cost of stability. Specifically, self-modifying methods apply global edits to a shared prompt or workspace, interfering with unseen tasks and overwriting earlier knowledge as the stream shifts. Conversely, similarity-based memory retrieval localizes updates across distant domains, but still retrieves foreign memories on unseen tasks and suffers from retrieval collision when similar queries require conflicting workflows. These findings show that current self-evolving agents remain incomplete continual learners.

Figures & tables

			ReasoningBank				GEPA				Meta-Harness
Stream	Task group	Seed	Term.	$\bar{\Delta}$	$P$	$S$	Term.	$\bar{\Delta}$	$P$	$S$	Term.	$\bar{\Delta}$	$P$	$S$
Topic	BIRD-F1	$52.5$	$0.0$	$0.5$	$1.5$	$-1.5$	$-2.0$	$3.0$	$7.5$	$-9.5$	$-3.5$	$-4.5$	$-4.5$	$0.0$
	BIRD-Code	$61.8$	$-0.5$	$-0.2$	$-1.0$	$0.0$	$0.5$	$0.0$	$1.0$	$-2.0$	$1.5$	$2.0$	$1.0$	$0.0$
	BIRD-TP	$43.2$	$1.5$	$-0.7$	$1.5$	$-1.8$	$-3.5$	$-3.2$	$-3.5$	$-3.0$	$-2.5$	$-1.5$	$-2.5$	$-1.0$
	Average	$52.5$	$\mathbf{0.3}$	$-0.1$	$0.7$	$-1.1$	$-1.7$	$-0.1$	$\mathbf{1.7}$	$-4.8$	$-1.5$	$-1.3$	$-2.0$	$\mathbf{-0.3}$
Workflow	BFCL-Miss	$53.8$	$4.5$	$8.5$	$12.5$	$-8.0$	$2.5$	$9.0$	$15.5$	$-13.0$	$11.5$	$10.5$	$9.5$	$0.0$

Table 1: Continual learning performance across the four drift scenarios on Gemini 3.6 Flash. Each stream is listed by its ordered task groups, followed by the per-stream Average and the four-stream Overall average . Seed reports baseline accuracy

B_{k}

(%); method columns report terminal advantage ( Term. ), average advantage (

\bar{\Delta}

, Eq. 8 ), within-block plasticity (

P

, Eq. 9 ), and cross-block stability (

S=-(I_{k}+F_{k})

, Eq. 11 ), all in percentage points (pp.) over

B_{k}

. Summary rows are shaded green for gains and red for losses, with the best method per metric in bold; the separate

-I_{k}

and

-F_{k}

decomposition is in Table 3 (Appendix B ).

Figure 1: Per-group learning dynamics across the four drift scenarios (columns) and three methods (rows), plotted as the advantage over each method’s seed harness,

A[t,k]-B_{k}

(pp.). Each group has one color, used for its curve and for the wash of the block in which it is trained.

Figure 2: Plasticity

P

against stability

S

across the four drift scenarios. Left: per-scenario scatter (one point per method). Right: overall correlation across all four scenarios and three methods (

n=12

; dashed line and shaded band show the linear least-squares fit and

95\%

confidence interval).

Figure 3: Mechanisms of post-block forgetting under self-modification. (a,b) Share of GEPA’s strategy prompt characters attributed to the block in which each line was first written on topic drift (a) and compound drift (b) (numbers above each panel give total strategy length; full four-stream comparison in Figure 8 , Appendix B ). (c) Percentage of BFCL-Miss evaluation episodes in which the agent withholds tool calls and asks for clarification before the BFCL-Miss block, after the BFCL-Miss block, and after the subsequent BFCL-Base block.

Figure 4: Mechanisms of interference and collision under ReasoningBank’s similarity-based retrieval. (a) Share (%) of evaluation episodes on compound drift whose retrieved memory came from a foreign task group (each row tinted in the evaluated group’s color, darker with the foreign share; empty cells are $0\%$ ; dashed lines separate training blocks). Before a group’s block begins, $100\%$ of retrievals come from earlier blocks; at position 200, $52\%$ of BFCL-Miss and $46\%$ of BFCL-Base episodes retrieve a memory from the opposite BFCL workflow. (b) Mean nearest-match query cosine similarity across the six task groups (dashed box: BIRD databases; red box: BFCL-Miss and BFCL-Base , whose cross-group similarity matches their within-group similarity).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Schedule	Term.	$\bar{\Delta}$	BIRD-Code	BIRD-F1	BFCL-Miss	BFCL-Base
ReasoningBank	Blocked	$+2.8$	$+1.5$	$-1.0$	$+4.5$	$+3.8$	$+3.8$
	Mix	$+2.3$	$+1.2$	$-2.5$	$+2.0$	$+7.2$	$+2.5$
GEPA	Blocked	$+5.8$	$+3.1$	$+6.0$	$+0.0$	$+12.0$	$+5.0$
	Mix	$+0.9$	$+0.4$	$-1.0$	$-4.5$	$+8.7$	$+0.2$
Meta-Harness	Blocked	$+0.8$	$+0.2$	$+1.5$	$-0.5$	$+3.5$	$-1.5$
	Mix	$+1.9$	$+0.5$	$+1.5$	$-3.0$	$+5.2$	$+3.8$

Appendix

Table 2: Comparison between the blocked drifting schedule and the uniformly interleaved Mix schedule on the four-group compound stream (

T=200

), reported in percentage points (pp.) over each method’s seed harness

B_{k}

Figure 5: GEPA strategy prompt composition ( left ) and per-group advantage over the seed ( right ) on the interleaved compound Mix stream. Rules not specific to any group are counted as generic.

			ReasoningBank				GEPA				Meta-Harness
Stream	Task group (block)	$B_{k}$	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.
Topic	BIRD-F1 ( $0\!\to\!50$ )	$52.5$	$+1.5$	—	$-1.5$	$+0.0$	$+7.5$	—	$-9.5$	$-2.0$	$-4.5$	—	$-0.0$	$-3.5$
	BIRD-Code ( $50\!\to\!100$ )	$61.8$	$-1.0$	$-0.0$	$-0.0$	$-0.5$	$+1.0$	$-1.5$	$-0.5$	$+0.5$	$+1.0$	$-0.0$	$-0.0$	$+1.5$
	BIRD-TP ( $100\!\to\!150$ )	$43.2$	$+1.5$	$-1.8$	—	$+1.5$	$-3.5$	$-3.0$	—	$-3.5$	$-2.5$	$-1.0$	—	$-2.5$
Workflow	BFCL-Miss ( $0\!\to\!50$ )	$53.8$	$+12.5$	—	$-8.0$	$+4.5$	$+15.5$	—	$-13.0$	$+2.5$	$+9.5$	—	$-0.0$	$+11.5$
	BFCL-Base ( $50\!\to\!100$ )	$77.3$	$+4.5$	$-1.5$	—	$+4.5$	$+6.5$	$-0.0$	—	$+6.5$	$-2.0$	$-1.0$	—	$-2.0$

Appendix

Table 3: Per-group continual learning decomposition across the four drifting streams.

B_{k}

is the mean seed accuracy (%);

P_{k}

is within-block plasticity at block end

\tau_{k}

;

-I_{k}

is pre-block interference prior to

\sigma_{k}

;

-F_{k}

is post-block forgetting from

\tau_{k}

T

; and Term. is terminal advantage

A[T,k]-B_{k}

(all in pp.). An em-dash marks a structurally undefined cell (

\sigma_{k}=0

for

-I_{k}

;

\tau_{k}=T

for

-F_{k}

Figure 6: Evolved system prompts at position 100 of compound drift (immediately after the two BIRD SQL blocks and before the BFCL-Miss block) for GEPA ( left ) and Meta-Harness ( right ). Highlighted regions mark mandatory SQLite exploration, file-inspection, and interactive testing instructions that are injected into subsequent BFCL-Miss episodes (§ 4.3 ), biasing the agent toward calling exploratory lookup tools rather than asking the user for missing arguments.

Figure 7: GEPA’s evolved strategy at position 150 of domain drift ( left ) and compound drift ( right ), abridged; highlights follow the benchmark palette (BFCL amber, ScienceWorld blue, BIRD green) and italic tags are our annotations. On domain drift, GEPA appends a BFCL-Base section and a BIRD section while keeping all six ScienceWorld sections verbatim (

4{,}372\to 7{,}621

characters). On compound drift, the BFCL-Miss block rewrites sections 1–6 and condenses the seven BIRD sections (

3{,}386

characters at position 100) into a single

1{,}034

-character section 7.

Figure 8: GEPA strategy composition ( top ; share of strategy characters, each line attributed to the block in which it was first written, with carried-over or edited lines keeping their original group; numbers above each panel give total strategy length) and per-group advantage over the seed ( bottom ) on the four drifting streams. Shading marks training blocks.

Mean pairwise cosine similarity
Group	BIRD-F1	BIRD-Code	BIRD-TP	BFCL-Miss	BFCL-Base	ScienceWorld
BIRD-F1	0.660	0.560	0.550	0.527	0.532	0.528
BIRD-Code	0.560	0.644	0.575	0.541	0.543	0.527
BIRD-TP	0.550	0.575	0.707	0.536	0.535	0.545
BFCL-Miss	0.527	0.541	0.536	0.666	0.667	0.568
BFCL-Base	0.532	0.543	0.535	0.667	0.664	0.570

Appendix

Table 4: Query-embedding cosine similarity across the six task groups under ReasoningBank (

50

training tasks per group; diagonal entries exclude self-matches). Top: mean pairwise cosine similarity

\mathbb{E}_{i\in g_{1},\,j\in g_{2}}[\cos(q_{i},q_{j})]

. Bottom: mean nearest-match cosine similarity

\mathbb{E}_{i\in g_{1}}[\max_{j\in g_{2}}\cos(q_{i},q_{j})]

Figure 9: Share (%) of ReasoningBank evaluation episodes whose injected memory entry was written under a task group other than the one evaluated, across the four drifting streams and Compound Mix . Each row is tinted in the color of the evaluated group, darker with the foreign share; empty cells are

0\%

. Dashed lines separate training blocks on the drifting streams.

Parameter	Value
temperature	$1.0$
max_output_tokens	$8192$
thinking_level	MEDIUM

Appendix

Table 5: Generation parameters.

Continual Learning for LLM Agents

Continual learning for LLM agents studies how agents adapt from experience while retaining skills and knowledge across tasks.

See all 199 papers

May 10, 2026

Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation

Recent advances in LLM agents enable systems that autonomously refine workflows, accumulate reusable skills, self-train their underlying models, and maintain persistent memory. However, we show that such self-evolution is often non-monotonic: adapting to new task distributions can progressively degrade previously acquired capabilities across all major evolution channels. We identify this phenomenon as capability erosion under self-evolution and show that it consistently emerges across workflow, skill, model, and memory evolution. To mitigate this issue, we propose Capability-Preserving Evolution (CPE), a general stabilization principle that constrains destructive capability drift during continual adaptation. Across all four evolution dimensions, CPE consistently improves retained capability stability while preserving adaptation performance. For example, in workflow evolution, CPE improves retained simple-task performance from 41.8% to 52.8% under GPT-5.1 optimization while simultaneously achieving stronger complex-task adaptation. Our findings suggest that stable long-horizon self-evolving agents require not only acquiring new capabilities, but also explicitly preserving previously learned ones during continual adaptation.

Ye Yu, Xiaopeng Yuan, Haibo Jin +3

University of Illinois Urbana-Champaign, IL, USA

Continual Learning for LLM Agents LLM Agent Self-Improvement Self-Evolving Agents

Ask PDF

Jul 31, 2026

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominantly adopt independent evaluation. Consequently, the behavior of self-evolving agents in realistic streaming settings, where agents adapt to diverse and complex task streams, remains poorly understood. To address this gap, we introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components by organizing agentic benchmarks into a configurable task stream and instantiating the Isolated, Sequential, and Interleaved streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and is non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.

Dong Yan, Jian Liang, Dapeng Hu +4

School of Artificial Intelligence, University of Chinese Academy of Sciences · Microsoft · Institute of Automation, Chinese Academy of Sciences +1

LLM Agent Evaluation AI Agent Benchmarks LLM Agent Reliability Self-Evolving Agents

Ask PDF

Oct 6, 2026

ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?

Large language model agents are increasingly deployed to perform complex tasks in real-world environments. However, the knowledge required for correct behavior in these environments is often implicit, undisclosed, and subject to change over time. Recent continual-learning harnesses seek to address this challenge by enabling agents to improve from serving experience. Yet the effectiveness and limitations of these methods are not yet well characterized. Existing benchmarks provide only partial coverage: some explicitly provide the target knowledge, others assume a static environment, and those that support continual adaptation remain limited in scale and knowledge diversity. To enable systematic evaluation, we formalize an evolving-environment streaming dataset (EESD), in which agents must infer, apply, and revise latent environment knowledge from interaction and outcome feedback as hidden policies evolve, and introduce ServeLearnBench, spanning retail support, banking, and sales-pitch generation with 53 environment windows and 7,718 tasks. We evaluate five learning harnesses (RAG, Mem0, SkillOpt, Continual Harness, and Prime) across six models (GPT-5.6 Terra, Opus 5, Kimi K3, GLM-5.3, DeepSeek V4.1 Flash, and GLM-5.3 Flash), covering 28 model-harness pairs and 252 learning runs. Our evaluation reveals three main findings: a substantial gap remains between task capability and learning from experience; continual adaptation is costly and can degrade already-correct behavior; and insufficient exploration emerges as a key bottleneck to effective adaptation. Overall, ServeLearnBench provides a controlled testbed for diagnosing these limitations and tracking progress toward agents that continually and reliably improve through serving experience.

Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2

Carnegie Mellon University · Amazon

Continual Learning for LLM Agents LLM Agent Evaluation LLM Agent Self-Improvement AI Agent Benchmarks

Ask PDF

Abstract

Continual Learning for LLM Agents

Explore similar work

Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?