Are Self-Evolving Agents Continual Learners?

Xiaoyin Chen; Yizheng Huang; Yinglun Xu; Ziyun Zhao; Emmett McQuinn; Julie Liu; Hope Yao

cs.AIOct 19, 2026

Are Self-Evolving Agents Continual Learners?

Authors: Xiaoyin Chen, Yizheng Huang, Yinglun Xu, Ziyun Zhao, Emmett McQuinn, Julie Liu, Hope Yao

Organizations: Mila – Quebec AI Institute · Université de Montréal · Work done as a student researcher at Google DeepMind · Google DeepMind

Abstract

Large language model agents are often deployed in environments where new knowledge, tools, and workflows keep arriving. Such an agent is therefore expected to adapt continually, learning new tasks as they arrive without forgetting earlier ones. This resembles the continual learning problem, but at a pace that conventional gradient updates cannot sustain. Nonparametric self-evolving agents are a natural alternative: they adapt by rewriting their own harness, such as prompts, memory, and skills, from task feedback while the model weights stay frozen. We study whether such agents are effective continual learners across four drifting task streams that shift in topic, workflow, or domain (built from BIRD, BFCL, and ScienceWorld). Evaluating three representative methods spanning experience retrieval and self-modification (ReasoningBank, GEPA, and Meta-Harness), we find that although self-evolution improves aggregate performance on most streams, higher plasticity consistently comes at the cost of stability. Specifically, self-modifying methods apply global edits to a shared prompt or workspace, interfering with unseen tasks and overwriting earlier knowledge as the stream shifts. Conversely, similarity-based memory retrieval localizes updates across distant domains, but still retrieves foreign memories on unseen tasks and suffers from retrieval collision when similar queries require conflicting workflows. These findings show that current self-evolving agents remain incomplete continual learners.

Figures & tables

			ReasoningBank				GEPA				Meta-Harness
Stream	Task group	Seed	Term.	$\bar{\Delta}$	$P$	$S$	Term.	$\bar{\Delta}$	$P$	$S$	Term.	$\bar{\Delta}$	$P$	$S$
Topic	BIRD-F1	$52.5$	$0.0$	$0.5$	$1.5$	$-1.5$	$-2.0$	$3.0$	$7.5$	$-9.5$	$-3.5$	$-4.5$	$-4.5$	$0.0$
	BIRD-Code	$61.8$	$-0.5$	$-0.2$	$-1.0$	$0.0$	$0.5$	$0.0$	$1.0$	$-2.0$	$1.5$	$2.0$	$1.0$	$0.0$
	BIRD-TP	$43.2$	$1.5$	$-0.7$	$1.5$	$-1.8$	$-3.5$	$-3.2$	$-3.5$	$-3.0$	$-2.5$	$-1.5$	$-2.5$	$-1.0$
	Average	$52.5$	$\mathbf{0.3}$	$-0.1$	$0.7$	$-1.1$	$-1.7$	$-0.1$	$\mathbf{1.7}$	$-4.8$	$-1.5$	$-1.3$	$-2.0$	$\mathbf{-0.3}$
Workflow	BFCL-Miss	$53.8$	$4.5$	$8.5$	$12.5$	$-8.0$	$2.5$	$9.0$	$15.5$	$-13.0$	$11.5$	$10.5$	$9.5$	$0.0$

Table 1: Continual learning performance across the four drift scenarios on Gemini 3.6 Flash. Each stream is listed by its ordered task groups, followed by the per-stream Average and the four-stream Overall average . Seed reports baseline accuracy

B_{k}

(%); method columns report terminal advantage ( Term. ), average advantage (

\bar{\Delta}

, Eq. 8 ), within-block plasticity (

P

, Eq. 9 ), and cross-block stability (

S=-(I_{k}+F_{k})

, Eq. 11 ), all in percentage points (pp.) over

B_{k}

. Summary rows are shaded green for gains and red for losses, with the best method per metric in bold; the separate

-I_{k}

and

-F_{k}

decomposition is in Table 3 (Appendix B ).

Figure 1: Per-group learning dynamics across the four drift scenarios (columns) and three methods (rows), plotted as the advantage over each method’s seed harness,

A[t,k]-B_{k}

(pp.). Each group has one color, used for its curve and for the wash of the block in which it is trained.

Figure 2: Plasticity

P

against stability

S

across the four drift scenarios. Left: per-scenario scatter (one point per method). Right: overall correlation across all four scenarios and three methods (

n=12

; dashed line and shaded band show the linear least-squares fit and

95\%

confidence interval).

Figure 3: Mechanisms of post-block forgetting under self-modification. (a,b) Share of GEPA’s strategy prompt characters attributed to the block in which each line was first written on topic drift (a) and compound drift (b) (numbers above each panel give total strategy length; full four-stream comparison in Figure 8 , Appendix B ). (c) Percentage of BFCL-Miss evaluation episodes in which the agent withholds tool calls and asks for clarification before the BFCL-Miss block, after the BFCL-Miss block, and after the subsequent BFCL-Base block.

Figure 4: Mechanisms of interference and collision under ReasoningBank’s similarity-based retrieval. (a) Share (%) of evaluation episodes on compound drift whose retrieved memory came from a foreign task group (each row tinted in the evaluated group’s color, darker with the foreign share; empty cells are $0\%$ ; dashed lines separate training blocks). Before a group’s block begins, $100\%$ of retrievals come from earlier blocks; at position 200, $52\%$ of BFCL-Miss and $46\%$ of BFCL-Base episodes retrieve a memory from the opposite BFCL workflow. (b) Mean nearest-match query cosine similarity across the six task groups (dashed box: BIRD databases; red box: BFCL-Miss and BFCL-Base , whose cross-group similarity matches their within-group similarity).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Schedule	Term.	$\bar{\Delta}$	BIRD-Code	BIRD-F1	BFCL-Miss	BFCL-Base
ReasoningBank	Blocked	$+2.8$	$+1.5$	$-1.0$	$+4.5$	$+3.8$	$+3.8$
	Mix	$+2.3$	$+1.2$	$-2.5$	$+2.0$	$+7.2$	$+2.5$
GEPA	Blocked	$+5.8$	$+3.1$	$+6.0$	$+0.0$	$+12.0$	$+5.0$
	Mix	$+0.9$	$+0.4$	$-1.0$	$-4.5$	$+8.7$	$+0.2$
Meta-Harness	Blocked	$+0.8$	$+0.2$	$+1.5$	$-0.5$	$+3.5$	$-1.5$
	Mix	$+1.9$	$+0.5$	$+1.5$	$-3.0$	$+5.2$	$+3.8$

Appendix

Table 2: Comparison between the blocked drifting schedule and the uniformly interleaved Mix schedule on the four-group compound stream (

T=200

), reported in percentage points (pp.) over each method’s seed harness

B_{k}

Figure 5: GEPA strategy prompt composition ( left ) and per-group advantage over the seed ( right ) on the interleaved compound Mix stream. Rules not specific to any group are counted as generic.

			ReasoningBank				GEPA				Meta-Harness
Stream	Task group (block)	$B_{k}$	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.	$P_{k}$	$-I_{k}$	$-F_{k}$	Term.
Topic	BIRD-F1 ( $0\!\to\!50$ )	$52.5$	$+1.5$	—	$-1.5$	$+0.0$	$+7.5$	—	$-9.5$	$-2.0$	$-4.5$	—	$-0.0$	$-3.5$
	BIRD-Code ( $50\!\to\!100$ )	$61.8$	$-1.0$	$-0.0$	$-0.0$	$-0.5$	$+1.0$	$-1.5$	$-0.5$	$+0.5$	$+1.0$	$-0.0$	$-0.0$	$+1.5$
	BIRD-TP ( $100\!\to\!150$ )	$43.2$	$+1.5$	$-1.8$	—	$+1.5$	$-3.5$	$-3.0$	—	$-3.5$	$-2.5$	$-1.0$	—	$-2.5$
Workflow	BFCL-Miss ( $0\!\to\!50$ )	$53.8$	$+12.5$	—	$-8.0$	$+4.5$	$+15.5$	—	$-13.0$	$+2.5$	$+9.5$	—	$-0.0$	$+11.5$
	BFCL-Base ( $50\!\to\!100$ )	$77.3$	$+4.5$	$-1.5$	—	$+4.5$	$+6.5$	$-0.0$	—	$+6.5$	$-2.0$	$-1.0$	—	$-2.0$

Appendix

Table 3: Per-group continual learning decomposition across the four drifting streams.

B_{k}

is the mean seed accuracy (%);

P_{k}

is within-block plasticity at block end

\tau_{k}

;

-I_{k}

is pre-block interference prior to

\sigma_{k}

;

-F_{k}

is post-block forgetting from

\tau_{k}

T

; and Term. is terminal advantage

A[T,k]-B_{k}

(all in pp.). An em-dash marks a structurally undefined cell (

\sigma_{k}=0

for

-I_{k}

;

\tau_{k}=T

for

-F_{k}

Figure 6: Evolved system prompts at position 100 of compound drift (immediately after the two BIRD SQL blocks and before the BFCL-Miss block) for GEPA ( left ) and Meta-Harness ( right ). Highlighted regions mark mandatory SQLite exploration, file-inspection, and interactive testing instructions that are injected into subsequent BFCL-Miss episodes (§ 4.3 ), biasing the agent toward calling exploratory lookup tools rather than asking the user for missing arguments.

Figure 7: GEPA’s evolved strategy at position 150 of domain drift ( left ) and compound drift ( right ), abridged; highlights follow the benchmark palette (BFCL amber, ScienceWorld blue, BIRD green) and italic tags are our annotations. On domain drift, GEPA appends a BFCL-Base section and a BIRD section while keeping all six ScienceWorld sections verbatim (

4{,}372\to 7{,}621

characters). On compound drift, the BFCL-Miss block rewrites sections 1–6 and condenses the seven BIRD sections (

3{,}386

characters at position 100) into a single

1{,}034

-character section 7.

Figure 8: GEPA strategy composition ( top ; share of strategy characters, each line attributed to the block in which it was first written, with carried-over or edited lines keeping their original group; numbers above each panel give total strategy length) and per-group advantage over the seed ( bottom ) on the four drifting streams. Shading marks training blocks.

Mean pairwise cosine similarity
Group	BIRD-F1	BIRD-Code	BIRD-TP	BFCL-Miss	BFCL-Base	ScienceWorld
BIRD-F1	0.660	0.560	0.550	0.527	0.532	0.528
BIRD-Code	0.560	0.644	0.575	0.541	0.543	0.527
BIRD-TP	0.550	0.575	0.707	0.536	0.535	0.545
BFCL-Miss	0.527	0.541	0.536	0.666	0.667	0.568
BFCL-Base	0.532	0.543	0.535	0.667	0.664	0.570

Appendix

Table 4: Query-embedding cosine similarity across the six task groups under ReasoningBank (

50

training tasks per group; diagonal entries exclude self-matches). Top: mean pairwise cosine similarity

\mathbb{E}_{i\in g_{1},\,j\in g_{2}}[\cos(q_{i},q_{j})]

. Bottom: mean nearest-match cosine similarity

\mathbb{E}_{i\in g_{1}}[\max_{j\in g_{2}}\cos(q_{i},q_{j})]

Figure 9: Share (%) of ReasoningBank evaluation episodes whose injected memory entry was written under a task group other than the one evaluated, across the four drifting streams and Compound Mix . Each row is tinted in the color of the evaluated group, darker with the foreign share; empty cells are

0\%

. Dashed lines separate training blocks on the drifting streams.

Parameter	Value
temperature	$1.0$
max_output_tokens	$8192$
thinking_level	MEDIUM

Appendix

Table 5: Generation parameters.

Continual Learning for LLM Agents

Continual learning for LLM agents studies how agents adapt from experience while retaining skills and knowledge across tasks.

See all 199 papers

Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation
May 10, 2026Ye Yu, Xiaopeng Yuan, Haibo Jin +3Continual Learning for LLM Agents LLM Agent Self-Improvement
University of Illinois Urbana-Champaign, IL, USA
AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?
Jul 31, 2026Dong Yan, Jian Liang, Dapeng Hu +4LLM Agent Evaluation AI Agent Benchmarks
School of Artificial Intelligence, University of Chinese Academy of Sciences · Microsoft · Institute of Automation, Chinese Academy of Sciences +1
ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?
Oct 6, 2026Haizhong Zheng, Yizhuo Di, Ranajoy Sadhukhan +2Continual Learning for LLM Agents LLM Agent Evaluation
Carnegie Mellon University · Amazon

May 10, 2026J/K move · Enter open · S save

Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation

Ye Yu, Xiaopeng Yuan, Haibo Jin, Heming Liu+2

University of Illinois Urbana-Champaign, IL, USA

Continual Learning for LLM Agents LLM Agent Self-Improvement Self-Evolving Agents

Open paper AskPDF arXiv

Recent advances in LLM agents enable systems that autonomously refine workflows, accumulate reusable skills, self-train their underlying models, and maintain persistent memory. However, we show that such self-evolution is often non-monotonic: adapting to new task distributions can progressively degrade previously acquired capabilities across all major evolution channels. We identify this phenomenon as capability erosion under self-evolution and show that it consistently emerges across workflow, skill, model, and memory evolution. To mitigate this issue, we propose Capability-Preserving Evolution (CPE), a general stabilization principle that constrains destructive capability drift during continual adaptation. Across all four evolution dimensions, CPE consistently improves retained capability stability while preserving adaptation performance. For example, in workflow evolution, CPE improves retained simple-task performance from 41.8% to 52.8% under GPT-5.1 optimization while simultaneously achieving stronger complex-task adaptation. Our findings suggest that stable long-horizon self-evolving agents require not only acquiring new capabilities, but also explicitly preserving previously learned ones during continual adaptation.

Abstract

Continual Learning for LLM Agents

Explore similar work

Do Self-Evolving Agents Forget? Capability Degradation and Preservation in Lifelong LLM Agent Adaptation

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?