When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents

Tao Yu; Zhongtian Luo; Yuchen Long; Yiheng Lu; Yuxuan Zhou; Yanpei Gong; Siyi Cao; Yuqing Wu

cs.LGOct 14, 2026

When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents

Authors: Tao Yu, Zhongtian Luo, Yuchen Long, Yiheng Lu, Yuxuan Zhou, Yanpei Gong, Siyi Cao, Yuqing Wu,

Organizations: JD Future Academy · CASIA · UCAS · Tsinghua University ·

Abstract

Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rules and affect later tasks. Existing evaluations largely examine clean learning or perturbations encountered after skills are frozen, leaving robustness during self-evolution insufficiently understood. We introduce RSE-Bench, to our knowledge the first benchmark for evaluating skill self-evolution under ordinary, non-adversarial noise. It spans four domains and six benchmarks, with controlled perturbations to instructions, environmental evidence, execution records, and outcome--record pairings that preserve the underlying tasks and native scoring rules. The protocol separates noise during learning from input-side noise at evaluation, testing frozen skill stores on held-out tasks. We also introduce RSEClaw, a method-agnostic framework supporting consistent noise injection and process diagnostics of actual exposure and skill updates. Across three methods and two backbones, noisy learning retains only $25.5\%$ -- $42.1\%$ of the aggregate gain achieved by clean self-evolution, depending on the noise type. Perturbations confined to execution records and outcome--record pairings cause average losses comparable to input-side noise, despite leaving the original executions and evaluations intact. Diagnostics further identify differences in candidate skill text that coexist with small changes in mean held-out performance.

Figures & tables

Figure 1: Overview of RSE-Bench and RSEClaw : clean tasks from six sources, noise injected at four entry points of the learning loop, and evaluation on held-out tasks with the skill store frozen.

		Noise Injection
Benchmark	Self- Evolution	Injected	Into Learning Loop	Non- Adversarial	Task- Preserving	Injection Positions	Clean Held-out Eval	Pluggable Methods	Metric
GSM-IC [ 12 ]	✗	✓	✗	✓	✓	1	✗	✗	Cap.
AgentNoiseBench [ 18 ]	✗	✓	✗	✓	✓	2	✗	✗	Cap.
StressWeb [ 1 ]	✗	✓	✗	✓	✗	3	✗	✗	Cap.
AgentPoison [ 2 ]	✗	✓	✗	✗	✗	1	✗	✗	ASR
MINJA [ 4 ]	✓	✓	✓	✗	✗	1	✓	✗	ASR

Table 1: Comparison of RSE-Bench with existing benchmarks. Self-Evolution : the benchmark evaluates agents that update persistent skills or memory from experience. Injected : controlled perturbations are injected. Into Learning Loop : injected noise can be distilled into the agent’s persistent skills, rather than affecting only a single episode or frozen artifacts. Non-Adversarial : noise models ordinary deployment imperfections rather than attacks. Task-Preserving : injections keep tasks solvable, with ground truth and scoring unchanged. Injection Positions : number of distinct injection channels, per each paper’s own definition. Clean Held-out Eval : evolved skills are evaluated on held-out tasks untouched by learning. Pluggable Methods : multiple self-evolution methods run behind a unified interface. Metric : capability (Cap.) or attack success rate (ASR).

Method	Model	Reference		Noisy learning				Noisy evaluation
Method	Model	Initial	$C\!\to\!C$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$
RSE-Bench (Avg)
ACE	DeepSeek-V4.1-Flash	$71.25$	$74.53$	$72.64_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.5% $}}$	$69.27_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 7.1% $}}$	$73.08_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.9% $}}$	$71.19_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 4.5% $}}$	$\underline{69.88}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.2% $}}$	$69.67_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.5% $}}$
ACE	GPT-5.2	$61.42$	$63.97$	$63.17_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.3% $}}$	$64.01_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.1% $}}$	$\underline{59.68}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.7% $}}$	$61.54_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.8% $}}$	$61.76_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.4% $}}$	$\underline{58.65}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 8.3% $}}$
RethinkSkill	DeepSeek-V4.1-Flash	$71.25$	$75.93$	$72.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 4.0% $}}$	$71.31_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.1% $}}$	$71.10_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.4% $}}$	$74.32_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.1% $}}$	$72.04_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 5.1% $}}$	$71.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 5.3% $}}$
RethinkSkill	GPT-5.2	$61.42$	$66.93$	$\underline{62.93}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.0% $}}$	$\underline{60.40}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 9.7% $}}$	$64.74_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.3% $}}$	$\underline{61.53}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 8.1% $}}$	$63.57_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 5.0% $}}$	$61.96_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 7.4% $}}$

Table 2: Main results on RSE-Bench (%, means over three independent runs; standard deviations in Table 14 ). Subscripts show relative changes,

100(s-b)/b

, from the corresponding

C\!\to\!C

baseline

b

: green

\downarrow

for decreases, gray

\uparrow

for increases, and

\leftrightarrow

for no change. Columns specify learning

\rightarrow

evaluation;

C

is clean, and Inst, Evid, Rec, and Bind denote instruction, evidence, record, and binding noise (

N_{\mathrm{Inst}}

–

N_{\mathrm{Bind}}

). Scores are macro-averaged equally over all six benchmarks before rates are computed. Underlines mark the largest relative decrease in each noise-condition column across the six method–model combinations, using unrounded rates and including exact ties. Per-benchmark results appear in Table 10 .

	Evaluation condition
Learning	Clean	$N_{\mathrm{Inst}}$	$N_{\mathrm{Evid}}$
Initial	79.58	83.75	82.50
Clean	90.00	77.50	78.75
$N_{\mathrm{Inst}}$	74.58	71.25	–
$N_{\mathrm{Evid}}$	74.17	–	70.00

Table 3: Crossed noise conditions for ACE on SpreadsheetBench (DeepSeek-V4.1-Flash; scores in %). Definitions and full results appear in Appendix Table 15 .

Figure 2: Case study: noisy evolution misreads pre-filled rows, overwrites the total formula, and flips the score from 1 to 0.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Reported metric	What it checks	Soft
SpreadsheetBench	hard_restriction $\in\{0,1\}$	cell equality over the declared answer ranges	—
SP-DocVQA	accuracy $\in\{0,1\}$	match against any gold answer string	ANLS
MP-DocVQA	accuracy $\in\{0,1\}$	match against any gold answer string	ANLS
ALFWorld	won $\in\{0,1\}$	goal satisfaction in the simulator	—
SkillFlow	reward $\in\{0,1\}$	exit code of the task’s native test suite	—
ContinualSkillBench	reward $\in[0,1]$	native judge: tests, match, tolerance, or F1	F1 subset

Appendix

Table 4: Reported metric per benchmark. Soft is a partial-credit signal that the harness exposes to a method’s own selection gate but never reports as a result; a benchmark without genuine partial credit reports none rather than a copy of the binary outcome.

Model	Role	Temperature	Max output (tokens)	Timeout (s)	Thinking mode
DeepSeek-V4.1-Flash	actor, optimizer (main, analyses)	$0$	$32{,}768$	$600$	disabled
GPT-5.2	actor, optimizer (main)	default	$32{,}768$	$600$	provider default
GPT-5.5	actor, optimizer (backbone analysis)	default	$32{,}768$	$600$	provider default
Kimi-K2.8	$N_{\mathrm{Inst}}$ / $N_{\mathrm{Evid}}$ generator, reviewer	$0$	$8{,}192$	$180$	disabled

Appendix

Table 5: Model settings. Upper block: evaluated backbones; the actor and the optimizer of a job use the same backbone and the same settings. Lower block: the offline noise-construction model, used for generation and review only. Maximum output and timeout are per request; all models are called through the Chat Completions protocol. For GPT backbones the temperature is left at the provider default.

Method	Update point	Learning input	Reads trajectory	Acceptance
ACE	once per pass	every training episode, per-episode reflection	yes	none (always accept)
SkillOpt	per step (40 / 10) + epoch end	failed and successful trajectories, minibatch 8	yes	strict improvement on selection
RethinkSkill	once per pass	evaluation ledger of every training episode	yes/no	four-branch gate on selection

Appendix

Table 6: Method configurations. Reads trajectory determines whether

N_{\mathrm{Rec}}

has a route into the method’s learning input; all three methods receive

N_{\mathrm{Inst}}

N_{\mathrm{Evid}}

and

N_{\mathrm{Bind}}

through their episodes or outcomes.

Benchmark	Family	Generator sees gold
SP-DocVQA	answer-aware page edit	yes
MP-DocVQA	answer-aware page edit	yes
SpreadsheetBench	workspace evidence	no (hidden)
SkillFlow	workspace evidence	no (hidden)
ContinualSkillBench	workspace evidence	no (hidden)
ALFWorld	observation replay	not applicable

Appendix

Table 7: N-Evid construction families and gold-answer access by benchmark. Relation labels are assigned per group by a seeded hash; the workspace-reference fallback is always related non-decisive.

Invariant	N-Inst	N-Evid	N-Rec	N-Bind
Task goal unchanged	Auto + Model + Human	Auto + Model + Human	Auto (question never read)	Auto (question never read)
Correct evidence reachable	Model + Human	Auto (replay) + Model + Human	Auto (execution untouched)	Auto (execution untouched)
True outcome unchanged	Auto (evaluator untouched)	Auto (evaluator untouched)	Auto (score immutable)	Auto (payloads permuted, not edited)
Scoring semantics unchanged	Auto + Model + Human	Auto + Model + Human	Auto	Auto
No answer leak or hint	Auto + Model + Human	Model + Human	—	—

Appendix

Table 8: Where each validity invariant is enforced. Auto : deterministic checks without model calls; Model : two-sided model review of the final material; Human : sampled re-review of each benchmark’s release by three independent annotators, a sampled variant passing only if all three pass it. N-Rec and N-Bind are deterministic transformations whose invariants are fully decidable, so they need neither model nor human review.

Method	Noise	IER	Selected version ( $v^{*}$ )	CCS [ref]
ACE	$N_{\mathrm{Inst}}$	$100\%$ $(150/150)$	$v2$	$0.764\,[0.814]$
	$N_{\mathrm{Evid}}$	$97\%$ $(146/150)$	$v3$	$0.702\,[0.814]$
	$N_{\mathrm{Rec}}$	$100\%$ $(150/150)$	$v1$	$0.775\,[0.814]$
	$N_{\mathrm{Bind}}$	$21\%$ $(32/150)$	$v2$	$0.727\,[0.814]$
RethinkSkill	$N_{\mathrm{Inst}}$	–	$v1$	$0.820\,[0.901]$
	$N_{\mathrm{Evid}}$	–	$v2$	$0.826\,[0.901]$

Appendix

Table 9: Noise process diagnostics for ACE and RethinkSkill on SpreadsheetBench (DeepSeek-V4.1-Flash). IER is the proportion of fit-task optimizer inputs containing an injected perturbation. Selected version (

v^{*}

) is the skill version selected on the validation set for final testing, with

v0

denoting the initial state. CCS is the mean term-frequency cosine similarity between candidates from a noise condition and candidates from a clean run. The bracketed reference,

\mathrm{CCS}_{\mathrm{ref}}

, is the mean CCS over all

\binom{K}{2}

pairs of

K

independent clean runs for each method, computed using the same cross-run procedure. A dash indicates an undefined or inapplicable quantity. CCS values are comparable within a method.

Method	Model	Reference		Noisy learning				Noisy evaluation
Method	Model	Initial	$C\!\to\!C$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$
SpreadsheetBench
ACE	DeepSeek-V4.1-Flash	$79.58$	$90.00$	$74.58_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 17.1% $}}$	$\underline{74.17}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 17.6% $}}$	$83.33_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 7.4% $}}$	$80.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 11.1% $}}$	$\underline{77.50}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 13.9% $}}$	$\underline{78.75}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 12.5% $}}$
ACE	GPT-5.2	$45.00$	$40.83$	$44.58_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 9.2% $}}$	$40.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.0% $}}$	$42.50_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 4.1% $}}$	$46.67_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 14.3% $}}$	$42.08_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 3.1% $}}$	$37.08_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 9.2% $}}$
RethinkSkill	DeepSeek-V4.1-Flash	$79.58$	$88.33$	$80.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 9.4% $}}$	$78.75_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 10.8% $}}$	$82.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.1% $}}$	$89.17_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 1.0% $}}$	$81.67_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 7.5% $}}$	$81.67_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 7.5% $}}$
RethinkSkill	GPT-5.2	$45.00$	$45.83$	$39.58_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 13.6% $}}$	$40.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 12.7% $}}$	$52.08_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 13.6% $}}$	$\underline{39.17}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 14.5% $}}$	$44.58_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.7% $}}$	$45.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.8% $}}$

Appendix

Table 10: Full results on RSE-Bench (%, means over three independent runs). Subscripts show relative changes,

100(s-b)/b

, from the corresponding

C\!\to\!C

baseline

b

: green

\downarrow

for decreases, gray

\uparrow

for increases, and

\leftrightarrow

for no change. Columns specify learning

\rightarrow

evaluation;

C

is clean, and Inst, Evid, Rec, and Bind denote instruction, evidence, record, and binding noise (

N_{\mathrm{Inst}}

–

N_{\mathrm{Bind}}

). Each benchmark block reports that benchmark’s split directly; the RSE-Bench (Avg) block averages scores equally over all six benchmarks before computing rates. Underlines mark the largest relative decrease in each noise-condition column across the six method–model combinations of its own block, using unrounded rates and including exact ties, with the same rule applied in the Avg block. Per-run standard deviations appear in Table 14 .

Method	Model	Reference		Noisy learning				Noisy evaluation
Method	Model	Initial	$C\!\to\!C$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$
ALFWorld
ACE	DeepSeek-V4.1-Flash	$95.42$	$99.17$	$97.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.3% $}}$	$96.25_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.9% $}}$	$100.00_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.8% $}}$	$97.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.3% $}}$	$\underline{93.75}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 5.5% $}}$	$95.83_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.4% $}}$
ACE	GPT-5.2	$84.17$	$96.25$	$\underline{93.33}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.0% $}}$	$96.67_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.4% $}}$	$\underline{84.17}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 12.6% $}}$	$92.08_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 4.3% $}}$	$94.17_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.2% $}}$	$90.00_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 6.5% $}}$
RethinkSkill	DeepSeek-V4.1-Flash	$95.42$	$99.17$	$97.92_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 1.3% $}}$	$97.08_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 2.1% $}}$	$94.58_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 4.6% $}}$	$100.00_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.8% $}}$	$99.58_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.4% $}}$	$100.00_{\scalebox{0.78}{$ \color[rgb]{0.45,0.45,0.45}\uparrow 0.8% $}}$
RethinkSkill	GPT-5.2	$84.17$	$97.08$	$94.17_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.0% $}}$	$\underline{87.92}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 9.4% $}}$	$85.83_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 11.6% $}}$	$\underline{83.33}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 14.2% $}}$	$93.75_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 3.4% $}}$	$\underline{88.33}_{\scalebox{0.78}{$ \color[rgb]{0,0.5,0}\downarrow 9.0% $}}$

Appendix

Table 11: Full results on RSE-Bench (continued). The notation and reporting conventions are the same as in the first part of Table 10 .

		Clean learning		Noisy learning
Entry point	Materialization	Clean eval	Noisy eval	Clean eval	Noisy eval
N-Inst	data, all partitions	✓ †	✓	✓	✓
N-Evid	data, all partitions	✓ †	✓	✓	✓
N-Rec	run-time seam	✓ †	–	✓	–
N-Bind	run-time seam	✓ †	–	✓	–

Appendix

Table 12: Noise conditions and data of RSE-Bench. N-Inst and N-Evid materialize a paired noisy variant of every task. N-Rec and N-Bind transform the learner’s inputs at run time and add no items. † The clean-learning, clean-evaluation cell is a single baseline shared by all rows. Dashes mark undefined settings: held-out evaluation produces no learner experience.

Domain	Benchmark	Source pool	Train	Sel.	Test
Spreadsheet	SpreadsheetBench	400	50	30	80
Document	SP-DocVQA	50,000	50	30	80
Document	MP-DocVQA	46,436	50	30	80
Interactive	ALFWorld	3,827	50	30	80
Sequential †	SkillFlow	166	50	30	50
Sequential †	ContinualSkillBench	500	50	30	80

Appendix

Table 13: Composition of RSE-Bench. Train, Sel., and Test denote the evolution-train, selection, and final-test partitions, respectively. Noise conditions follow the protocol in Table 12 , with N-Rec and N-Bind applied only during learning. Sequential benchmarks (marked † ) are split by stream position rather than random sampling; ALFWorld draws its partitions from the official train, valid-seen, and valid-unseen splits, respectively.

Method	Model	Reference		Noisy learning				Noisy evaluation
Method	Model	Initial	$C\!\to\!C$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$
SpreadsheetBench
ACE	DeepSeek-V4.1-Flash	$79.58_{\scalebox{0.78}{$ \pm 0.72 $}}$	$90.00_{\scalebox{0.78}{$ \pm 0.00 $}}$	$74.58_{\scalebox{0.78}{$ \pm 1.44 $}}$	$\underline{74.17}_{\scalebox{0.78}{$ \pm 0.72 $}}$	$83.33_{\scalebox{0.78}{$ \pm 1.44 $}}$	$80.00_{\scalebox{0.78}{$ \pm 0.00 $}}$	$\underline{77.50}_{\scalebox{0.78}{$ \pm 0.00 $}}$	$\underline{78.75}_{\scalebox{0.78}{$ \pm 0.00 $}}$
ACE	GPT-5.2	$45.00_{\scalebox{0.78}{$ \pm 2.17 $}}$	$40.83_{\scalebox{0.78}{$ \pm 0.72 $}}$	$44.58_{\scalebox{0.78}{$ \pm 0.72 $}}$	$40.00_{\scalebox{0.78}{$ \pm 2.17 $}}$	$42.50_{\scalebox{0.78}{$ \pm 3.31 $}}$	$46.67_{\scalebox{0.78}{$ \pm 1.44 $}}$	$42.08_{\scalebox{0.78}{$ \pm 1.44 $}}$	$37.08_{\scalebox{0.78}{$ \pm 1.44 $}}$
RethinkSkill	DeepSeek-V4.1-Flash	$79.58_{\scalebox{0.78}{$ \pm 0.72 $}}$	$88.33_{\scalebox{0.78}{$ \pm 0.72 $}}$	$80.00_{\scalebox{0.78}{$ \pm 1.25 $}}$	$78.75_{\scalebox{0.78}{$ \pm 0.00 $}}$	$82.92_{\scalebox{0.78}{$ \pm 0.72 $}}$	$89.17_{\scalebox{0.78}{$ \pm 3.15 $}}$	$81.67_{\scalebox{0.78}{$ \pm 0.72 $}}$	$81.67_{\scalebox{0.78}{$ \pm 1.91 $}}$
RethinkSkill	GPT-5.2	$45.00_{\scalebox{0.78}{$ \pm 2.17 $}}$	$45.83_{\scalebox{0.78}{$ \pm 0.72 $}}$	$39.58_{\scalebox{0.78}{$ \pm 1.44 $}}$	$40.00_{\scalebox{0.78}{$ \pm 0.00 $}}$	$52.08_{\scalebox{0.78}{$ \pm 2.89 $}}$	$\underline{39.17}_{\scalebox{0.78}{$ \pm 4.02 $}}$	$44.58_{\scalebox{0.78}{$ \pm 0.72 $}}$	$45.00_{\scalebox{0.78}{$ \pm 1.25 $}}$

Appendix

Table 14: Run-to-run variability on RSE-Bench. Scores (%) are shown as

\mathrm{mean}_{\pm\mathrm{SD}}

over three independent runs of the complete evolution and evaluation pipeline, with 50 held-out tasks for SkillFlow and 80 for each other benchmark. Columns specify learning

\rightarrow

evaluation;

C

is clean, and Inst, Evid, Rec, and Bind denote instruction, evidence, record, and binding noise. Underlines match the main table: within each benchmark and noise-condition column, they mark the largest relative decrease from the corresponding

C\!\to\!C

baseline across the six method–model combinations, using unrounded rates and including exact ties.

Reference				Noisy learning				Noisy evaluation
$\mathrm{Initial}\!\to\!C$	$C\!\to\!C$	$\mathrm{Initial}\!\to\!N_{\mathrm{Inst}}$	$\mathrm{Initial}\!\to\!N_{\mathrm{Evid}}$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$	$N_{\mathrm{Inst}}\!\to\!N_{\mathrm{Inst}}$	$N_{\mathrm{Evid}}\!\to\!N_{\mathrm{Evid}}$
79.58	90.00	83.75	82.50	74.58	74.17	83.33	80.00	77.50	78.75	71.25	70.00

Appendix

Table 15: Crossing learning and evaluation conditions for ACE with DeepSeek-V4.1-Flash on SpreadsheetBench. Scores are percentages, and Initial denotes the agent without an evolved skill store. Under

N_{\mathrm{Inst}}

and

N_{\mathrm{Evid}}

evaluation, the cleanly evolved agent scores

6.25

and

3.75

percentage points below the initial agent, respectively. Learning under the matching noise condition yields scores

6.25

and

8.75

percentage points below clean learning under the same noisy evaluation conditions.

Training data	$R=1$	$R=2$	$R=3$
Clean	85.00	86.25	90.00
$N_{2}$	76.25	75.00	73.75
$\Delta$ ( $N_{2}-\mathrm{Clean}$ , pp)	$-8.75$	$-11.25$	$-16.25$

Appendix

Table 16: Ablation on training rounds (ACE, DeepSeek-V4.1-Flash, SpreadsheetBench; scores in %). Results are from one run per learning condition. Each column reports performance on clean held-out tasks using the skill checkpoint at the end of round

R

Reference		Noisy learning				Noisy evaluation
Initial	$C\!\to\!C$	$N_{\mathrm{Inst}}\!\to\!C$	$N_{\mathrm{Evid}}\!\to\!C$	$N_{\mathrm{Rec}}\!\to\!C$	$N_{\mathrm{Bind}}\!\to\!C$	$C\!\to\!N_{\mathrm{Inst}}$	$C\!\to\!N_{\mathrm{Evid}}$
$88.75$	$95.00$	$93.75_{\downarrow 1.3\%}$	$88.75_{\downarrow 6.6\%}$	$90.00_{\downarrow 5.3\%}$	$91.25_{\downarrow 3.9\%}$	$86.25_{\downarrow 9.2\%}$	$88.75_{\downarrow 6.6\%}$

Appendix

Table 17: Backbone ablation with GPT-5.5 for ACE on SpreadsheetBench. Scores are percentages. Subscripts report relative changes,

100(s-b)/b

, from the clean-learning, clean-evaluation baseline

b=95.00

. Among the four learning-side noise conditions,

N_{\mathrm{Evid}}

produces the largest score decrease.

Agent Skill Learning

Agent skill learning develops reusable procedures and behaviors that help AI agents solve new tasks and adapt from experience.

See all 174 papers

Aug 13, 2026

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

LLM agents increasingly save experience from past tasks as reusable skills through skill evolution. When an agent completes an unsafe task, skill evolution can keep the unsafe procedure inside an otherwise useful skill, and a later benign task can repeat the harm after the unsafe instruction is gone. We call this failure skill misevolution. It requires that the agent write the unsafe procedure into a skill and that a later task retrieve and follow that skill. We introduce SkillMisevo-Gym, which clears every task session but keeps the skill library, so each of these steps can be checked separately. We also build SkillMisevo-Bench with 525 tasks in 25 episodes, where each episode alternates malicious and related benign tasks before three benign tasks in a new session. Across 21 combinations of agents and evolution methods, all of them write unsafe skills and 15 carry the harm into the new session, so the risk comes from skill evolution itself. Evolution raises benign utility in 15 combinations but also raises malicious-task harm in 17, so ranking methods by utility alone would hide this risk. We propose SafeEvolve, which checks each skill when it is written and when a later task retrieves it. It cuts unsafe retrieval in the new session from 35.3% to 8.7% and harm from 21.3% to 4.0% while keeping benign utility during learning within 0.4 points. SafeEvolveUp cuts new-session harm by 16.0 points and keeps its utility within 1.3 points of evolution without this mitigation. As agents learn from their own experience, safety must cover the skills that carry past behavior into future tasks. Code is available at https://github.com/henrymao2004/misevolve.

Xutao Mao, Liangjie Zhao, Xiang Zheng +1

City University of Hong Kong · Adelaide University

LLM Agent Skill Learning AI Agent Safety Benchmarks Self-Improving Agents LLM Agent Safety

Ask PDF

Aug 6, 2026

When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents

Self-evolving agents accumulate capability by distilling reusable skills from their execution trajectories, but we find this process is not monotonic: past a critical pool size, newly added skills degrade performance instead of improving it. We formalize this capability-contamination phase transition and trace it to a structural cause: once a defective skill enters the decision context, it becomes reference material for distilling later skills, forming cross-round contamination chains. We further show the contamination is structurally irreversible: removing a source skill after the fact cannot erase the flawed reasoning its descendants have already inherited, so post-hoc rollback recovers only a small fraction of the lost performance. This makes skill admission a pre-commit necessity rather than a post-hoc fix, and motivates Verifier-as-Gatekeeper (VaG): a progressive trust hierarchy whose three heterogeneous critics - structural validity, behavioral harmlessness, and semantic consistency - filter each skill individually, coupled with a marginal-gain subset selection that removes combinatorial contamination at the top tier before skills reach the runtime context. On Terminal-Bench 2, unconditional accumulation rises to a peak and then degrades, giving back most of its gains as the pool keeps growing, and post-hoc removal of the culprit skills recovers only a small part of the drop - the empirical signature of irreversibility. In contrast, VaG improves every round, reaching 72% pass@1 with a pool roughly 5x smaller, and its frozen skill pool transfers positively to four other backbones and a second benchmark without re-evolution. Ablations confirm the three critics are complementary and mutually non-substitutable, each intercepting a largely disjoint class of harmful skills.

Linfang Shang, Ming Xu, Yiding Sun +4

Tencent

AI Agent Reliability LLM Agent Skill Learning LLM Agent Verification Self-Evolving Agents

Ask PDF

Jun 4, 2026

OpenSkill: Open-World Self-Evolution for LLM Agents

Self-evolving agents requires adaptation after deployment, but existing approaches assume a usable learning loop, such as curated skills, successful trajectories, or verifier signals. Real open-world deployments may provide none of these, offering only a task prompt. In this work, we study open-world self-evolution, where an agent must build both its skills and its own verification signals from scratch, using open-world resources but no target-task supervision. We propose OpenSkill, a framework that bootstraps this loop: it acquires grounded knowledge and verification anchors from documentation, repositories, and the web, synthesizes them into transferable skills, and refines those skills against self-built virtual tasks grounded in the anchors rather than in target answers. The open world thus supplies both the knowledge to be learned and a supervision-independent practice environment, with target-task supervision reserved for final evaluation. Across three benchmarks and two target agents, OpenSkill attains the best automated pass rate while satisfying the no-supervision constraint. Analysis shows its skills transfer across models without model-specific adaptation, and its self-built verifier aligns with ground-truth outcomes despite never accessing them.

Zhiling Yan, Dingjie Song, Hanrong Zhang +8

Lehigh University · University of Illinois Chicago · University of British Columbia +3

LLM Agent Skill Learning LLM Agent Verification Self-Evolving Agents

Ask PDF

Abstract

Agent Skill Learning

Explore similar work

Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents

When Self-Evolution Backfires: Pre-Commit Gating against Skill Contamination in LLM Agents

OpenSkill: Open-World Self-Evolution for LLM Agents