Self-evolving agents rewrite a persistent state from feedback, but it is largely untested whether this lets them learn a rule nobody told them. An agent can also pass by writing down each correction as an exception, and its list then grows with every new rule. We introduce CLR-Bench, a Curriculum of Latent Rules benchmark. Each chain adds one hidden rule per level on top of the earlier ones. A rule acts on an opaque label that stands for a named grouping published on the Web, such as the founding members of an airline alliance. The task gives one example value, practice feedback reveals one more member and one non-member, and the test checks unseen members and similar non-members. Probes certify that the base model knows the members of each grouping and does not guess it without feedback, and that the evidence identifies the grouping once labels are read as named groupings. An automated pipeline builds chains from a new data domain, and we release 20 chains over 10 domains. We evaluate six recent self-evolution methods. With the certifying model they barely learn. The best gain at most 5 points over a frozen agent and exclude only 6-12% of the unseen members from the count, fewer than the 20% reached by simply keeping the raw practice history. A fixed lesson that states the named-grouping convention lifts the same methods by 14.5 to 16.5 points. Their states hold the revealed member in 75-98% of cases and almost never the class name, and when practice exposes a member the list misses, they add that member to the list. With the stronger GPT-6-Sol, ACE and Dynamic Cheatsheet learn from feedback alone. They reach 78% accuracy and solve 59-60% of the single cells, against 53% and 8% with the certifying model, and they surpass the raw history. CLR-Bench therefore measures how well an agent learns, and this ability differs across models.
Figures & tables
Figure 1. Overview of CLR-Bench . (a) A level hides a published grouping behind an opaque label, with one example A ; practice reveals one member ( m0 ) and one non-member ( np ), and the test checks unseen members ( m1,m2 ) and test non-members ( Nt ). (b) Rules accumulate over four levels; the agent practices, learns, and is tested. (c) Four probes certify each candidate.
Table 2
Figure 2. Composition of CLR-Bench . (a) Hidden classes by domain (inner), trap field (middle), and kind of grouping (outer). (b) Other published lists that contain each example. (c) Size of each value universe. (d) Length of the test task per level.
Probe
Model is given
Requirement
Known
pool values, one feedback, lesson
names it, soft score ≥ 0.8
Hidden
label and example, no feedback
does not recover the class
Learnable
practice feedback, no lesson
assigns 1/6–2/3 of unseen
Solvable
feedback and the lesson
soft score ≥ 0.8
Table 3. Probes that certify every hidden class (sampled over the three choices of the revealed member).
Figure 3. Test accuracy per level (20 chains) from feedback alone (open) and with the lesson (filled), with GPT-6-Luna (red) and GPT-6-Sol (blue). Gray band: the designed staircase (80, 60, 40, 20) of an agent that applies each rule only to the example.
Figure 4. Generalization against specificity (GPT-6-Luna, levels 1–4 pooled): each method from feedback alone (open) to + lesson (filled). The inset enlarges the top-left corner, where Training-Free GRPO and the frozen agent overlap.
Figure 5. GPT-6-Luna, learning from feedback alone. (a) Exclusion of the revealed member m0 and of the unseen members m1,m2 . (b) Exclusion of m2 , which is never shown, under standard and rewarded practice (where m1 is revealed from the second attempt).
Figure 6. (a) What the state says about each level’s label (ACE and Dynamic Cheatsheet, 160 states per row, annotated without test results; names the class: the class, a broader grouping, or its members listed; wrong rule: another grouping or an attribute rule). (b) Unseen members excluded and non-members kept by the program, by state content, over all annotated states.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Chain
Level 1
Level 2
Level 3
Level 4
CH1
Stockholm Convention Annex A
Commonwealth of Nations members
Star Alliance airlines
Chemical Weapons Convention Schedule 3
CH2
Chemical Weapons Convention Schedule 3
Group of Seven members
oneworld airlines
Stockholm Convention Annex A
GR1
Commonwealth of Nations members
Star Alliance airlines
League of European Research Universities members
NATO members
GR2
Schengen Area members
Five original Star Alliance airlines
ASEAN University Network members
ASEAN members
LG1
BRICS member country-code top-level domains
Office LTSC Professional Plus 2021 applications
Microsoft server product port assignments
Schengen Area member state country-code top-level domains
LG2
OECD member country country-code top-level domains
Microsoft productivity applications
Plaintext application protocol ports
G7 member country-code top-level domains
Appendix
Table 4. Hidden classes, one row per chain (levels 1–4). TR: Travel audit, SY: Pharmacy review, WS: Web security, LG: System logs, SP: Sponsorship, CH: Chemical imports, MN: Raw materials, SH: Port inspections, UN: Scholarships, GR: Travel grants.
Figure 7. Failure modes and cost (GPT-6-Luna, learning from feedback alone unless marked +L). (a) How single cells fail. (b) Practice levels passed and (c) practice input tokens against single-cell accuracy.
Figure 8. Retention: unseen members excluded (top) and non-members kept (bottom) for rule j (columns) at level k (rows).
CWE (934/86), page language (177/63), host TLD (1285/40)
System logs
TLD (1436/53), client application (131/36), port (6063/24)
Sponsorship
club (151/12), co-sponsor (692/12), market country (249/10)
Chemical imports
substance (654/18), exporter country (195/14), air carrier (239/7)
Appendix
Table 5. The ten domains: value-universe size and number of candidate lists per trap field.
Condition
Added
Tests
Frozen
–
nothing, no state
the frozen staircase
Self (6 methods)
U
–
induction from feedback
+ lesson
U
convention + procedure
learnable from evidence
rewarded practice
–
unseen m1 from try 2
response to failed lists
History
S
raw practice history
update vs. raw evidence
+ lesson
S
history + lesson
lesson, no method
Appendix
Table 6. Conditions: text added to the update step (U) or the solver prompt (S).
L1
L2
L3
L4
Avg
Δ
Single
inc u
exc
Frozen
80
57
34
20
47.8
–
0
1
94
History
84
65
37
23
52.3
+4.5
13
20
82
rewarded practice ∗
90
75
51
39
63.8
+16.0
31
42
87
History + lesson
85
70
55
44
63.5
+15.8
44
59
75
History + convention
88
68
57
46
64.8
+17.0
47
59
73
History + procedure
82
67
40
28
54.3
+6.5
18
32
82
Appendix
Table 7. All conditions, GPT-6-Luna (%; Δ : change over the frozen agent). Rows without + learn from feedback alone. † With a format note (Appendix A.3 ). ∗ m1 is revealed in practice; inc u counts m2 only.
Update-step text
L1
L2
L3
L4
Avg
Single
inc u
exc
ACE
procedure, neutral heading
83
65
45
27
55.0
10
15
98
convention, neutral heading
91
85
65
45
71.5
47
57
91
lesson, neutral heading
89
83
66
50
72.0
48
60
94
convention at level 1 only
83
75
58
39
63.8
34
47
94
class names + feedback
93
82
64
55
73.5
52
66
88
DC
procedure, neutral heading
77
64
46
33
55.0
18
28
90
Appendix
Table 8. Text added to the update step (GPT-6-Luna); columns as in Table 7 . Neutral heading: “UPDATE NOTE” instead of “VERIFIED LESSON”.
Frozen
> stair
Oracle
Single
inc u
exc
GPT-6-Luna (high) ∗
.80/.57/.34/.20
0
58.0
40
49
58
GPT-5.6-Luna (high)
.80/.60/.40/.18
0
62.8
45
54
63
GPT-5.6-Terra (high)
.76/.48/.38/.14
1
57.0
42
49
51
Claude Sonnet 5.5 (low)
.80/.55/.27/.20
4
–
–
–
–
GPT-6-Sol (high)
.77/.32/.15/.06
1
–
–
–
–
Appendix
Table 9. Other base models (%). Frozen: accuracy per level and level scores above the staircase (of 80). Oracle: the class-name oracle. ∗ Evaluated and certifying model.
GPT-6-Luna
GPT-6-Sol
Acc
Single
inc u
exc
Acc
Single
inc u
exc
Frozen
47.8
0
1
94
32.5
1
6
39
History
52.3
13
20
82
70.3
49
68
83
+ lesson
63.5
44
59
75
83.8
71
84
91
ACE
52.5
8
12
92
78.3
59
68
95
+ lesson
67.0
46
62
86
81.3
68
82
92
Appendix
Table 10. Both base models (%). Methods learn from feedback alone unless marked + lesson. † With a format note (Appendix A.3 ).
Domain
Frozen
History
ACE
+L
DC
+L
Hist.+L
Travel audit
50.0
57.5
70.0
70.0
50.0
72.5
87.5
Pharmacy review
50.0
35.0
47.5
55.0
50.0
55.0
35.0
Web security
50.0
52.5
50.0
40.0
50.0
47.5
70.0
System logs
45.0
50.0
47.5
67.5
50.0
67.5
85.0
Sponsorship
32.5
47.5
47.5
65.0
52.5
65.0
70.0
Chemical imports
50.0
55.0
50.0
62.5
62.5
67.5
40.0
Appendix
Table 11. Average test accuracy (%, levels 1–4) per domain, two chains each. +L: with the lesson. The lesson adds little in web security and pharmacy review.
Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rules and affect later tasks. Existing evaluations largely examine clean learning or perturbations encountered after skills are frozen, leaving robustness during self-evolution insufficiently understood. We introduce RSE-Bench, to our knowledge the first benchmark for evaluating skill self-evolution under ordinary, non-adversarial noise. It spans four domains and six benchmarks, with controlled perturbations to instructions, environmental evidence, execution records, and outcome--record pairings that preserve the underlying tasks and native scoring rules. The protocol separates noise during learning from input-side noise at evaluation, testing frozen skill stores on held-out tasks. We also introduce RSEClaw, a method-agnostic framework supporting consistent noise injection and process diagnostics of actual exposure and skill updates. Across three methods and two backbones, noisy learning retains only 25.5%--42.1% of the aggregate gain achieved by clean self-evolution, depending on the noise type. Perturbations confined to execution records and outcome--record pairings cause average losses comparable to input-side noise, despite leaving the original executions and evaluations intact. Diagnostics further identify differences in candidate skill text that coexist with small changes in mean held-out performance.
Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution via Ability-guided transfer across four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content transfers reliably across model families, but no current automatic method sustains positive gain in all settings. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, and uptake. The benchmark is publicly available at https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.
Xingze Gao, Chuanrui Hu, Hongda Chen +9
1Anhui University · 2EverMind, Shanda Group · 3Southeast University
Self-evolving tool-integrated agents learn from tasks and feedback generated within their own training loop. A Curriculum Agent generates tasks, while an Executor Agent learns from self-consistency signals through reinforcement learning. However, relying solely on the current Executor for feedback has two limitations: group-relative advantages vanish under full consensus, while uncertainty-based curriculum rewards favor disagreement without showing whether the generated tasks support further learning. These limitations motivate an additional reference beyond the current Executor. We propose AnchorLoop, which introduces a frozen copy of the previous iteration's Executor as a historical reference and reuses it on both sides of the training loop. For the Executor, the anchor provides a cross-reference advantage that evaluates current outputs against both current and historical majority answers. For the Curriculum, it provides an agreement-based reference based on differences in sampled majority agreement. Since the Executor and anchor have identical parameters during Curriculum training, this comparison serves as a proxy for task selection rather than evidence of inter-version improvement or correctness. Across 13 reasoning benchmarks, AnchorLoop improves over Agent0 by 2.5% on mathematical reasoning and 2.8% on general reasoning tasks. It also maintains higher effective-advantage variance and continues improving in later iterations as the unanchored baseline shows diminishing gains. These results demonstrate the benefit of introducing a lightweight historical reference into self-evolving tool-integrated agents without external task or answer supervision.