cs.AIOct 6, 2026

Agent Plasticity: Measuring Self-Improvement Through Experience

Authors: Harman Singh, Anton Bakhtin, Rulin Shao, Gabriel Synnaeve, Ilia Kulikov, Rob Fergus, Sanjeev Arora, Kurt Keutzer, +3 more

Organizations: UC Berkeley · Meta Superintelligence Labs · University of Washington · Princeton University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.

Figures & tables

Figure 1: Frontier agents differ widely in how efficiently experience helps them self-improve. (left) Held-out ID (in-distribution environment) score against cumulative learning cost. (right) plasticity PsatP^{\mathrm{sat}} , i.e., Held-out ID score gain up to saturation point (solid stars on the left) per unit of learning cost. See Section 3.4 .
Figure 2: Frontier agents differ widely in how much experience helps. (a) A frozen model revises its persistent artifacts from experience, and every checkpoint is scored on held-out games. (b) Held-out gain ΔS\Delta S , averaged over chess, Go, and Hex. See Section 2 .
Figure 3: Persistent improvement varies across models. Checkpoint scores in Hard chess on training, held-out ID, and held-out OOD interactions. Scores are 100(W+0.5D)/N100(W+0.5D)/N . Easy chess, Go, and Hex are shown in Appendix Figure 9 . See Section 3.2 .
Figure 4: Artifact reuse and failure composition reveal different candidate bottlenecks. Panels show Hard chess on held-out ID games. Left: relevant decisions (moves where at least one saved artifact is relevant to choosing or playing the move, used or not) are classified as successful reuse, reuse with failure remaining, or missed reuse. Each decision contributes once. Right: identified failures are classified by whether a covering artifact was absent, unused, or used while failure remained. Failure shares do not measure overall failure frequency. Counts pool checkpoints 1–20. Easy chess, Go, and Hex appear in Appendix Figure 29 ; training games appear in Appendix Figure 28 . See Section 3.5 .
Figure 5: Claude Opus 5.5 shows a persistent gain in NetHack. (a) Median score and (b) mean deepest dungeon level over the ten games of each checkpoint, from 0 to 40. (c) PsatP^{\mathrm{sat}} in normalized score points per $1,000. (n.s.: no reliable rise in that analysis). Details in Appendix C . See Sections 3.3 and 3.4 .
Figure 6: Success when an artifact is used does not determine game-level improvement. Hard chess condition. Left: checkpoint-20 score minus checkpoint-0 score. Right: artifact-conditioned success rate, i.e., the share of moves with no identified failure among moves where a relevant artifact was used (checkpoints 1–20 pooled). This rate can be high even when artifacts are rarely used. Easy chess appears in Appendix Figure 26 . See Section 3.5 .
Figure 7: Dynamics across checkpoints do not determine game-level improvement. These graphs use the Hard chess setup. Each point pools training and held-out ID decisions over checkpoints 1,…,t1,\ldots,t , separately for each lineage; columns show decision-level reuse of artifacts, identified failures per relevant decision (a move where at least one saved artifact is relevant to choosing or playing it, used or not), and the shares of failures with absent and used covering artifacts. Easy chess, Go, and Hex appear in Appendix Figures 39 and 40 . See Section 3.5 .
Figure 8: Fixes that persist after a failure. Top (chess): after a checkpoint-6 game is scored 0 at the 300-ply limit, Claude Fable 5 makes its self-made engine value draws more as the limit nears; at checkpoint 18 the engine holds a draw until Stockfish errs, then mates. Bottom (NetHack): after dying at checkpoint 1 while praying through raw keystrokes, Claude Opus 5.5 adds a rule to pray only through its own controller; at checkpoint 10 the prayer heals it. Details in Figures 16 and 12 . See Section 3.5 .
Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Checkpoint performance in the Easy chess condition, Go, and Hex. Checkpoint performance, measured as 100(W+0.5D)/N100(W+0.5D)/N , in the Easy chess condition (top), Go (middle), and Hex (bottom), with columns as in Figure 3 . See Sections 3.2 and 3.3 .
Research directionPrimary focusOur emphasis
Reflection, memory, and skills ( Shinn et al., 2023 ; Wang et al., 2024 )Mechanisms for retaining and reusing experience.Measuring how efficiently different models construct and exploit persistent artifacts.
Context and harness optimization ( Zhang et al., 2026c ; Agrawal et al., 2026 )Optimizing prompts, context, or agent implementations.Evaluating a model that acts and improves its own future behavior under a common protocol.
Long-context evaluation ( Hsieh et al., 2024 ; Yen et al., 2025 )Retrieving, reasoning over, and following information in long contexts.Measuring durable capability gains from self-constructed artifacts, including executable tools.
Sequential learning benchmarks ( Dou et al., 2025 ; He et al., 2026 )Measuring improvement and, in some settings, learning efficiency across interactions.Combining cost-normalized held-out learning curves with artifact-level failure diagnostics.
Parametric adaptation ( Zweiger et al., 2025 )Persistent improvement through model-weight updates.Isolating non-parametric adaptation through persistent agent artifacts.
Appendix
Table 1: Positioning agent plasticity relative to prior work. Related approaches study complementary aspects of adaptation. Our evaluation combines persistent self-constructed artifacts, cost-normalized held-out improvement, and artifact-level diagnostics under a common fixed-weight learning protocol.
SettingTask and scoreOpponent or difficultyGames per checkpointTraining feedback available to reflectionOffline annotation for failure analysis
Chess (Hard)Standard 8×88\times 8 chess; win/draw/loss scores 1/0.5/01/0.5/0 .Stockfish 16 at 20k nodes/move. Train: skills 1–8; held-out ID: 3–8; held-out OOD: 12.16 Train, 12 ID, 12 OOD; both colors.After each training game, a separate skill-20 Stockfish evaluates every actor move. Centipawn loss gives ok , inaccuracy , mistake , or blunder .The analysis normalizes the game, model, and sandbox event streams and carries the recorded Stockfish annotations into failure extraction; no second engine pass is added.
Chess (Easy)The same board, rules, score, move limit, and annotation procedure.Stockfish 16 at 20k nodes/move. Train and held-out ID: skills 2–5; held-out OOD: skills 8–10.16 Train, 16 ID, 12 OOD; both colors.The same skill-20 counterfactual analysis and centipawn-loss severity labels as Hard.The same trace normalization and recorded-annotation binding as Hard.
Go5×55\times 5 Go, Chinese area scoring, positional superko, no suicide, komi 9.5; win/loss scores 1/01/0 .GNU Go levels 5–9 for Train and ID; level 10 for OOD; 75-action cap.20 Train, 20 ID, 12 OOD; both colors.Deterministic descriptive diagnostics: captures, liberties, self-atari, avoidable passing while behind, and area-margin change; no expert severity label.Frozen KataGo replays each position at 64 visits and refines candidates at 512 visits. Stable chosen-versus-best regret supplies mistake/blunder evidence; raw requests, responses, alternatives, perspective, and uncertain cases are recorded.
HexDeterministic 6×66\times 6 Hex; connection win/loss scores 1/01/0 .Native alpha–beta levels 10k–1M nodes/move for Train and ID; 10M for OOD.40 Train, 40 ID, 32 OOD; both colors.Rule-based missed immediate wins, preventable immediate losses, and own/opponent connection-distance changes; immediate tactical errors are blunder .The frozen native engine compares legal actions at a common completed depth, refines likely errors at 1M nodes, and adjudicates threshold-near or high-impact cases at 10M. Exact, proven, stable, uncertain, and not-failure dispositions are preserved.
NetHackNetHack Challenge v0; raw per-game maximum score, summarized by checkpoint mean, median, and maximum.Lawful dwarf female Valkyrie on deterministic fresh seeds. The Codex and Claude Code agent interfaces; at most 50k primitive actions per game for all models; two-hour active play (details below).10 fresh games per lineage.The reflector receives all ten verified public actor traces, terminal reasons, raw score, depth, experience, and action metrics, actor workspace evidence, and prior accepted inventories. No optimal-action oracle is used. 
Appendix
Table 2: Experimental setup across the four environments. Task distributions, opponent difficulty, evaluation-set sizes, training feedback exposed to the reflection loop, and separate offline annotation used for the failure analysis. Only training trajectories reach reflection; held-out trajectories and scores never do. Chess games cover both colors of every listed skill.
Figure 10: NetHack learning curves. Mean score of the ten games at each checkpoint (log scale) against (a) checkpoint. Curves cover checkpoints 0–40. Claude Opus 5.5 sustains higher performance after its early gain, with substantial variation across checkpoints; the other lineages’ changes are modest or negative. See Section 3.3 and Appendix C .
Figure 11: How far NetHack games get. (a) Median score and (b) mean deepest dungeon level per checkpoint, over checkpoints 0–40. See Appendix C .
LineageTool code (files / lines)TestsInstr. file linesTool calls per game (share)Actions per gameDug downMean score
Claude Opus 5.56 / 2,713125419622 (97%)3,58559,015
GPT-5.6 Sol1 / 2,08958176218 (59%)1,2110417
Claude Opus 4.85 / 1,3415841310 (2%)7162933
Claude Sonnet 54 / 1,68381467914 (96%)9551402
GPT-5.6 Luna0 / 001460 (0%)8800239
Claude Opus 513 / 8,237505452293 (96%)1,70411,919
Appendix
Table 3: Artifact libraries and play at checkpoint 10 , where all lineages play the same ten games. Tool code: Python files and lines; tests: test functions; instruction file: lines of the top-level CLAUDE.md or AGENTS.md; tool calls: shell commands that run a library tool (share of all shell commands); dug down: games in which the agent dug through a floor. More code or tool use does not explain Claude Opus 5.5’s lead: Claude Opus 5 has three times as much tool code. Caveats: Claude Opus 5.5 already scored 2.6–25 ×\times the others at checkpoint 0; its three best games give 69% of its checkpoint-10 total; and the other lineages used earlier harness versions (a crash guard added later fired in two Claude Opus 5.5 games; without them its mean is still 7,806).
Figure 12: Claude Opus 5.5 bypasses its own prayer check at checkpoint 1 and uses it at checkpoint 10. In NetHack, praying heals only when the character is in serious trouble and has not prayed recently. The agent’s controller ( tools/nhctl.py ) refuses risky prayers, but raw keystrokes ( ./nh.sh ) skip it. Top: at checkpoint 1 the agent, at 3 of 18 hit points, prays through raw keystrokes and dies. Middle: the check, and the rule “Pray only via nhctl pray ” added after checkpoint 1. Bottom: at checkpoint 10 the agent waits until the controller allows a prayer (8 of 94 hit points), is fully healed, and plays on. Later agents still sometimes bypass the check and die while praying. Screens: @ is the agent; DL dungeon level, XL experience level, HP hit points, T turn, S score. Below each screen: the agent’s messages (italics), commands ( ),andoutput(), and output (\rightarrow$ ). See Section 3.5 and Appendix D .
Figure 13: One open door, two models, same checkpoint-10 game. The door in the room’s east wall is open, but the text screen shows it as - , like a wall; we draw it in tan. GPT-5.6 Sol’s route planner never treats it as passable, so the agent kicks it, declares the level sealed, climbs back, and ends on dungeon level 1 with 119 points. Claude Opus 5.5’s controller also stalls, but the agent uses NetHack’s travel command to a square past the door and then finds the stairs down where its notes guess them (levels here often repeat the previous layout). This game scores 15,718, though almost all of it is earned later. Bottom: the lines in each model’s files behind each behavior. Layout as in Figure 12 . See Appendix D .
Figure 14: GPT-5.6 Sol stops playing by hand and lets its own engine play. Two held-out games (games the model never reviews) as Black against Stockfish at skill 8 (skill runs from 0 to 20; higher is stronger). At checkpoint 0 the agent picks every move itself, and 11…Qd7 (Black’s eleventh move) loses a pawn and then a knight. At checkpoint 18 a chess engine that the agent wrote into its library at checkpoint 1 plays every move, and the game ends in checkmate on move 42. Reading these figures: linesaretheagent’scommands,lines are the agent’s commands,\rightarrow$ lines their output, and grey lines our notes; boxes quote the artifacts with line numbers. Arrows mark the agent’s mistakes (red), good moves (green), and other moves discussed (blue), and the opponent’s moves (grey). See Section 3.5 and Appendix E .
Figure 15: A mistake that a later checkpoint avoids. In a checkpoint-14 training game GPT-5.6 Sol plays 3.Qg4, which Stockfish rates as a mistake costing 108 cp (centipawns, hundredths of a pawn), and loses. When writing checkpoint 15, the model adds every such costly move to an avoid list in its engine (here: in this position, do not play 3.Qg4). At checkpoint 16 the same position arises in a held-out game and the engine plays 3.Nf3 instead. Table: with the entry removed, later engines play 3.Qg4 again, so the entry alone changes the move. The fix memorizes a single position and helps only when it recurs. (Comments inside the artifact count from the start of a continuation run, so “checkpoint-013” is checkpoint 14.) See Section 3.5 and Appendix E .
Figure 16: Claude Fable 5 learns to value draws near the move limit. At checkpoint 6 a game reaches the 300-ply limit (a ply is one move by either side) and is scored 0, like a loss. When writing checkpoint 7, the model makes its engine value a draw more and more as the limit nears (from 60 cp at ply 170 up to 900 cp at ply 280; 100 cp is about one pawn), so that it steers toward a draw instead of running out of moves. Plot: the engine’s evaluation of its own moves in both games; the dashed line is the value the rule gives a draw. In a checkpoint-18 held-out game the engine holds these draw values from ply 227 until Stockfish errs with 124…Rb8; it then wins by checkmate on move 136. The win comes from the opponent’s error; the rule kept the game going until then. See Section 3.5 and Appendix E .
Figure 17: Claude Opus 5’s engine plays for a draw when it is losing. After training games drawn from lost positions, the model changed its engine so that, when it is behind by 250 cp or more, a draw is valued as if it were ahead by 120 cp; such moves are tagged drawline . In this checkpoint-19 held-out game it first tries to repeat moves (51…Qe1, 52…Qa5), which draws if a position occurs three times. Later, at −663-663 cp, it gives away its queen with 61…Qxg2+; after 62.Kxg2 Black has no legal move but is not in check. This is stalemate, a draw (score 0.5) instead of a loss. See Section 3.5 and Appendix E .
Figure 18: GPT-5.5’s rules against a known mistake do not stop it. From checkpoint 5 on, GPT-5.5’s notes and code checks forbid capturing the e5 pawn early with Nxe5 when the knight can be taken back. Plot: number of such rules per checkpoint (code checks solid, notes dashed). But the opening book (a fixed list of prepared moves) answers before the checks run, and at checkpoint 20 it still picks 6.Nxe5 in this held-out game, a blunder costing 263 cp; the game is lost. Bars: early-Nxe5 mistakes found by our failure analysis in held-out (dark) and training games. The rules pile up while the mistake continues. See Section 3.5 and Appendix E .
Figure 19: GPT-5.6 Luna’s notes tell it to play the first legal move listed. To avoid losing games by running out of its 64 model calls per game, the notes at checkpoint 20 tell the agent to copy the first move in the list of legal moves without comparing options. In this held-out game every move is the first one listed, including 3.Nxh7, which loses a knight, and 13.Rh1, which allows checkmate next move. Plot: share of held-out moves that are the first move listed, 0.65 at checkpoint 20 and below 0.2 before. Some checkpoint-20 games still run out of calls. See Section 3.5 and Appendix E .
Figure 20: Score improvement with learning tokens in chess. Learning tokens count the model calls spent learning: playing the training games and reflecting on them to write the next checkpoint (held-out games are not counted). Lines are three-checkpoint averages. See Section 3.4 and Appendix F .
Figure 21: Score improvement with learning tokens in Go. Score against cumulative learning tokens. Lines are three-checkpoint averages. See Section 3.4 and Appendix F .
Figure 22: Score improvement with learning tokens in Hex. Score against cumulative learning tokens. Lines are three-checkpoint averages. See Section 3.4 and Appendix F .
Figure 23: Score improvement with learning cost in chess. Score against cumulative learning cost in dollars, the cost used in PsatP^{\mathrm{sat}} (Section 2.2 ). See Section 3.4 and Appendix F .
Figure 24: Score improvement with learning cost in Go. Score against cumulative learning cost. Lines are three-checkpoint averages. See Section 3.4 and Appendix F .
Figure 25: Score improvement with learning cost in Hex. Score against cumulative learning cost. Lines are three-checkpoint averages. See Section 3.4 and Appendix F .
StageInputWhat the prompt asksOutput
Failure identificationGame rules; for each decision the position, legal actions, chosen action, and engine evaluation; recorded failures and uncertain engine candidates. No artifacts.Describe each recorded failure from direct evidence. Label each uncertain candidate; heuristic or unstable engine evidence alone is not a failure. Add failures only when same-decision evidence shows them, at most one optional failure per decision.Failure list with names, descriptions, and cited events.
Failure verificationSame input plus the draft, marked as an untrusted proposal.Re-check independently and return the complete corrected list: keep every recorded failure, relabel candidates, keep only directly evidenced additions, and drop duplicates.Final failure list.
Failure–artifact relationVerified failures and the records of their decisions; the full contents of every artifact available at that checkpoint; recorded artifact reads and invocations.Judge whether an artifact covers the failure from its contents, not its name. Judge use separately, from evidence before or during the failure: an invocation and its end event for executables, access and application for text.Per failure: no covering artifact, or covering artifact used or not used.
Relation verificationSame input plus the draft.Check every failure against the complete artifact list, including artifacts the draft omitted. Producing the failing action does not by itself make an artifact covering.Final coverage and use judgment.
Decision-level reuseEvery game-play decision, in batches of up to 20, with its verified failures and relation judgments; artifacts; recorded reads and invocations.For each decision and artifact, judge relevance to choosing or executing the action, use from same-decision evidence, and outcome. An artifact that covers a failure at the decision is relevant.Per pair: relevant, used, and successful or failure remained.
Reuse verificationSame input plus the draft.Re-audit and return complete corrected rows. Availability, file listings, names, automatic loading, and evidence from other decisions do not establish use.Final reuse judgments.
Appendix
Table 4: Analyzer stages in the order they run. Each call goes to Gemini 3.1 Pro, which is told to treat the records as data, use only the given evidence and game rules, and return one JSON object. A claim that an artifact was used is kept only if the logs show it being run or read. Requests also include engine evaluations that the agents never saw. Coverage: training and held-out ID games at checkpoints 0–20.
Figure 26: Score change and success after reuse in the Easy chess condition. Same as Figure 6 , which shows the Hard condition. See Section 3.5 and Appendix H .
Figure 27: Improvement and outcomes after artifact use in Go and Hex. Left: checkpoint-20 score minus checkpoint-0 score in each evaluation split. Right: artifact-conditioned success rate, i.e., how often a decision succeeds when an artifact was used (an association, not a causal effect). See Section 3.5 and Appendix H .
Figure 28: Reuse and failure breakdown on training games. Same as Figures 4 and 29 , which use held-out ID games. See Section 3.5 and Appendix H .
Figure 29: Reuse and failure breakdown in the Easy chess condition, Go, and Hex. Held-out ID games; same as Figure 4 , which shows the Hard chess condition. See Section 3.5 and Appendix H .
Figure 30: Relation between identified failures and persistent artifacts in Go and Hex. Each verified failure contributes once: no covering artifact existed, a covering artifact existed but was not used, or an artifact was used but the failure remained. Fractions are shown within each model, game, and split; counts beneath labels show verified/identified failures. See Section 3.5 and Appendix H .
Figure 31: How chess failures change from early to late checkpoints. “E” sums checkpoints 1–5 and “L” checkpoints 16–20 (80 training and 60 held-out ID games per lineage in each). Claude Fable 5 and Claude Opus 5 make fewer failures late. Late failures are mostly “artifact available but unused” for Gemini 3.1 Pro and GPT-5.6 Luna, and mostly “failed despite use” for Claude Opus 5, Claude Opus 4.8, and GPT-5.6 Sol; GPT-5.5 moves toward non-use. These are descriptive, not causal. See Section 3.5 and Appendix H .
Figure 32: Number of Go failures and their relation to artifacts. Bar length is the number of failures over checkpoints 1–20, split into the three classes (no covering artifact, artifact not used, failed despite use); nn is the number of failures. See Section 3.5 and Appendix H .
Figure 33: Number of Hex failures and their relation to artifacts. Same as Figure 32 . Counts can be compared within a game, not across games. See Section 3.5 and Appendix H .
Figure 34: Artifact coverage and reuse at each checkpoint, chess training games. The first three panels show, per failure, how often no artifact covered it, a covering artifact existed but was not used, or it happened even though an artifact was used. The last panel shows the share of relevant decisions (moves where at least one saved artifact is relevant to choosing or playing the move) where an artifact was used successfully. Faint markers are single checkpoints; lines are three-checkpoint averages. See Section 3.5 and Appendix H .
Figure 35: Artifact coverage and reuse at each checkpoint, chess held-out ID games. Same as Figure 34 . The model never reviews these games, so they show how the artifacts carry over to unseen games. See Section 3.5 and Appendix H .
Figure 36: Chess reuse outcomes over time. For each model, four bars for checkpoints 1–5, 6–10, 11–15, and 16–20. Each relevant decision (a move where at least one saved artifact is relevant to choosing or playing it) counts once: reused successfully, reused but the failure remained, or missed (a relevant artifact was not used); nn is the number of relevant decisions. Summed over the four phases, the totals match Figures 4 and 28 . See Section 3.5 and Appendix H .
Figure 37: Go reuse outcomes over time. For each model, bars run from checkpoints 1–5 to 16–20. Each relevant decision (a move where at least one saved artifact is relevant to choosing or playing it) counts once: reused successfully, reused but the failure remained, or missed; nn is the number of relevant decisions. See Section 3.5 and Appendix H .
Figure 38: Hex reuse outcomes over time. Same as Figure 37 . See Section 3.5 and Appendix H .
Figure 39: Cumulative improvement-loop metrics in chess, all eight models. Same as Figure 7 , for the Hard (top) and Easy (bottom) chess conditions. Each point pools training and held-out ID decisions from checkpoint 1 to tt . Columns: reuse rate; failures per relevant decision (a move where at least one saved artifact is relevant to choosing or playing it); FabsentF_{\mathrm{absent}} ; and FusedF_{\mathrm{used}} . Dotted: fewer than 30 failures so far. See Section 3.5 and Appendix H .
Figure 40: Cumulative improvement-loop metrics in Go and Hex. Same as Figure 7 ; each point pools training and held-out ID decisions from checkpoint 1 to tt . Claude Opus 4.6 and Gemini 3.1 Pro are analyzed in chess only. Dotted: fewer than 30 failures so far. See Section 3.5 and Appendix H .

Explore similar work

CardsList
Aug 4, 2026cs.CL

PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agents

Recursive self-improvement requires agents to turn accumulated experience into better future behavior. Personal AI agents offer a concrete setting for studying this capability because they retain preferences, task histories, tool routines, and learned skills across sessions. Yet whether retained experience actually improves them over time has not been systematically tested. We introduce PAST-Bench, a benchmark designed to isolate this question. Each agent runs through ordered sequences of fresh-session tasks under matched conditions that turn retained experience on and off. It spans 26 scenarios and 204 episodes across memory, procedural reuse, information gathering, and update. We report both later-task gains and whether those gains follow the intended save, retrieve, and update pathway. Across seven base models and four agent frameworks, improvement is real but uneven across capabilities. Agents with the same headline gain can differ markedly in whether that gain is supported by evidence of the intended pathway. Guided by these findings, we develop Hermes+, which extends Hermes with five targeted interventions across stages of the agent loop. Hermes+ raises the average gain from retained experience and provides clearer pathway evidence, with its strongest improvement on tasks requiring outdated state to be replaced, although the effect remains capability- and model-dependent. Together, PAST-Bench and Hermes+ provide an evaluation and diagnostic foundation for studying how persistent agents can progress from retaining experience to systematically improving through it. Code: https://github.com/Gen-Verse/PAST-Bench

Shuhan Xue, Zixin Ding, Yichen Shen +6

Self-Improving AgentsRecursive Self-ImprovementLong-Term Agent MemoryArtificial Intelligence AgentsDepth History
AskPDF
Jul 14, 2026cs.AI

Self-Improvements in Modern Agentic Systems: A Survey

Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self-improving agents as adaptive systems that convert experience into accumulated capability gains. We offer a system-level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self-improvement is formalized as a self-induced update operator that obtains and commits updates to model parameters or scaffold components. We organize prior work by update target and by the signals that drive change, then review applications and discuss evaluation, before closing with open problems and future directions. For convenience, we track technical updates on https://github.com/selfimproving-agent/awesome-Self-Improving-Agents.

Zhe Ren, Yimeng Chen, Dandan Guo +9

School of Artificial Intelligence, Jilin University · King Abdullah University of Science and Technology (KAUST) · Independent Researcher +2

Self-Improving AgentsRecursive Self-ImprovementProduction Agentic SystemsAutonomous AgentsSurvey
AskPDF
Aug 13, 2026cs.AI

Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development

Autonomous agents are increasingly capable of improving models, systems, and other technical artifacts through long-horizon experimentation. To understand the current state of this capability, however, evaluation must go beyond final scores, which neither reveal where progress is gained or lost nor indicate whether accumulated experience improves later decisions. We therefore present a systematic evaluation of seven frontier models on 36 long-horizon tasks based on a new framework that uses rule-based metrics to characterize within-run behavior through Solution Framing, Execution, and Feedback Control and controlled comparisons to assess experience reuse within and across tasks. The results show that current agents operate more like engineering optimizers than fully autonomous researchers: they can formulate and implement practical solutions, but their performance varies substantially across runs, their strongest solutions mainly adapt or combine established techniques, and genuine methodological novelty remains rare. Detailed analysis reveals that observed performance is shaped by multiple factors, including distinct process bottlenecks behind similar final outcomes, experience reuse that can help or mislead subsequent decisions, and harness designs that affect performance stability. These findings suggest concrete directions for improving model training, inference-time strategies, experience management, and harness design.

Yiwei Li, Wanli Yang, Hexiang Tan +10

1Meituan · University of Chinese Academy of Sciences

Long-Horizon AgentsAutonomous AgentsScores
AskPDF