Certified Long-Horizon Code Agent Evolution via Validation-Gated Skill Optimization

Yifan Wang; Hao Cheng; Xiaomin Li; Yuexing Hao; Hemanth Neelgund Ramesh; Dongwon Jung; Hao Tang; Keru Wang

cs.AISep 26, 2026

Certified Long-Horizon Code Agent Evolution via Validation-Gated Skill Optimization

Authors: Yifan Wang, Hao Cheng, Xiaomin Li, Yuexing Hao, Hemanth Neelgund Ramesh, Dongwon Jung, Hao Tang, Keru Wang,

Organizations: Purdue University · Microsoft · University of Washington · University of California, Davis ·

Abstract

Long horizon agent self-evolution without model weight updates is essential for enabling deployed agents to accumulate reusable skills and improve over time. Prior self-evolution work has focused primarily on short-horizon tasks, while repository-level software engineering remains unexplored despite being an ideal testbed for long-horizon adaptation. In this setting, agents are required to solve streams of sequential tasks, navigate complex dependencies with evolving repositories and persistently store and reuse experience. Text-based skill optimization offers an efficient, non-parametric approach for such adaptation. However, existing methods often suffer from unstable updates, performance drawdown, and agent collapse over extended deployments. In this paper, we formalize the concept of in-context self-evolution and introduce VALVE, a validated-gated framework for long-horizon skill optimization. We establish finite convergence, provide theoretical guarantees for future-task gain and drawdown, and derive the validation and evaluation holdout sizes required for a prescribed tolerance, with leading-order scaling Empirically, our pipeline, VALVE achieves stable self-improvement over evolution horizon spanning more than 1,000 SWE tasks, with average final and peak gains of $14.9$ and $16.5$ points across three frontier models (GPT-5.5, Claude-4.6 and MiniMax-M2.7). The validation gate reduces average drawdown by 75% and produces an 11x more compact skill bank than ungated evolution. We further present extensive ablations identifying the design choices most critical to long-horizon skill evolution.

Figures & tables

Panel of Figure 1 — Figure 1: Validation stabilizes long-horizon self-evolution. (a) Avg@3 resolved rate over approximately $1{,}000$ SWE-smith task arrivals. Both runs improve early, but the ungated run falls from a $91.7\%$ peak to $80.0\%$ , while the validated run peaks at $97.0\%$ and ends at $92.3\%$ . (b) Across all twelve model–repository runs, validation reduces average drawdown by $75\%$ .

MiniMax-M2.7
	Final resolved rate (avg@3, %)
Setting	pygments	sqlfluff	deepdiff	pydicom	Model Avg.
Base	$52.5$	$66.5$	$56.3$	$60.3$	$58.9$
SkillOpt	$60.0\ {\color[rgb]{0.1758,0.4883,0.332}(+7.5)}$	$66.0\ {\color[rgb]{0.668,0.2734,0.2734}(-0.5)}$	$\mathbf{69.0}\ {\color[rgb]{0.1758,0.4883,0.332}\mathbf{(+12.7)}}$	$60.7\ {\color[rgb]{0.1758,0.4883,0.332}(+0.4)}$	$63.9\ {\color[rgb]{0.1758,0.4883,0.332}(+5.0)}$
VALVE w/o val	$53.3\ {\color[rgb]{0.1758,0.4883,0.332}(+0.8)}$	$66.7\ {\color[rgb]{0.1758,0.4883,0.332}(+0.2)}$	$61.7\ {\color[rgb]{0.1758,0.4883,0.332}(+5.4)}$	$58.7\ {\color[rgb]{0.668,0.2734,0.2734}(-1.6)}$	$60.1\ {\color[rgb]{0.1758,0.4883,0.332}(+1.2)}$
VALVE	$\mathbf{61.0}\ {\color[rgb]{0.1758,0.4883,0.332}\mathbf{(+8.5)}}$	$\mathbf{72.7}\ {\color[rgb]{0.1758,0.4883,0.332}\mathbf{(+6.2)}}$	$65.3\ {\color[rgb]{0.1758,0.4883,0.332}(+9.0)}$	$\mathbf{61.7}\ {\color[rgb]{0.1758,0.4883,0.332}\mathbf{(+1.4)}}$	$\mathbf{65.2}\ {\color[rgb]{0.1758,0.4883,0.332}\mathbf{(+6.3)}}$

Table 1: Final performance after long-horizon SWE-smith repository-level evolution. Parentheses show gains over the corresponding base model; best results are bold.

MiniMax-M2.7
Setting	Final gain	Peak gain	Draw- down
SkillOpt	$+5.0$	$+7.4$	$\mathbf{2.3}$
VALVE w/o val	$+1.2$	$+5.4$	$4.2$
VALVE	$\mathbf{+6.3}$	$\mathbf{+9.0}$	$2.7$
GPT-5.5
SkillOpt	$+6.2$	$+12.8$	$6.6$

Table 2: Gain retention over evolution. Best results are bold.

Figure 3: Ablation of component-level design choices on pydicom with GPT-5.5. Appendix C.1 maps prior self-evolving methods to the compared alternatives; the step-0 baseline is $38.2\%$ .

Figure 4: Final and peak avg@3 resolved rate versus validation sizes using Claude-4.6 on repository pygments .

MiniMax-M2.7
	Final bank size (skills / principles)
Setting	pygments	sqlfluff	deepdiff	pydicom	Total
VALVE w/o val	$244/231$	$237/220$	$238/239$	$205/197$	$924/887$
VALVE	$30/34$	$38/38$	$14/13$	$34/31$	$116/116$
GPT-5.5
VALVE w/o val	$249/194$	$234/160$	$267/210$	$174/127$	$924/691$

Table 3: Final bank size after repository-level evolution. Entries report skills / principles.

Figure 5: Accumulated bank size over evolution for Claude-4.6 on pygments . Solid curves show avg@3 resolved rate, dashed curves show bank size, and triangles mark bank updates.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Figure 7: Holdout size required for a prescribed tolerance, from Corollary 1 . The vertical axes are logarithmic; horizontal lines mark the $100$ -task evaluation set and the $50$ -task validation panel used in our runs.

	Evaluation set $n_{\mathcal{H}}$			Validation panel $n$
Tolerance $\gamma$	$A=4$	$A=8$	$A=16$	$K\leq 3$	$K\leq 5$	$K\leq 7$
$2$ pp	$680$	$787$	$895$	$235$	$324$	$396$
$3$ pp	$328$	$380$	$432$	$195$	$268$	$327$
$5$ pp	$136$	$157$	$179$	$142$	$195$	$239$
$8$ pp	$62$	$72$	$82$	$99$	$136$	$166$
$10$ pp	$44$	$51$	$58$	$81$	$111$	$136$

Appendix

Table 4: Holdout size required for a prescribed tolerance, at

\delta=0.05

and

\sigma^{2}=\sigma^{2}_{\mathcal{H}}=0.025

. Left: tasks needed so that the certified population drawdown exceeds the measured one by at most

\gamma

, for

A

distinct banks among the checkpoints. Right: tasks needed so that every accepted update of a run making at most

K

of them costs no more than

\gamma

, at a gate-query horizon

T=60

and anchored at the observed accepted panel gain of

5.8

points.

Component	Reference setting
Data	One repository-level task stream from the SWE-smith training split
Evaluation set	100 tasks sampled without replacement (seed 42)
Validation panel	50 tasks sampled from the remaining stream (seed 123); the same fixed panel is used for every acceptance decision
Agent	Frozen code-agent backbone (MiniMax-M2.7, GPT-5.5, or Claude-4.6-sonnet, according to the run) with an interactive shell environment
Adaptation	3 independently sampled rollouts per task
Evolution trigger	Mixed-outcome tasks only; at most 100 mixed tasks

Appendix

Table 5: Reference configuration shared by the main repository-level runs.

Ablation	Method-level intervention
Sentence skills	Replace structured rich bodies with one-sentence principles and scoped skill sentences throughout extraction, storage, and injection.
No principles	Constrain Extract to return an empty principle set; retrieve and inject skills only.
Single document	Replace itemized banks with one principle document and one skill document, updated through section-level add/replace/delete operations and injected in full. This also removes item-level retrieval.
Inject all	Remove task-conditioned ranking and inject every stored item in reverse creation order.
Batch of four	Buffer four completed tasks and perform one cross-task reflection; use one validation rollout per validation task.
All tasks	Apply Extract and Validate after all-pass and all-fail tasks as well as mixed tasks; the 100-step horizon consequently counts raw tasks.

Appendix

Table 6: Method-level definitions of the ablations.

Component	Prior work with our choice	Prior work with the ablated choice
Retrieval Selection $\to$ inject all	Voyager ( Wang et al., 2023 ) , ReasoningBank ( Ouyang et al., 2025 ) , Memp ( Fang et al., 2025 )	AWM ( Wang et al., 2025b ) , ExpeL ( Zhao et al., 2024 ) , SkillOpt ( Yang et al., 2026 )
Training stream Contrastive only $\to$ all tasks	ExpeL ( Zhao et al., 2024 ) , H 2 R ( Ye et al., 2025 ) , Training-Free GRPO ( Cai et al., 2025 )	ReasoningBank ( Ouyang et al., 2025 ) , ACE ( Zhang et al., 2026 ) , SkillOpt ( Yang et al., 2026 )
Principle memory Remove principles	MARS ( Hou et al., 2026 ) , SkillRL ( Xia et al., 2026 ) , H 2 R ( Ye et al., 2025 )	Voyager ( Wang et al., 2023 ) , AWM ( Wang et al., 2025b )
Skill format Rich markdown $\to$ sentence	Trace2Skill ( Ni et al., 2026 ) , EvoSkill ( Alzubi et al., 2026 ) , SkillOpt ( Yang et al., 2026 )	ExpeL ( Zhao et al., 2024 ) , ACE ( Zhang et al., 2026 ) , Training-Free GRPO ( Cai et al., 2025 )
Skill storage Bank $\to$ single document	SkillRL ( Xia et al., 2026 ) , ReasoningBank ( Ouyang et al., 2025 ) , SWE-Exp ( Chen et al., 2025 )	SkillOpt ( Yang et al., 2026 ) , ACE ( Zhang et al., 2026 ) , Dynamic Cheatsheet ( Suzgun et al., 2025 )
Generation granularity Batch size $1\to 4$	ReasoningBank ( Ouyang et al., 2025 ) , ACE ( Zhang et al., 2026 ) , Dynamic Cheatsheet ( Suzgun et al., 2025 )	SkillOpt ( Yang et al., 2026 ) , Trace2Skill ( Ni et al., 2026 ) , Training-Free GRPO ( Cai et al., 2025 )

Appendix

Table 7: Prior self-evolving methods that adopt our choice or the ablated choice for each component in Figure 3 .

		Train+validation wall-clock time (h)
Model	Setting	pygments	sqlfluff	deepdiff	pydicom	Model Avg.
MiniMax-M2.7	SkillOpt	$118.5$	$134.1$	$105.7$	$121.7$	$120.0$
	Evolve w/o val	$42.9$	$46.3$	$47.2$	$73.9$	$52.6$
	Evolve w/ val	$127.5$	$107.8$	$139.0$	$153.5$	$131.9$
GPT-5.5	SkillOpt	$131.9$	$140.5$	$90.5$	$132.6$	$123.9$
	Evolve w/o val	$40.1$	$97.6$	$55.5$	$67.3$	$65.1$

Appendix

Table 8: Total wall-clock runtime in hours for the main SWE-smith experiments, with held-out test evaluation excluded. The Model Avg. column averages repositories within each model, and the Repo average block averages models within each repository. MiniMax-M2.7 is served on local GPUs, so its times also reflect local inference throughput.

ID	Component	Configuration	Hours / step
Ref.	Full pipeline	Mixed-only, validate $50$	$2.11$
A1	Skill format	Sentence	$2.41$
A2	Principle memory	No principles	$2.20$
A3	Storage	Single document	$2.08$
A4	Selection	Inject all	$1.83$
A5	Extraction batch	Batch size $4$	$1.58$

Appendix

Table 9: Wall-clock runtime per evolution step for the GPT-5.5 pydicom component ablations. The reference configuration is shaded. Only per-step time is reported; total runtime and the number of evolved steps are omitted.

Steps	Item	Type	Category	Gain
1	scope_reported_vs_scope_tested	Principle	Repair strategy	$+5.33$
1	diff_from_baseline_before_fixing	Principle	Repair strategy	$+9.00$
1	audit_full_modified_file_for_multiple_bugs	Skill	Repair strategy	$+2.00$
2	partial_fix_can_fail_tests_that_full_fix_passes	Principle	Repair strategy	$+5.00$
2	co_introduced_bugs_cluster_in_same_commit	Principle	Repair strategy	$+7.00$
2	verify_fix_completeness_with_full_test_suite	Skill	Repair strategy	$+2.33$

Appendix

Table 10: Single-item utility for all items in the final Claude-4.6 pygments bank, listed in learning order. Gain is measured relative to the no-memory baseline on the

100

-task holdout.

Self-Evolving Agents

Self-evolving agents improve their skills, memory, tools, policies, or environments by learning from interaction and evaluation feedback.

See all 427 papers

Sep 28, 2026

Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses

Harness evolution improves LLM agents by learning from execution trajectories, but existing experience- and skill-based methods are less effective on long-horizon tasks. As interactions grow, useful evidence can be buried by redundant or outdated context, making context management itself a key bottleneck. We introduce ContextEvo, a framework that learns a context policy from long-horizon trajectories. ContextEvo reconstructs the model-visible context at key decision points, identifies context-related failures, and applies targeted policy updates. Starting from the open-source Pi-agent harness, ContextEvo improves performance across three long-horizon task benchmarks, achieving results comparable to or better than several prominent agent harnesses, including Codex, OpenCode, and OpenClaw. Additional analyses show that fixed or locally evolved context strategies can fall short under long-horizon information pressure, while our methods adapt to the information demands of each environment.

Weiyuan Li, Jinghan Xu, Aili Chen +4

School of Data Science, Fudan University · Shanghai Key Laboratory of Data Science · College of Computer Science and Artificial Intelligence, Fudan University

Agent Harness Optimization Long-Horizon Agent Tasks LLM Context Management Self-Evolving Agents

Ask PDF

Jul 6, 2026

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill

s

and a branch-local meta-skill

m=(ψ,σ,α,π,\varepsilon)

whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

Zefeng Wang, Minxi Yan, Jinhe Bi +3

LMU Munich · The Chinese University of Hong Kong · MCML +1

LLM Agent Self-Improvement LLM Agent Skill Learning Recursive Self-Improvement

Ask PDF

Jun 4, 2026

OpenSkill: Open-World Self-Evolution for LLM Agents

Self-evolving agents requires adaptation after deployment, but existing approaches assume a usable learning loop, such as curated skills, successful trajectories, or verifier signals. Real open-world deployments may provide none of these, offering only a task prompt. In this work, we study open-world self-evolution, where an agent must build both its skills and its own verification signals from scratch, using open-world resources but no target-task supervision. We propose OpenSkill, a framework that bootstraps this loop: it acquires grounded knowledge and verification anchors from documentation, repositories, and the web, synthesizes them into transferable skills, and refines those skills against self-built virtual tasks grounded in the anchors rather than in target answers. The open world thus supplies both the knowledge to be learned and a supervision-independent practice environment, with target-task supervision reserved for final evaluation. Across three benchmarks and two target agents, OpenSkill attains the best automated pass rate while satisfying the no-supervision constraint. Analysis shows its skills transfer across models without model-specific adaptation, and its self-built verifier aligns with ground-truth outcomes despite never accessing them.

Zhiling Yan, Dingjie Song, Hanrong Zhang +8

Lehigh University · University of Illinois Chicago · University of British Columbia +3

LLM Agent Skill Learning LLM Agent Verification Self-Evolving Agents

Ask PDF

Abstract

Self-Evolving Agents

Explore similar work

Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses

MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

OpenSkill: Open-World Self-Evolution for LLM Agents