Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

Qiankai Xu

cs.AISep 29, 2026

Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

Authors: Qiankai Xu

Organizations: Nanjing University

Abstract

A harness is the code around a language-model agent that organizes prompts, calls tools, manages context, and controls execution. As models grow stronger, recent work has begun to let agents improve their own harnesses, a line of work known as self-evolving harnesses. In most existing methods, a separate proposer running on a human-designed harness modifies the solver's harness, and a separate harness is evolved for each benchmark. Real-world tasks come from many domains, so both the evolution and the evaluation of a harness should cover a diverse range of tasks. We propose a framework close to recursive self-improvement: the same frozen model, on the same version of the harness, first solves tasks as the solver and then, as the proposer, reads the complete run records and directly edits the harness that runs it. Each evolution batch draws tasks from five benchmarks in different domains. To measure generalization, training and held-out tasks are strictly separated, and we additionally evaluate on five out-of-distribution benchmarks never used during evolution. We frame the evolution process as deep-learning training with two stages, multi-task pretraining and continual training. Starting from a 49-line seed harness, the harness obtained at the end of the first stage improves the average score by 4.48 points on the in-distribution benchmarks and by 12.64 points on the out-of-distribution benchmarks, surpassing Codex on the former and matching it on the latter. In the second stage, continued evolution on Claw-Eval, one of the out-of-distribution benchmarks, further raises the score on that benchmark from 66.17 to 68.06, exceeding Codex. We also provide an in-depth analysis of the mechanisms that emerged during evolution, including output truncation, history compaction, and independent review.

Figures & tables

Figure 1: Overview of the framework. The solver and the proposer are the same agent, run in separate sessions with the same frozen model $M$ and the same harness, which consists of the code $C_{t}$ and the evolution notes $N_{t}$ . In Iterations 1–5, each batch mixes new tasks from five benchmarks; in Iterations 6–7, it combines new Claw-Eval tasks with success anchors from the first stage. The solver runs the batch and produces the run records $E_{t}$ : trajectories, scores, and grader outputs. The proposer reads these records and the notes, modifies the harness code within the budget of new lines $a_{t}$ , and updates the notes. Once the new version passes the engineering checks, its code $C_{t+1}$ and notes $N_{t+1}$ are used in the next iteration.

Figure 2: Harness evolution viewed as two-stage training (schematic). In the first stage (Iterations 1–5), each batch takes 12 new tasks from each of five benchmarks, 60 tasks in total. In the second stage (Iterations 6–7), each batch combines 40 tasks from a new benchmark with 20 success anchors from the first stage, rerun with the current harness. The shaded surfaces sketch the loss of the two objectives over the space of harness code. After Iteration 5 the objective changes while the code stays at

C_{5}

(dashed line), and Iterations 6 and 7 continue from there. Seed, Iteration 5, and Iteration 7, the three versions evaluated on held-out tasks, have 49, 430, and 524 lines of Python. Bottom left: the budget of new lines

a_{t}

shrinks during the first stage, is raised again when the second stage begins, and then shrinks again, like a learning-rate schedule. Bottom right: each update turns the run records

E_{t}

and the notes

N_{t}

into a code edit and revised notes

N_{t+1}

. Surface shapes, step lengths, and bar heights are illustrative.

Method	Same model	Same harness	Start	Editable scope	Evolution tasks	Acceptance
SICA	✓	✓	Hand-designed	All code	Same-type benchmarks	Score-based selection
DGM	✗	✓	Minimal	All code	Single benchmark	Score-based selection
Meta-Harness	✗	✗	Hand-designed	All code	Same-type benchmarks	Score-based selection
AHE	✓	✗	Minimal	Preset components	Single benchmark	Effect-based rollback
Self-Harness	✓	✓ †	Minimal	Preset components	Single benchmark	Regression tests
HarnessX	✗	✗	Hand-designed	Preset components	Single benchmark	Regression tests

Table 2: Comparison of recent self-evolving harness work with ours. Same model : the proposer that modifies the harness and the solver that solves tasks use the same base model. Same harness : the proposer itself runs on the harness being evolved. Evolution tasks : the tasks from which one harness receives feedback during evolution; single benchmark means one benchmark, and same-type benchmarks means several benchmarks or datasets of the same task type. Many methods are evaluated on several benchmarks but evolve a separate harness for each, which still counts as a single benchmark. Acceptance : how a modified version is admitted to the next round.

Benchmark	Task type	Total	Train	Held-out
In-distribution benchmarks
Terminal-Bench 2.1	Long-horizon engineering tasks in a terminal	89	60	29 $\times$ 3
GAIA2	Asynchronous interaction in dynamic app environments	800	60	100
OfficeQA Pro	Treasury document retrieval and numerical reasoning	133	60	73 $\times$ 2
$\tau^{3}$ -Bench banking	Multi-turn service dialogue with a simulated user	97	60	37 $\times$ 2
SWE-Bench Pro	Issue resolution in code repositories	731	60	100

Table 3: Benchmarks and task counts. Benchmarks with few held-out tasks are evaluated repeatedly and averaged: each of the 29 Terminal-Bench 2.1 tasks is run three times, written as 29

\times

3; for OfficeQA Pro and

\tau^{3}

-Bench banking we take two complete runs, written as 73

\times

2 and 37

\times

2. The out-of-distribution benchmarks have no first-stage training tasks, and the Claw-Eval training tasks are used only in the second stage.

Figure 3: Held-out scores of the three harnesses with GPT-5.6 Sol (high). (a) The five in-distribution benchmarks; (b) the five out-of-distribution benchmarks. Scores on APEX-Agents and Claw-Eval are the mean official score multiplied by 100; on the other benchmarks they are success rates (%). Averages weight the five benchmarks equally.

Figure 4: Results on the Claw-Eval held-out tasks; the four harnesses use the same 100 tasks and the same model. (a) Mean official score over the 100 tasks, multiplied by 100. (b) Number of tasks with a score of at least 0.75 in each category; parentheses give the number of tasks in the category.

State	Lines	Main changes
Seed	49	System prompt, bash tool, and a basic tool-calling loop
Iteration 1	279	Output truncation; command time limits; history compaction; image viewing
Iteration 2	343	Completion check before finishing; image size cap
Iteration 3	394	Independent review in a fresh context
Iteration 4	414	End-of-conversation detection by word matching; image format validation
Iteration 5	430	Task-specific rules (set operations, deadlines); review prioritizes unchecked requirements

Table 4: Total Python lines of the harness at each iteration (including blank lines and comments) and the main code changes.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Share of tasks (%) on each benchmark in which Seed aborted because of context overflow, and in which each of three mechanisms of Iteration 5 was triggered, computed over all Seed and Iteration 5 evaluation records used in Figure 3 . Truncation: the model received a clipped tool output. Compaction: earlier turns of the conversation were removed. Independent review: at the end of the task, the result was checked again in a fresh context.

Explore similar work

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
Aug 9, 2026Tailin ZhouEvolving Harness Large Language Model Agents
HKUST
Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
May 28, 2026Minhua Lin, Juncheng Wu, Zijun Wang +14Evolving Harness Evolutionary Optimization Methods
1The Pennsylvania State University · 2UC Santa Cruz · 3Amazon +3
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Aug 3, 2026Luan Zhang, Ruochen Zhou, Dandan Song +9Evolving Harness Agent Harness
Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China

Aug 9, 2026, cs.AIJ/K move · Enter open · S save

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

Tailin Zhou

HKUST

Evolving Harness Large Language Model Agents Recursive Self-Improvement Feedback Frozen Backbone

Open paper AskPDF arXiv

Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the \emph{harness}---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is \emph{task-specific and continuously evolvable}: each task family maintains its own harness, which is hot-swapped across iterations through a fixed task-injection seam and rewritten using environment feedback. We introduce \textbf{Hierarchical Self-Improvement (HSI)}, a framework in which a single frozen LLM

M

operates across three hierarchical scopes: a task harness

H

that executes tasks, an evolver that rewrites

H

, and a meta-evolver that rewrites the evolver's strategy code under a frozen outer anchor. A thinking-on/off design isolates the contribution of harness evolution by disabling reasoning during task execution while enabling it during self-modification. HSI is bounded by two factors: a \emph{feedback-fidelity bound}, since evolution requires informative reward signals to guide selection, and a \emph{backbone capability bound}, since harness redesign cannot overcome limitations of the frozen model. On BALROG with DeepSeek-V4-Flash-Preview as the frozen backbone, HSI achieves consistent gains over the initial harness on moderate-difficulty tasks (

+39.3

on BabyAI,

+33.0

on Crafter,

+25.0

on TextWorld, and

+15.0

on MiniHack, all in raw % Progress), while obtaining strong held-out generalization on BabaIsAI sub-suites (

0.98

best-test on BreakStop and

1.00

on GoTo from a

20\%

unseen split). On tasks beyond the backbone's capability (NLE), harness evolution provides no improvement. These results demonstrate task-specific harness evolution as a viable axis for improving frozen LLM agents under clear empirical limits. Code is available at https://github.com/TailinZhou/hsi.

Abstract

Explore similar work

Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses