CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning

Jianwen Chen; Peng Xia; Kaide Zeng; Siwei Han; Zhen-Yu Zhang; Charles Fleming; Xinyu Ye; Huaxiu Yao

cs.LGOct 27, 2026

CoHARL: Co-Evolving Agent Policies and Executable Harnesses with Reinforcement Learning

Authors: Jianwen Chen, Peng Xia, Kaide Zeng, Siwei Han, Zhen-Yu Zhang, Charles Fleming, Xinyu Ye, Huaxiu Yao

Organizations: University of North Carolina at Chapel Hill · RIKEN · Cisco

Abstract

Large language model (LLM) agents interact with their environments through executable harnesses that govern tool use and feedback. Optimizing a harness for a fixed policy overlooks how its behavior changes during learning, while training a policy under a fixed harness leaves the interface that generates its training experience unchanged. This motivates adapting the harness throughout policy training. We introduce CoHARL, a population-based reinforcement learning framework that co-evolves a shared policy and a population of executable harnesses. The policy learns from rollouts across the population, and a proposer uses its execution traces to revise harness code during training. CoHARL achieves a 27.6% mean resolved rate on SWE-rebench, improving over harness-only search by 6.4 percentage points. Experiments on SWE-bench Verified and Terminal-Bench 2.0 show gains in software engineering and terminal-based tasks. Under matched training budgets, CoHARL also outperforms searching for a harness once before RL and alternating policy updates with revisions to a single active harness.

Figures & tables

Figure 1: Coupling policy optimization with harness evolution. Policy-only RL adapts policy weights under a fixed harness (a), whereas harness-only search adapts the harness under a fixed policy (b). CoHARL jointly adapts a shared policy and a harness population (c), using execution traces to guide harness edits and population rollouts to update the policy.

Figure 2: Overview of CoHARL. A shared policy learns from population rollouts, while a proposer revises harness code using current-policy traces and historical accuracy–cost trade-offs. Accepted candidates enter the pool at step boundaries. The validation cascade shown applies to TB-2; SWE uses contract checks followed by pool trials. Blue and green rollout blocks represent generated tokens and tool observations, respectively; only generated tokens contribute to the policy loss.

Figure 3: Training rewards for the full method and three controls in Table 4 . Each point is the mean raw reward over retained training samples, before length penalties, KL adjustments, or normalization. Curves show unsmoothed values, not evaluation resolved rates.

Method	Weights	Harness	Resolved
Base	Qwen3.5-35B-A3B	SWE-mini	17.3%
Meta-Harness	Qwen3.5-35B-A3B	Meta-Harness ∗	21.2%
Plain GRPO	RL-trained	SWE-mini	20.3%
CoHARL (ours)	RL-trained	Co-evolved	27.6%

Table 1: Performance on the 110-task SWE-rebench evaluation split. Resolved rates are averaged over three attempts per task (avg@3) at temperature 0.7. ∗ Our Meta-Harness reproduction, initialized from SWE-mini.

Methods	Weights	Harness	Resolved
Closed-source models
GPT-4o †	-	-	33.2%
Claude 3.5 Sonnet †	-	-	49.0%
Qwen3.5-35B-A3B-based methods
Base	Fixed	SWE-mini	44.2%
Meta-Harness	Fixed	Meta-Harness ∗	52.0%

Table 2: Resolved rates on the 500-task SWE-bench Verified benchmark with one attempt per task. SWE-rebench-trained agents are evaluated without further adaptation. † Provider-reported results under original evaluation settings ( OpenAI, 2025 ; Anthropic, 2025 ) . ∗ Our Meta-Harness reproduction, initialized from SWE-mini.

Method	Harness	Success
Base	Terminus-2	26.6%
Meta-Harness	Meta-Harness	33.7%
Flat GRPO (Shepherd) †	Terminus-2	34.2%
CoHARL (ours)	Co-evolved	36.8%

Table 3: Task success rates on Terminal-Bench 2.0, evaluated through Harbor with Terminus-2 as the baseline harness. For our evaluations at temperature 0.6, success is averaged within each task over returned episodes and then across tasks; tasks with no returned episode score zero.

\dagger

: Shepherd ( Yu et al., 2026a ) , 89 tasks, avg@5.

Configuration	Resolved rate	Resolved
CoHARL (full)	25.5%	28/110
A1: Fixed-harness RL	20.0%	22/110
A2: No archive context	20.0%	22/110
A3: No execution traces	20.9%	23/110
A4: Search-then-RL	22.7%	25/110
A5: Single-harness alternating RL	19.1%	21/110

Table 4: Component ablations and training-strategy comparisons on SWE-rebench under matched training budgets (30 steps). We report strict resolved rates and counts over 110 tasks with one attempt per task.

Figure 4: Harness utilization and population turnover. (a) Per-step rollout shares across harness versions; darker blue indicates larger shares, while gray denotes unknown pool membership. (b) Recorded lifecycle events over the same 36-step segment.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Group	Hyperparameter	Value
Model	Policy	Qwen3.5-35B-A3B (MoE, $\approx$ 3B active)
	Initialization	Initial model checkpoint
	Adaptation	LoRA, rank 32
RL	Algorithm	GRPO, group-relative advantage
	Importance-ratio clip (DAPO)	$[0.80,\,1.28]$
	PPO epochs	2

Appendix

Table 5: Training configuration for CoHARL on SWE-rebench.

Setting	SWE	TB-2	Historical A1–A3
Training entries	400	843	400
Task groups $\times$ rollouts/group	$12\times 4$	$18\times 8$	$8\times 4$
Training temperature	0.7	0.9	0.7
Episode timeout (s)	600	1,200	600
Policy-monitoring tasks	32	20	32

Appendix

Table 6: Domain-specific and historical diagnostic configurations. The last column describes earlier A1–A3 diagnostics, not the matched 30-step comparisons in Table 4 . Sampling budgets are nominal.

Configuration	Statistic	Group retention
CoHARL (full)	Mean over recorded filtering steps	79.1%
A1: Fixed-harness RL	Mean over recorded filtering steps	52.2%
CoHARL (full)	First included step ( $11/12$ groups)	91.7%
A1: Fixed-harness RL	First included step ( $3/8$ groups)	37.5%
A2: No archive context	Mean over 27 recorded filtering steps	64.2%
A3: No execution traces	Mean over 24 recorded filtering steps	74.0%

Appendix

Table 7: SWE group-retention diagnostics. Means are arithmetic averages over recorded filtering steps, excluding protective fallback steps. First-step values refer to the first included step, not necessarily the first scheduler step.

Figure 5: Harness-edit example and historical Pareto archive. (a) An observational parent–child example: a repeated-command warning is followed by a different command, although both runs remain unresolved. Code excerpts are abridged. (b) A separate schematic of accuracy–cost archive retention, not measured results from the example in (a). Archive retention is distinct from training-pool admission.

Framework	Adaptation target	Edit generation	Training organization
EvoTrainer ( Chen et al., 2026a )	Training-side diagnostics and training recipes	Trainer agent diagnoses artifacts and proposes interventions	Versioned training branches with promotion, pruning, and skill reuse
WHALE ( Kim et al., 2026 )	Executable harness and model weights	Meta-Harness proposer searches under frozen weights	Weight updates under one selected harness alternate with search; instantiated with online RSFT
HASE ( Luo et al., 2026 )	Guidance harness; local evaluator where permitted	Task policy emits both solutions and harness edits	GRPO with rollout-local edits; reviewed candidates update one persistent harness at phase boundaries
HarnessForge ( Chen et al., 2026b )	Population of harness–policy pairs	Meta-agent uses trajectory feedback and an archive	Each surviving harness receives a dedicated policy adapter
HarnessBandit ( Wei et al., 2026a )	Shared policy and harness scheduling	No online harness-code evolution	One harness per update, selected from a given set using learnability and transferability signals
CoHARL	Population of executable agent-side harnesses and shared policy weights	Separate proposer uses current-policy traces and a historical Pareto archive	Shared-policy RL across scheduled pool members; accepted candidates enter at step boundaries

Appendix

Table 8: Design comparison with closely related frameworks, based on their described methods. These distinctions are not a performance ranking.

Agent Harness Optimization

Agent harness optimization improves the prompts, tools, memory, workflows, and control code that shape how AI agents act.

See all 185 papers

EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
Jun 2, 2026Guhong Chen, Yingcheng Shi, Yongbin Li +6RL for Code Generation Agentic RL
1Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · 2Tongyi Lab , Alibaba Group · 3Alibaba Group +1
Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
Aug 3, 2026Shuai Shao, Kangning Zhang, Qingyao Li +7Agent Harness Optimization LLM Agent Self-Improvement
Shanghai Jiao Tong University · Xiaohongshu Inc. · Southeast University
Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents
Jul 17, 2026Zhengyu Chen, Teng Xiao, Huaisheng Zhu +3Agent Harness Optimization LLM Agent Workflow Optimization
Meituan · Allen Institute for AI · Independent

Jun 2, 2026J/K move · Enter open · S save

EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning

Guhong Chen, Yingcheng Shi, Yongbin Li, Binhua Li+5

1Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · 2Tongyi Lab , Alibaba Group · 3Alibaba Group+1

RL for Code Generation Agentic RL Reinforcement Learning Agent Harness Optimization LLM Agent Skill Learning LLM Agent Training+1

Open paper AskPDF arXiv

Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static. This limitation sharpens in agentic RL, where shifting bottlenecks and scalar rewards mask diverse failure modes. We introduce EvoTrainer, an autonomous training framework that co-evolves LLM policies and training-side harnesses through empirical feedback: it diagnoses rollout-level evidence, revises diagnostics, backtests interventions, and accumulates reusable skills. Evaluated on mathematical reasoning, competitive-programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds the human-engineered RL references under the same data, codebase, and evaluation protocol, with the largest gain on long-horizon agentic SWE. Trajectory analyses show that retained strategies diverge across domains, evolving diagnostics prevent invalid high-scoring branches from being promoted, and reusable skills shape later search. Autonomous LLM RL should move beyond recipe search toward joint evolution of policies and the training harnesses that interpret them.

Abstract

Agent Harness Optimization

Explore similar work

EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning

Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories

Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents