The performance of language agents depends jointly on model weights and the harness. Optimizing either alone can be constrained by the other component, limiting overall performance gains. Our analysis of naive alternating training reveals progressive incompatibility between the model and harness, which increasingly interferes with task execution and may also undermine model training, leading to performance degradation. To coordinate the evolution of models and harnesses, we introduce JoMoHa, to our knowledge the first framework to jointly train an executable harness generation policy and a task execution policy through reinforcement learning. JoMoHa outperforms the evaluated weight-only and harness-only baselines and achieves the highest success rates among the evaluated methods on ALFWorld, WebShop, and DBBench, averaging 70.3% and reaching 97.0% on ALFWorld. By learning both task execution and runtime improvement from its own generated trajectories, JoMoHa performs recursive self-improvement (RSI).
Figures & tables
Figure 1: Learning the runtime writer alongside the Solver on ALFWorld. The joint system reaches 97.0%, versus 90.5% for Frozen Engineer, 89.5% for Weight-only, and 80.5% for Harness-only. Weight-only does not use harness. Frozen Engineer keeps the initial code-writing model fixed but generates harnesses while training the Solver.
Figure 2: From one-sided learning to joint model–harness RL. Weight-only trains the Solver under the fixed base executor, without a learned harness. Harness-only trains the Engineer against a frozen Solver. JoMoHa trains both policies from the same executed task outcome. Solid policy boxes denote updated parameters; dashed boxes denote fixed components. Dashed arrows carry training feedback, not gradients through the generated program. SAO denotes the single-rollout actor–critic backbone ( Hou et al., 2026 ) ; Section 3 introduces the learning terms and notation.
Figure 3: Joint reinforcement learning of the Engineer and Solver. The Engineer writes a runtime program under which the Solver interacts with the task. The resulting shared reward R trains both policies, each on its own generated tokens.
Method
Harness adaptation
Engineer
Solver
Search-R1 / ReTool
Prescribed
–
RL
Meta-Harness
Code search
External
Fixed
Harness-R1
Learned edits
RL
Fixed
WHALE
Code search
External
RSFT
JoMoHa
Learned generation
RL
RL
Table 1: Comparison of model–harness learning procedures. Engineer and Solver columns describe parameter updates. A fixed Engineer can still search for new harnesses. For Harness-R1, the Solver is frozen during Engineer training. A dash denotes no separate harness Engineer.
Training scheme
Checkpoint
Success (%)
Alternating training
θS(0),h0
83.0
θS(1),h1
90.5
θS(2),h2
78.5
θS(3),h3
48.5
θS(4),h4
31.5
JoMoHa
144 rounds
97.0
Table 2: Performance degradation during alternating training on ALFWorld. Each alternating stage uses its corresponding Solver–harness pair (θS(k),hk) . The highest alternating-stage success and the result of JoMoHa are bolded. The fixed-code diagnosis is reported in Table 3 .
Solver
No harness
Fixed h2
Harness effect
Earlier θS(2)
82.5
78.5
−4.0
Later θS(3)
48.0
31.0
−17.0
Change
−34.5
−47.5
−13.0
Table 3: Execution interference and Solver degradation during alternating training. (a) Aggregate success rates; harness effect is the difference within each row in percentage points. (b) Case 1 from the ALFWorld trajectory analysis: outcomes and episode lengths across four Solver–harness combinations. Both h2 conditions execute the identical saved harness. No harness retains the base executor but removes the learned harness.
Method
ALFWorld
WebShop
DBBench
Avg.
Pick
Look
Clean
Heat
Cool
Pick2
All
Score
Succ.
Succ.
Qwen3.5-9B (with default harness)
75.8
66.7
16.9
28.8
9.2
47.5
40.6
66.0
31.2
61.0
44.3
Prompt-based Agentic Methods
ReAct
79.8
54.8
27.3
27.4
10.3
53.3
43.4
66.8
37.4
61.7
47.5
Self-Refine
70.7
45.2
11.7
28.8
6.9
57.4
39.0
61.7
29.0
57.3
41.8
Reflection †
91.9
90.5
24.7
56.2
19.5
73.8
59.2
61.2
43.6
64.7
55.8
Table 4: Benchmark comparison across agent and harness-optimization methods. Frontier models serve as fixed harness editors. Reflection uses success@2 ( † ). Avg. is the mean of ALFWorld All, WebShop success, and DBBench success; it excludes WebShop score. Best and second-best distinct values are bold on red and underlined on blue, including ties.
Harness
Earlier Solver
Later Solver
No harness
90.0
91.5
Initial program
91.5
92.5
Earlier program
95.0
95.0
Later program
95.0
96.5
Table 5: Harness utility at fixed Solver weights. Fixed-program cross-evaluation: columns fix Solvers and rows fix programs. Earlier and later denote programs and Solvers saved at checkpoints 48 and 144, respectively. The initial program comes from the initial Engineer. Colors follow Table 4 .
Variant
Success (%)
Joint-only
88.5
+ Schedule
92.0
+ Context
92.5
+ Peer
92.0
+ ValueNorm
91.5
JoMoHa
97.0
Table 6: Coordination-component additions. Success uses the saved harness and Solver. Each addition row changes one coordination component relative to Joint-only; the full method combines all four. Best and second best follow Table 4 .
Agent performance depends jointly on the model parameters and the executable harness code that manages context and control flow. Optimizing either component in isolation can leave the system bottlenecked by its frozen counterpart: weight updates can change which harness is effective, while harness updates can change which model capabilities are exposed. Existing joint-adaptation methods optimize weights and textual prompts but leave the broader harness fixed. We propose Weight-Harness Alternating LEarning (WHALE), a simple recipe that alternates two phases: updating the model under the current harness, then searching for a better harness under the updated model. We instantiate these two phases with online rejection-sampling fine-tuning and Meta-Harness, respectively. When to switch is a key design choice: to separate real improvements from noise without over-optimizing against a changing counterpart, WHALE uses either fixed phase durations or an adaptive patience rule over training signals. Using Qwen3.5-2B/4B agents across three domains (search question answering, mathematical reasoning, and chess puzzles), WHALE outperforms weight-only, harness-only, and Fast-Slow Training by 4.15-24.38 percentage points in best mean@8 accuracy. Either component can be the bottleneck: harness search matches peak weight-only accuracy with far fewer rollouts in SearchQA, but improves math accuracy only after a weight update. Small interleaved updates also outperform stagewise weight-then-harness optimization in accuracy and rollout cost. The code is available at https://github.com/krafton-ai/WHALE.
Language agents rely on external harnesses to track state, organize workflows, and verify answers. Beyond providing tools and information, these harnesses supply control decisions about what to investigate, whether to revise, and when to stop. Training on successful harness-supported trajectories can improve task performance while leaving these decisions dependent on runtime intervention. We ask whether harness-supported experience can also teach the model to make these decisions, allowing the division of control to change as the model learns. We call this objective harness internalization: learning to assume specified control responsibilities while retaining task performance after the corresponding support is withdrawn. We introduce HARNESS ANNEALING TRAINING (HAT), which combines explicit control supervision with a curriculum over teacher trajectories collected under progressively weaker harnesses. Experiments with 9B and 35B models on SWE-QA and SWE-QA-Pro evaluate every checkpoint under four deployment harnesses. Selected annealed checkpoints operating with tools alone achieve scores close to those of their respective starting checkpoints deployed with the full harness. The benefits vary with model scale and deployment configuration, and further annealing does not uniformly improve performance. These findings suggest that harness-supported experience can help reduce the runtime control required by a trained agent.
Yingxuan Yang, Huacan Chai, Ying Wen
Shanghai Jiao Tong University · University of California, Berkeley
Language-model agents are increasingly deployed through diverse harnesses that differ in system prompts, tool schemas, control loops, and trajectory formats. The same model can perform unevenly across these interfaces, making robustness to harness variation an important objective. A natural approach is to train a shared policy through multiple harnesses, but doing so introduces a scheduling problem: each training step should favor a harness that currently provides a useful learning signal while also producing an update that benefits the other harnesses. We develop HarnessBandit, an online scheduler that selects one harness per optimizer step. After a group-relative policy optimization (GRPO) update, it observes learnability -- the mean absolute advantage on the batch -- and transferability -- the cosine between a low-dimensional gradient sketch of the current harness and exponential moving averages of the remaining harnesses. The two signals are fused after pooled sliding-window min-max normalization and sampled with a visit-dependent bonus and an explicit exploration floor. We train Qwen3.5-2B across six harnesses on ClawGym and evaluate on PinchBench (held-out tasks, in-distribution OpenClaw) and ClawEval (held-out tasks and harness). HarnessBandit improves over mixed-batch multi-harness training on both benchmarks, while training diagnostics indicate that learnability and transferability provide distinct, evolving signals.