The performance of language agents depends jointly on model weights and the harness. Optimizing either alone can be constrained by the other component, limiting overall performance gains. Our analysis of naive alternating training reveals progressive incompatibility between the model and harness, which increasingly interferes with task execution and may also undermine model training, leading to performance degradation. To coordinate the evolution of models and harnesses, we introduce JoMoHa, to our knowledge the first framework to jointly train an executable harness generation policy and a task execution policy through reinforcement learning. JoMoHa outperforms the evaluated weight-only and harness-only baselines and achieves the highest success rates among the evaluated methods on ALFWorld, WebShop, and DBBench, averaging 70.3% and reaching 97.0% on ALFWorld. By learning both task execution and runtime improvement from its own generated trajectories, JoMoHa performs recursive self-improvement (RSI).
Figures & tables
Figure 1: Learning the runtime writer alongside the Solver on ALFWorld. The joint system reaches 97.0%, versus 90.5% for Frozen Engineer, 89.5% for Weight-only, and 80.5% for Harness-only. Weight-only does not use harness. Frozen Engineer keeps the initial code-writing model fixed but generates harnesses while training the Solver.
Figure 2: From one-sided learning to joint model–harness RL. Weight-only trains the Solver under the fixed base executor, without a learned harness. Harness-only trains the Engineer against a frozen Solver. JoMoHa trains both policies from the same executed task outcome. Solid policy boxes denote updated parameters; dashed boxes denote fixed components. Dashed arrows carry training feedback, not gradients through the generated program. SAO denotes the single-rollout actor–critic backbone ( Hou et al., 2026 ) ; Section 3 introduces the learning terms and notation.
Figure 3: Joint reinforcement learning of the Engineer and Solver. The Engineer writes a runtime program under which the Solver interacts with the task. The resulting shared reward R trains both policies, each on its own generated tokens.
Method
Harness adaptation
Engineer
Solver
Search-R1 / ReTool
Prescribed
–
RL
Meta-Harness
Code search
External
Fixed
Harness-R1
Learned edits
RL
Fixed
WHALE
Code search
External
RSFT
JoMoHa
Learned generation
RL
RL
Table 1: Comparison of model–harness learning procedures. Engineer and Solver columns describe parameter updates. A fixed Engineer can still search for new harnesses. For Harness-R1, the Solver is frozen during Engineer training. A dash denotes no separate harness Engineer.
Training scheme
Checkpoint
Success (%)
Alternating training
θS(0),h0
83.0
θS(1),h1
90.5
θS(2),h2
78.5
θS(3),h3
48.5
θS(4),h4
31.5
JoMoHa
144 rounds
97.0
Table 2: Performance degradation during alternating training on ALFWorld. Each alternating stage uses its corresponding Solver–harness pair (θS(k),hk) . The highest alternating-stage success and the result of JoMoHa are bolded. The fixed-code diagnosis is reported in Table 3 .
Solver
No harness
Fixed h2
Harness effect
Earlier θS(2)
82.5
78.5
−4.0
Later θS(3)
48.0
31.0
−17.0
Change
−34.5
−47.5
−13.0
Table 3: Execution interference and Solver degradation during alternating training. (a) Aggregate success rates; harness effect is the difference within each row in percentage points. (b) Case 1 from the ALFWorld trajectory analysis: outcomes and episode lengths across four Solver–harness combinations. Both h2 conditions execute the identical saved harness. No harness retains the base executor but removes the learned harness.
Method
ALFWorld
WebShop
DBBench
Avg.
Pick
Look
Clean
Heat
Cool
Pick2
All
Score
Succ.
Succ.
Qwen3.5-9B (with default harness)
75.8
66.7
16.9
28.8
9.2
47.5
40.6
66.0
31.2
61.0
44.3
Prompt-based Agentic Methods
ReAct
79.8
54.8
27.3
27.4
10.3
53.3
43.4
66.8
37.4
61.7
47.5
Self-Refine
70.7
45.2
11.7
28.8
6.9
57.4
39.0
61.7
29.0
57.3
41.8
Reflection †
91.9
90.5
24.7
56.2
19.5
73.8
59.2
61.2
43.6
64.7
55.8
Table 4: Benchmark comparison across agent and harness-optimization methods. Frontier models serve as fixed harness editors. Reflection uses success@2 ( † ). Avg. is the mean of ALFWorld All, WebShop success, and DBBench success; it excludes WebShop score. Best and second-best distinct values are bold on red and underlined on blue, including ties.
Harness
Earlier Solver
Later Solver
No harness
90.0
91.5
Initial program
91.5
92.5
Earlier program
95.0
95.0
Later program
95.0
96.5
Table 5: Harness utility at fixed Solver weights. Fixed-program cross-evaluation: columns fix Solvers and rows fix programs. Earlier and later denote programs and Solvers saved at checkpoints 48 and 144, respectively. The initial program comes from the initial Engineer. Colors follow Table 4 .
Variant
Success (%)
Joint-only
88.5
+ Schedule
92.0
+ Context
92.5
+ Peer
92.0
+ ValueNorm
91.5
JoMoHa
97.0
Table 6: Coordination-component additions. Success uses the saved harness and Solver. Each addition row changes one coordination component relative to Joint-only; the full method combines all four. Best and second best follow Table 4 .