Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

Kangning Zhang; Wenxiang Jiao; Bowen Fang; Shuai Shao; Jiachen Zhu; Rong Shan; Shijian Wang; Hao Wang

cs.AINov 3, 2026

Harness-S: Learning from Sibling Harnesses for Generalizable Agent Training

Authors: Kangning Zhang, Wenxiang Jiao, Bowen Fang, Shuai Shao, Jiachen Zhu, Rong Shan, Shijian Wang, Hao Wang,

Organizations: Shanghai Jiao Tong University · Xiaohongshu Inc. · Zhongguancun Institute of Artificial Intelligence · Southeast University ·

Abstract

Harness evolution from a common base exhibits Locality, Complementarity, and Nonstationarity: variants repair different failures, and their utility changes as the policy learns. Rather than seeking a universally superior harness, we introduce Harness-S to selectively rescue failures of a base execution system through sibling harnesses and transfer the resulting capabilities into a single policy that remains effective under the original harness. Directional Harness Evolution develops specialized sibling harnesses along distinct repair directions, forming a reusable Sibling Harness Pool that remains fixed during policy training. Complementary Harness Rescue Training invokes this pool only when all initial attempts under the base harness fail. The same policy generates fresh rollouts under selected auxiliary harnesses, and these trajectories replace the failed group for reinforcement learning. An Adaptive Harness Router selects the Top-k variants using Harness Behavior Profiles and voting across multiple LLMs. Policy-aware profile re-anchoring refreshes this evidence as the policy learns. Across AppWorld, SWE-Gym, and DBBench, the trained policy substantially outperforms policy-only training when returned to the original base harness. Behavioral analyses and held-out cases support the transfer of harness-elicited behaviors into the policy itself. Reusing the router for Rank-1 inference provides additional gains at the same executor rollout count.

Figures & tables

Figure 1: Locality, complementarity, and nonstationarity in harness evolution (Qwen3.5-4B, AppWorld). (a) Repaired and regressed tasks relative to

H_{0}

;

\Delta

Solved denotes repaired minus regressed tasks. (b) Individual success rates (bars), oracle coverage using one rollout under each of

H_{1},\ldots,H_{K}

(purple), and

H_{0}

pass@

K

using the same number of rollouts (orange). (c) Paired outcomes for fixed harnesses as the policy is trained;

\Delta

Solved is measured against

H_{0}

at each checkpoint.

Figure 2: Overview of Harness-S. (a) Directional Harness Evolution develops distinct repairs and retains valid variants in a Sibling Harness Pool, fixed during policy training. (b) Complementary Harness Rescue Training routes all-failure tasks to Top-

K

auxiliary harnesses and updates the shared policy, with periodic profile re-anchoring. (c) Rescue training transfers knowledge from sibling harnesses into the shared policy, improving execution under the original

H_{0}

; Rank-1 routing provides further gains (illustrative AppWorld example).

Model / method	Param.	Test harness	AppWorld		SWE-Gym		DBBench
Model / method	Param.	Test harness	Pass@3	Avg@3	Pass@3	Avg@3	Pass@3	Avg@3
Frontier models
GPT-6-Sol	—	$H_{0}$	82.82	76.34	53.13	47.50	71.33	68.89
DeepSeek-v4-Flash	—	$H_{0}$	81.68	77.61	51.88	46.25	75.00	70.44
GLM-5.3-Flash	—	$H_{0}$	80.15	65.65	46.88	43.13	76.67	72.11
Base model and policy-only training

Table 1: Main results on long-horizon agent tasks. All scores are percentages; higher means are better. Bold marks the best mean at each capacity among the compared non-frontier methods.

H_{\mathrm{best}}

is one selected harness held fixed across test tasks;

H_{\mathrm{rank1}}

is the router’s top-ranked harness selected separately for each task. DSH denotes DeepSeek Harness. Scores are averaged over three independent randomized runs, each with three executor rollouts per task. Harness-S entries report mean

\pm

standard deviation across these runs.

Figure 3: Training signals and policy improvement for Qwen3.5-4B on AppWorld. (a) Fraction of effective groups containing both successes and failures. (b) Fraction of retained policy tokens belonging to these effective groups. Both quantities are aggregated within each ten-step stage. (c) Checkpoint success on the same validation set under

H_{0}

Rescue success by training stage
	GRPO	Random Harness	Rule Router (w/o Profile)	Top-4 $\times 2$	Top-8 Fixed	Top-8 Adaptive
Steps 1–10	—	25.00	40.00	43.48	35.71	52.38
Steps 11–20	—	18.18	26.67	43.75	24.24	66.67
Steps 21–30	—	20.00	27.78	38.89	26.67	45.71
Steps 31–40	—	25.00	44.44	33.33	38.89	50.00
Final test performance

Table 2: Rescue success and final test performance on AppWorld (Qwen3.5-4B), in percent. Rule Router uses no Harness Behavior Profiles. Top-8 Fixed uses unchanged profiles; Top-8 Adaptive (Harness-S) uses profile re-anchoring. Bold marks the best result; — denotes no rescue.

Behavior	Base	GRPO	Harness-S
AppWorld
Schema lookup before first API use $\uparrow$	22.9	38.2	67.6
Multiple API calls in one execute block $\uparrow$	24.8	20.5	31.5
Explicit pagination loops $\uparrow$	13.8	33.5	38.3
Repeated execute code $\downarrow$	15.4	30.1	9.8
Read-back after the last write $\uparrow$	69.2	56.9	69.4

Table 3: Behavior frequencies under

H_{0}

(%). An LLM labels complete evaluation trajectories; percentages report the fraction exhibiting each behavior. Arrows indicate preferred directions.

Stage	Observed execution
Case 1: Set intersection (held-out test instance)
Training rescue	H2 selects liked songs that also occur in the library or playlists, then updates and sends the draft. This successful rollout is used at update 10.
GRPO	Uses a union of liked songs and the other sources, sending an overinclusive list. Failure.
Harness-S	Applies the intersection to the new song instances and sends the correct email. Success.
Case 2: Account recovery (held-out validation instance)
Training rescue	H5 completes a Venmo account workflow: reads email codes, resets the password, and verifies login. This successful rollout is used at update 3.

Table 4: Two AppWorld behavior-transfer cases. All eight initial

H_{0}

attempts fail in training; the successful rescue enters an applied policy update. Held-out comparisons use

H_{0}

Harness	Before training			After training
	Pass@3	Avg@3	$H_{0}$ rate	Pass@3	Avg@3	$H_{0}$ rate
$H_{0}$	42.62	31.93	—	67.56	56.87	—
Rank-1	56.87	45.42	1.53	68.32	61.45	22.52
Rank-2	53.44	42.11	6.87	66.03	55.60	14.89
Rank-3	50.38	39.19	4.96	64.89	54.20	13.36

Table 5: AppWorld router rankings before and after Harness-S training (Qwen3.5-4B; %). Each harness receives three rollouts per task.

H_{0}

rate is the task fraction placing

H_{0}

at that rank (—: fixed baseline). Bold marks best task scores.

Figure 4: Cross-harness Pass@3 of Qwen3.5-4B. Mixed-harness GRPO uses DSH

H_{0}

and Claude Code. Annotations show Harness-S gains over GRPO in percentage points.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Train	Validation	Test	Total
AppWorld	320	150	262	732
DBBench	1,600	200	300	2,100
SWE-Gym	240	100	160	500

Appendix

Table 6: Dataset splits. Number of task instances in each benchmark.

Case 1: $S_{6}$ — complete pagination across sources
Edit: add an after-tool reminder to continue pagination until explicit exhaustion.
state.hooks.pagination-completion-guidance
Before ( $H_{0}$ )	After ( $S_{6}$ )
⬇ null	⬇ { " action ": " warn ", " condition ": " pagination_present ", " event ": " after_tool ", " message ": " Pagination metadata is present . Continue only until an empty or short page or an authoritative total ; do not treat the first page as complete .", " parameters ": {} }
When	Aggregating information across inbox, spam, and archive.
Behavior	Paginate through all specified sources, merge the results, and remove duplicates. The paired $H_{0}$ execution attempts a single-pass match and omits information from some sources.

Appendix

Table 7: Three sibling configuration edits and favorable behavior-profile excerpts on AppWorld. Each paired example succeeds with the sibling and fails under

H_{0}

. Code shows the changed entry; [...] marks omitted rule text.

Figure 5: Shared repair coverage. Each cell gives the percentage of the row sibling’s repairs also supplied by the column sibling; diagonal entries are omitted.

Setting	Value
Search	3 rounds; proposal budgets 7 / 8 / 8; 4 parent slots
Development tasks	Sampled from the 320-task training set
Repair direction	2–3 related root causes per candidate; at least 2 distinct failed tasks per cause
Local evaluation	At most 30 tasks, including at most 10 successful controls
Revision budget	Initial candidate plus at most one revision
Harness editor	GLM-5.3; temperature 0.6; thinking enabled with high reasoning effort

Appendix

Table 8: Selected settings for the AppWorld directional evolution run. Proposal counts are budgets, not counts of accepted variants.

Setting	Training rollouts	Training GPU $\cdot$ h	$H_{0}$ test Avg@3 (%)
Plain $H_{0}$ GRPO	2,560 (100.00%)	50.58	48.98
GRPO ( $H_{0}$ rescue)	3,616 (141.25%)	97.77	51.91
DSH $H_{0}$ + Top-4	2,560 (100.00%)	58.47	47.84
DSH $H_{0}$ + Claude Code	2,560 (100.00%)	50.03	46.69
Harness-S / Top-8	3,032 (118.44%)	67.19	56.87

Appendix

Table 9: GRPO training cost and

H_{0}

test Avg@3 on AppWorld (Qwen3.5-4B). All settings use 40 updates. Parentheses give rollout counts relative to plain

H_{0}

GRPO.

Method	Search rollouts	Training rollouts	GPU $\cdot$ h	API tokens (M)
Meta-Harness	8,000	0	35.24	4.89
WHALE	8,640	2,560	95.84	9.80
HarnessForge	5,680	0	65.39	24.40
Harness-S	4,010	3,032	75.79	18.69

Appendix

Table 10: Full-pipeline resources for harness search and policy adaptation. Training rollouts count additional samples; HarnessForge reuses successful search trajectories. API tokens are in millions (M).

Agent Harness Optimization

Agent harness optimization improves the prompts, tools, memory, workflows, and control code that shape how AI agents act.

See all 192 papers

HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning
Sep 12, 2026Hongliang Wei, Xiaobing Tu, Yinggui Wang +6RL for LLM Agents Agent Harness Optimization
Harbin Institute of Technology · Alibaba Cloud
Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses
Aug 30, 2026Xinke Jiang, Zhixin Zhang, Zhibang Yang +6Multi-Agent LLM Systems Agent Harness Optimization
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +3
Turbo Harness: Instance-Adaptive Harness Optimization
Sep 30, 2026Tunyu Zhang, Hao Wang, Kai Xu +1Agent Harness Optimization Self-Improving Agents
Rutgers University · Red Hat AI Innovation · MIT-IBM Watson AI Lab

Sep 12, 2026J/K move · Enter open · S save

HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning

Hongliang Wei, Xiaobing Tu, Yinggui Wang, Zhengxi Liu+5

Harbin Institute of Technology · Alibaba Cloud

RL for LLM Agents Agent Harness Optimization Multi-Armed Bandits

Open paper AskPDF arXiv

Language-model agents are increasingly deployed through diverse harnesses that differ in system prompts, tool schemas, control loops, and trajectory formats. The same model can perform unevenly across these interfaces, making robustness to harness variation an important objective. A natural approach is to train a shared policy through multiple harnesses, but doing so introduces a scheduling problem: each training step should favor a harness that currently provides a useful learning signal while also producing an update that benefits the other harnesses. We develop HarnessBandit, an online scheduler that selects one harness per optimizer step. After a group-relative policy optimization (GRPO) update, it observes learnability -- the mean absolute advantage on the batch -- and transferability -- the cosine between a low-dimensional gradient sketch of the current harness and exponential moving averages of the remaining harnesses. The two signals are fused after pooled sliding-window min-max normalization and sampled with a visit-dependent bonus and an explicit exploration floor. We train Qwen3.5-2B across six harnesses on ClawGym and evaluate on PinchBench (held-out tasks, in-distribution OpenClaw) and ClawEval (held-out tasks and harness). HarnessBandit improves over mixed-batch multi-harness training on both benchmarks, while training diagnostics indicate that learnability and transferability provide distinct, evolving signals.

Abstract

Agent Harness Optimization

Explore similar work

HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning

Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses

Turbo Harness: Instance-Adaptive Harness Optimization