Harness Annealing: Learning to Act with Less External Control

Yingxuan Yang; Huacan Chai; Ying Wen

cs.CLOct 1, 2026

Harness Annealing: Learning to Act with Less External Control

Authors: Yingxuan Yang, Huacan Chai, Ying Wen

Organizations: Shanghai Jiao Tong University · University of California, Berkeley

Abstract

Language agents rely on external harnesses to track state, organize workflows, and verify answers. Beyond providing tools and information, these harnesses supply control decisions about what to investigate, whether to revise, and when to stop. Training on successful harness-supported trajectories can improve task performance while leaving these decisions dependent on runtime intervention. We ask whether harness-supported experience can also teach the model to make these decisions, allowing the division of control to change as the model learns. We call this objective harness internalization: learning to assume specified control responsibilities while retaining task performance after the corresponding support is withdrawn. We introduce HARNESS ANNEALING TRAINING (HAT), which combines explicit control supervision with a curriculum over teacher trajectories collected under progressively weaker harnesses. Experiments with 9B and 35B models on SWE-QA and SWE-QA-Pro evaluate every checkpoint under four deployment harnesses. Selected annealed checkpoints operating with tools alone achieve scores close to those of their respective starting checkpoints deployed with the full harness. The benefits vary with model scale and deployment configuration, and further annealing does not uniformly improve performance. These findings suggest that harness-supported experience can help reduce the runtime control required by a trained agent.

Figures & tables

Figure 1: Learning to retain task performance with less external control. (a) Control ownership. Following verifier feedback does not ensure independent judgment after its removal; state, workflow, and tools remain. (b) Harness annealing. HAT combines explicit control supervision with a curriculum over progressively weaker source harnesses. Paths schematically contrast potential control gaps with intended behavior under the same reduced support.

Nested configuration	Added runtime support	Targeted responsibility
$H_{1}=\text{{Tools}}$	Tool access and execution safeguards	Gather task evidence using external tools.
$H_{2}=H_{1}+\text{{State}}$	Recent history, progress ledger, and repetition detection	Track evidence and gaps; avoid unproductive repetition.
$H_{3}=H_{2}+\text{{Workflow}}$	Locate $\to$ Read $\to$ Connect $\to$ Answer	Organize investigation and answer construction; choose phase transitions.
$H_{4}=H_{3}+\text{{Verification}}$	Heuristic draft checks and bounded repair	Check draft support; revise, gather evidence, or finish.

Table 1: Deployment hierarchy and targeted control responsibilities. Each level retains all preceding support; bold marks the added component. Tools, execution guards, and token budgets remain external throughout. The verifier has no access to the reference answer.

Figure 2: Training and evaluation of HAT. Initial SFT produces the Layered

H_{4}

-trained checkpoint. Three annealing stages then introduce records derived from

H_{3}

H_{2}

, and

H_{1}

rollouts, supplemented by stronger-source replay and cue-dropout variants. The rollout labels identify source harnesses; targeted checkpoint labels identify cumulative training through the corresponding source level. Every checkpoint is evaluated under all four deployment harnesses. Red cells test removal of the control layer targeted at that stage: verification at

H_{3}

, workflow at

H_{2}

, and state at

H_{1}

. Related cue-removal patterns may occur in training through dropout.

Model	Benchmark	Checkpoint	Deployment Harness				Row summary
Model	Benchmark	Checkpoint	$H_{1}$	$H_{2}$	$H_{3}$	$H_{4}$	Mean	Worst	$\Delta$
Qwen3.5-9B	SWE-QA	Base	59.93	62.92	67.28	66.79	64.23	59.93	7.35
		$H_{4}$ -trained	71.99	75.94	77.47	75.70	75.28	71.99	5.48
		$H_{3}$ -targeted	77.41	77.15	75.10	75.66	76.33	75.10	2.31
		$H_{2}$ -targeted	77.00	76.62	76.43	76.45	76.63	76.43	0.57
		$H_{1}$ -targeted	76.30	75.12	74.43	74.16	75.00	74.16	2.14

Table 3: Every training checkpoint is evaluated under every deployment configuration. Scores are mean reference-based judge scores; higher is better. Red cells test the control responsibility targeted at each stage after its external support is removed:

H_{4}

-trained under

H_{3}

H_{3}

-targeted under

H_{2}

, and

H_{2}

-targeted under

H_{1}

. Related cue-removal patterns may occur in training through dropout.

H_{1}

exposes the bare tool interface and each subsequent harness adds one layer of control, so that

H_{1}\subset H_{2}\subset H_{3}\subset H_{4}

. Within each model–benchmark block, bold marks the best in each column. The starting row is Layered

H_{4}

-trained; each targeted checkpoint continues from the row above.

Explore similar work

HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness
Jun 11, 2026Xiaoxuan Wang, Haixin Wang, Alexander Taylor +3Agent Harness Large Language Model Agents
University of California, Los Angeles
Harness-Zero: Harness Distillation via Agent-as-Harness
Sep 21, 2026Haoran Ye, Yuxing Lu, Haonan Dong +2Agent Harness Symbolic Distillation
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · College of Future Technology, Peking University · Google +1
Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
Jul 5, 2026Haiwen Yi, Xinyuan SongLarge Language Model Agents Agent Harness
University of Toronto · Emory University

Jun 11, 2026, cs.AIJ/K move · Enter open · S save

HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness

Xiaoxuan Wang, Haixin Wang, Alexander Taylor, Jason Cong+2

University of California, Los Angeles

Agent Harness Large Language Model Agents Learning-Based Control Action Expert

Open paper AskPDF arXiv

Large language models are increasingly deployed as agents for long-horizon tasks, yet their performance is shaped not only by model capability and environment design, but also by the harness that mediates agent--environment interaction. Existing harnesses are largely manually engineered, making them difficult to scale as trajectories grow longer and interactions become more complex. In this work, we ask whether harness can be generated by a learnable plug-in module that can be trained in an end-to-end fashion. We introduce HarnessBridge, a lightweight learnable harness controller that parameterizes the agent--environment interface as a bidirectional projection. HarnessBridge learns two bidirectional projections: observation projection, which distills raw trajectories into compact, decision-relevant states, and action projection, which converts proposed actions into executable transitions or trajectory-grounded rejections. We train HarnessBridge on a harness supervision dataset via unified instruction tuning. On Terminal-Bench~2.0 and SWE-bench Verified, HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length, and generalizes from smaller generators to larger commercial models.

Abstract

Explore similar work

HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness

Harness-Zero: Harness Distillation via Agent-as-Harness

Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning