Latent Harness: Compiling Agent Behaviors into Post-Block Activation Shifts
Authors: Ran Li, Shimin Di, Zeyu Ren, Ling Yue, Jiachuan Wang, Lei Chen
Organizations: Hong Kong University of Science and Technology · Southeast University · Rensselaer Polytechnic Institute · University of Tsukuba · Hong Kong University of Science and Technology(Guangzhou)
Harness engineering and self-evolving agents improve language-model agents quickly, yet their gains live outside the model. The harness is re-sent and re-paid on every call, and the standard route to internalization is a weight update that is costly to iterate and fixed once merged. We propose compiling the harness instead. Latent Harness distills an external condition into a rank-8 activation shift. The shift is applied at the output of every decoder block of a frozen backbone, at 0.03-0.05% of parameters. The condition is either an explicit process-level behavior instruction or the behavioral contract of an agentic harness. At inference the compiled bundle carries no harness text and adds no tokens. It exposes three runtime controls for injection strength, injection phase, and shift-space mixing. Compiled behaviors recover the source-harness gain in 8 of 10 preregistered settings. Where the explicit source pays the heaviest per-call token tax, the compiled form exceeds it. The same bundles transfer zero-shot across tasks (5 of 7 pairs positive) and compose in shift space, while stacked text instructions interfere destructively. Used as a search space, they lift budget-matched harness search by 1.45 times. On three agentic benchmarks the operator aligns policy to the harness con- tract. Under matched data, capacity, and effective strength it outperforms same-capacity LoRA, which needs 11.1 times the parameters to reach the same band, and it regresses nothing where the base policy is already strong. Across 1B-to-8B models the method skeleton is invariant while gain locations move with the model-task pair, which delimits where compiled harnesses should be deployed.
Figures & tables
Figure 1: Latent Harness compiles external conditions into post-block activation shifts. Teacher traces or strictly filtered success trajectories train a rank-8 shift at every block output of a frozen backbone. The bundle injects at run time with strength, phase, and mixing controls.
ToolHop
SearchQA
API-Bank
Model
Method
Params (%)
Ans.
Path
Hotpot
2Wiki
Overall
Succ.
API.
Qwen3-4B
ADAS
0
40.00
44.99
28.67
28.00
28.33
57.31
60.99
AgentSquare
0
29.23
39.46
29.33
30.00
29.67
47.82
46.10
AFlow
0
31.28
43.81
30.67
29.33
30.00
45.72
41.13
MaAS
0
42.05
51.80
30.67
30.00
30.33
62.74
63.83
MermaidFlow
0
44.62
58.74
31.33
28.00
29.67
64.33
60.28
Table 1: Main results on the HarnessForge benchmark suite, following its baseline suite. Search-style rows search over agent harnesses, training-style rows update policy weights, and HarnessForge is the full co-evolution framework. Ours is evaluated under our controlled single-round alignment protocol, which uses the official round-3 harness pairing and evaluation pools disjoint from training, and which is protocol-distinct from the co-evolution rows above. Params (%) is the trainable fraction of backbone parameters (LoRA for training-style rows and HarnessForge, while search-style rows update no weights). Best in bold , second best underlined , per backbone.
Figure 2: Policy compilation, efficiency, stability, and safety. (a) Capacity alignment on ToolHop. (b) Run-level repeatability. (c) Zero regression on API-Bank at 4B and 8B.
Setting
EXPL.
SOFT-PROMPT
TASK-ADAPTER
OURS
ΔL ( R )
M1 ⋅ ARC × COMPUTE
0.300
0.347
0.533
0.322
+0.082 (1.23)
M1 ⋅ ARC × CONCISE
0.380
0.376
0.533
0.362
+0.122 (0.83)
M1 ⋅ MBPP × TESTCONSULT
0.066
0.079
0.308
0.323
+0.317 (5.86)
M2 ⋅ MBPP × DECOMPOSE
0.410
0.508
0.562
0.661
+0.619 (1.66)
M2 ⋅ MBPP × COMPUTE
0.470
0.521
0.554
0.660
+0.618 (1.43)
M2 ⋅ MBPP × CONCISE
0.396
0.531
0.570
0.687
+0.645 (1.80)
Table 2: Controlled test-set comparison, one touch and preregistered. SOFT-PROMPT is parameter-matched and trained on the same distillation signal. TASK-ADAPTER is trained on gold labels at matched capacity. ΔL is ours over base, and R=ΔL/ΔE uses the explicit gains in Fig. 7 . All ΔL CIs exclude zero, and 8 of 10 enrolled settings meet the criterion R≥0.5 .
Figure 3: Instruction compilation. (a) Behavior recovery on the test set. (b) Token cost against accuracy.
Figure 4: Three-chain attribution. (a) Accuracy against behavior markers (M2 × MBPP average). (b) Score recovery against marker recovery. (c) Harmful signatures compile symmetrically and transfer out of distribution.
Figure 5: Reuse of compiled behaviors. (a) Cross-task transfer of frozen bundles. (b) Text stacking against shift-space composition. (c) Search gain at matched evaluation budgets.
Figure 6: Mechanism. (a) Phase decomposition of the gain. (b) Strength response with plateau and cliff. (c) ToolHop with the harness removed, by configuration.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Code
Model
Layers ×d
Trainable
Share
M1
Llama-3.2-1B-Instruct
16 × 2048
524,288
0.0424%
M2
Qwen3-4B
36 × 2560
1,474,560
0.0367%
M3
Llama-3.1-8B-Instruct
32 × 4096
2,097,152
0.0312%
v2
Llama-3.2-1B (base)
16 × 2048
524,288
0.0424%
–
Qwen3-8B (policy)
36 × 4096
2,359,296
0.0288%
Instruction tasks are BoolQ, ARC-Challenge, GSM8K, and MBPP.
Appendix
Table 3: Models and coverage. Rank-8 post-block adapters cover all layers and all tokens, with a SiLU bottleneck for instruction compilation and a linear map for policy compilation. The backbone is frozen, with every parameter verified identical to the original.
Figure 7: Screening forest (left) and the MBPP format-floor unlock (right).
Figure 8: MBPP marker displacements under explicit harnesses (left) and absolute marker transitions from base to the compiled form (right).
Figure 9: (a) Preregistered marker recoveries across settings. (b) Independent replication on the base 1B model.
Figure 10: (a) Harmful-signature transfer to SVAMP. (b) Transfer contrast between compiled behaviors and gold-label task adapters.
Figure 11: (a) The 25-point shift-space composition grid. (b) The three budget-matched search configurations.
Figure 12: (a) All ToolHop configurations, with runs as points. (b) Action counts per configuration. (c) Reproducibility self-check.
Figure 13: (a) Failure staging at high strength. (b) The same failure family at 4B and 8B.
Table 4: Cross-scale account for Qwen3-4B against Qwen3-8B. The method skeleton is invariant. Gain locations (task × phase × scale) move with the model × task pair.
Figure 14: (a) Gain locations across benchmarks and scales. (b) 8B ToolHop across strengths.
Figure 15: (a) 8B SearchQA by subset. (b) 8B API-Bank by configuration.
The performance of large language model (LLM) systems depends not only on model weights, but also on their harness: the code that determines what information to store, retrieve, and present to the model. Yet harnesses are still designed largely by hand, and existing text optimizers are poorly matched to this setting because they compress feedback too aggressively. We introduce Meta-Harness, an outer-loop system that searches over harness code for LLM applications. It uses an agentic proposer that accesses the source code, scores, and execution traces of all prior candidates through a filesystem. On online text classification, Meta-Harness improves over a state-of-the-art context management system by 7.7 points while using 4x fewer context tokens. On retrieval-augmented math reasoning, a single discovered harness improves accuracy on 200 IMO-level problems by 4.7 points on average across five held-out models. On agentic coding, discovered harnesses surpass the best hand-engineered baselines on TerminalBench-2. Together, these results show that richer access to prior experience can enable automated harness engineering.
An agent harness is the code that organizes context, maintains state, and coordinates tool calls for a language model. We study how to improve the harness under a limited evaluation budget while keeping model weights fixed. Our method, MESH-Harness, organizes each harness into functional modules with explicit role-specific interfaces, allowing alternative implementations of each module to be substituted and recombined. It uses shared module representations and full-covariance LinUCB to score candidate combinations based on predicted performance and exploration value. Mixed-start coordinate ascent selects complete configurations for evaluation without enumerating the combinatorial space. Validation traces then guide local code edits, and the resulting candidates are incorporated into fixed-capacity role-specific pools for subsequent recombination. On text tasks, retrieval-augmented mathematical reasoning, code generation, and interactive scientific tasks, MESH-Harness outperforms Meta-Harness by 5.70, 7.01, 2.00, and 5.00 points, respectively, under matched candidate-evaluation budgets. Iterative harness optimization improves MESH-Harness by 5.63-7.79 points over its first-round configurations. For the reported configurations, aggregate test-time cost is 44.2% lower than that of Meta-Harness, while total cost including search is 14.6% lower. These results show that combining module-level design reuse with feedback-driven compositional search can systematically improve agent harnesses while keeping overall optimization cost under control.
Zhiwei Shang, Yu Huo, Mingrong Gong +6
The Chinese University of Hong Kong, Shenzhen · DeepWisdom · The Chinese University of Hong Kong +3
As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly termed the agent harness. Changing this harness while holding model weights fixed can substantially alter task performance. Current agent evaluations typically report downstream performance under a chosen harness, leaving a model's ability to develop the harness itself comparatively underexplored. We introduce HarnessDev, a benchmark that shifts the unit of evaluation from task outputs to runnable infrastructure. HarnessDev covers two stages. In Creation, the agent starts from a minimal seed and a small number of cases, then builds a complete execution system. In Evolution, it starts from its own created harness and iteratively revises it using downstream execution feedback, with the goal of improving benchmark performance. We then evaluate each constructed harness on capability (task success on held-out benchmarks) and efficiency (execution-token cost). The reported Creation results cover six creator LLMs, four domains, and five downstream benchmarks totaling 2,207 unique downstream instances, with hidden evaluation tasks withheld from development. We find that generated harnesses remain substantially behind mature human-engineered references on code and on search and research, while matching or exceeding the selected references on writing and machine-learning experimentation, with large variation in execution cost. Evolution produces some performance gains, but they are unstable and transfer only partially to held-out tasks. Experiments with a fixed runtime model further show that the gains depend strongly on the model executing the harness, indicating limited transfer across models.