Task-Aware Harness Provisioning for LLM Agents in Mission-Critical Infrastructure Operations

Liangtao Lin; Qingang Zhang; Zhaomeng Zhu; Tianwei Zhang; Yonggang Wen

cs.AIAug 18, 2026

Task-Aware Harness Provisioning for LLM Agents in Mission-Critical Infrastructure Operations

Authors: Liangtao Lin, Qingang Zhang, Zhaomeng Zhu, Tianwei Zhang, Yonggang Wen

Organizations: Nanyang Technological University Singapore

Abstract

LLM agents have been widely adopted to operate mission-critical infrastructure (MCI). These agents normally rely on a harness that determines what information they can access, which tools they can use, and what actions they can take. Existing systems often expose the same comprehensive harness to every task, which may not be necessary and cause resource wastes. In this paper, we focus on the identification of optimal harness configurations, and view it as a resource-matching problem between what each task requires and what the harness provides. To measure this match, we classify MCI tasks based on the mathematical representation of the underlying system and rank harness configurations by the amount and type of information they provide. We then construct task-to-harness mappings from two sources: mining research literature and measuring controlled agent execution. Leveraging the measured mapping, we propose a new harness provisioning algorithm: map-guided escalation. It begins with a task-specific harness and expands to full provision only after a failed self-check. We evaluate our method in two representative MCI tasks: in liquid cooling, it improves the agent accuracy from 0.652 under full provision to 0.715 and achieves accuracy comparable to Reflexion with 48% fewer tokens; In power grids, full provision remains accuracy-optimal, while map-based provisioning offers lower-cost alternatives. These findings show that harness provisioning follows a domain-dependent accuracy-cost Pareto frontier rather than a universal optimum.

Figures & tables

Figure 1. Overview of the Proposed Framework, which defines the task and harness spaces, estimates literature- and execution-derived task-to-harness maps, and uses the resulting map for harness selection with optional self-check-triggered escalation.

Figure 2. Literature-derived Harness Distributions across the 12 Task Classes. Based on 1,220 MCI O&M papers, each panel shows the number of papers assigned to harness levels

K_{1}

–

K_{5}

for one task class; the outlined bar marks the modal level while the triangle and background marks the mean. The distributions generally shift toward higher harness levels for Act, Future, and latent-state tasks, while most classes peak at either

K_{3}

K_{5}

Figure 3. Task-to-Harness maps. Cells report the mean construction-split accuracy for each task class and harness level (5 tasks per class, 3 runs). Amber boxes mark the lowest level within $\epsilon_{\mathrm{exec}}=0.05$ of the best execution score, while the black staircase marks the literature-derived level. Columns are ordered by the literature mean harness level.

			Liquid			Grid
Group	Policy	Basis	Acc. $\pm$ std	Lat.	Tok.	Acc. $\pm$ std	Lat.	Tok.
Fixed	Full-K5	maximal provision	$0.652\pm 0.001$	$1.00\times$	$1.00\times$	$\mathbf{0.806\pm 0.009}$	$\mathbf{1.00\times}$	$\underline{1.00\times}$
Routing	ITR	semantic relevance	$\underline{0.670\pm 0.015}$	$0.90\times$	$0.99\times$	$0.780\pm 0.010$	$1.20\times$	$1.02\times$
	LLM-Route	LLM judgment	$0.626\pm 0.021$	$\mathbf{0.74\times}$	$0.93\times$	$0.744\pm 0.024$	$1.29\times$	$1.03\times$
	LLM+Exp	judgment with experience	$0.655\pm 0.010$	$\underline{0.75\times}$	$0.96\times$	$0.766\pm 0.011$	$1.26\times$	$1.09\times$
Cascades	AutoMix	trained check from K3	$0.639\pm 0.011$	$1.08\times$	$1.94\times$	$\underline{0.785\pm 0.029}$	$3.03\times$	$2.61\times$

Table 1. Harness Provisioning on the Test Split. All policies use the same frozen executor and differ only in how the harness is selected. Latency and tokens are normalized to Full-K5. Bold and underline denote the best and second-best deployable results in each column.

Figure 4. Accuracy–Cost Trade-offs of Harness Provisioning Policies on the Test Split. Latency and tokens are normalized to Full-

K_{5}

; dashed lines show the empirical Pareto frontiers.

Method	Adaptation lever	Acc. $\pm$ std	Tok.
Map-ESC	harness provisioning	$\underline{0.715\pm 0.014}$	$\mathbf{1.00\times}$
Reflexion	retry and reflection	$0.711\pm 0.027$	$1.91\times$
ExpeL	trajectory retrieval	$\mathbf{0.731\pm 0.009}$	$1.44\times$

Table 2. Provisioning Versus Execution-side Adaptation on the Liquid Test Split. Tokens are normalized to Map-ESC.

Policy	Map source	Liquid	Grid
Full- $K_{5}$	none	$0.706$	$\mathbf{0.808}$
Exec-Lookup	GPT-5.4	$\mathbf{0.726}$	$\underline{0.797}$
Exec-Lookup	Qwen	$0.626$	$0.762$

Table 3. Cross-executor Transfer of the Execution-derived Map. All conditions use Qwen3.5-27B as the executor.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5. The literature corpus. Left: sector distribution. Right: data-driven task and method vocabulary; word size is corpus frequency.

	3:0	2:1	adjudicated	pairwise agreement
Task coordinate	46%	54%	30 papers	$\kappa$ 0.49–0.84 by axis
Harness level	68%	29%	36 papers	exact 73–84%; QWK 0.61–0.78

Appendix

Table 4. Consensus composition and inter-annotator agreement.

Figure 7. Snapshot of the interactive literature explorer to be released with the project. Each point represents one paper, grouped by task class and consensus harness level. The inspection panel presents the selected paper’s metadata, supporting evidence, individual annotations, and consolidated labels.

Figure 8. Three-dimensional view of the literature-derived harness landscape. The vertices represent the 12 task classes defined by output mode, target time, and target element, and surface colour indicates the mean literature-derived harness level $\bar{K}$ . Conventional task names are shown only as approximate interpretations of the formal coordinates.

Task class $\langle\omega,\tau,e\rangle$	$n$	$K_{1}$	$K_{2}$	$K_{3}$	$K_{4}$	$K_{5}$	$\bar{K}$	Mode
I-N- $y$	254	1	3	208	13	29	3.26	$K_{3}$
I-N- $m$	242	2	27	132	22	59	3.45	$K_{3}$
I-F- $y$	178	0	2	127	14	35	3.46	$K_{3}$
A-N- $m$	118	0	9	65	21	23	3.49	$K_{3}$
I-N- $x$	118	0	1	44	36	37	3.92	$K_{3}$
A-N- $y$	77	0	0	28	22	27	3.99	$K_{3}$

Appendix

Table 5. Literature-derived harness distributions across the 12 task classes, based on the consensus labels of three annotators. Counts are reported by harness level, and task classes are ordered by mean

K

Class	Task	Question (abridged)	Grading
I-N- $y$	I-N-S-01	What is the outlet water temperature at cold plate rack_a4_s1 right now?	numeric, $\pm$ 0.6 K
I-N- $x$	I-N-X-01	How hot is the (unsensored) metal base plate of rack_a4_s1 right now?	numeric, $\pm$ 1.5 K
I-N- $m$	I-N-M-02	Outlets run high on both sides of the hall; which single device is operating far outside its typical range?	set match
I-F- $y$	I-F-S-01	The return-header temperature is drifting; using data up to now, where will it be at $t{=}650$ s?	numeric, $\pm$ 0.4 K
I-F- $x$	I-F-X-02	rack_a3_s2 steps from 503 W to 1400 W at $t{\approx}305$ s; what will its base-plate temperature reach?	numeric, $\pm$ 1.5 K
I-F- $m$	I-F-M-03	If the pump were raised to 4500 rpm at today’s load, where would the hottest outlet settle?	numeric, $\pm$ 0.8 K

Appendix

Table 6. Representative Liquid benchmark tasks, one per class. S/X/M denote signal(

y

)/state(

x

)/mechanism targets(

m

); graders are deterministic rule checks combined with an LLM judge (Appendix D ).

Level	Liquid	Grid
K1 model-only	no tools; parametric reasoning over the task prompt	no tools; parametric reasoning over the task prompt
K2 static	static_knowledge (catalog, parameter schemas, topology summary, typical ranges, units); web_search	static_knowledge ; search_knowledge and search_sop (RAG over references and operator playbooks); web_search
K3 temporal	list_signals , get_signal_window , compute_signal_stats , find_events over gated telemetry	same four tools over the 142-column grid trajectory
K4 structure/physics	query_topology , get_fluid_properties , check_physics (energy balance, pressure drop, mixing), check_constraints	query_topology , get_component_limits , check_physics (power balance, loading, per-unit V), check_constraints
K5 simulation	simulate_steady (hydraulic/coupled), what_if , compute_sensitivity , simulate_forward (transient rollout)	run_powerflow (AC, with overrides), what_if , contingency_analysis (N-1), compute_sensitivity

Appendix

Table 7. Resources exposed at each harness level. Tools are OpenAI function-calling interfaces; each level unions all lower levels.

$\epsilon$	$\Delta$ fl.	Acc $\pm$ std	Tok	$\Delta$ fl.	Acc $\pm$ std	Tok
	Liquid			Grid
0.00	2	$0.659\pm 0.021$	14,738	0	$0.762\pm 0.023$	11,829
0.02	2	$0.659\pm 0.021$	14,738	0	$0.762\pm 0.023$	11,829
0.05	0	$0.670\pm 0.020$	14,009	0	$0.762\pm 0.023$	11,829
0.10	3	$0.673\pm 0.022$	14,246	1	$0.749\pm 0.024$	12,253
0.15	4	$0.671\pm 0.024$	12,851	2	$0.742\pm 0.035$	13,406

Appendix

Table 8. Sensitivity of the execution-derived map to

\epsilon_{\text{exec}}

. “

\Delta

floors” counts classes whose selected level differs from

\epsilon{=}0.05

; accuracy/tokens are test-split lookup results.

	Liquid			Grid
Class	floor	$P$ (floor)	LOO	floor	$P$ (floor)	LOO
I-N- $y$	K3	0.70	K3	K4	0.91	K4
I-N- $x$	K3	0.99	K3	K5	0.99	K5
I-N- $m$	K3	0.99	K3	K3	0.99	K3
I-F- $y$	K5	0.35	K3/K4/K5	K3	0.75	K3/K4
I-F- $x$	K5	0.68	K4/K5	K5	0.73	K3/K5

Appendix

Table 9. Execution-derived level stability at

\epsilon{=}0.05

: bootstrap probability of re-selecting the adopted floor over construction tasks, and leave-one-out floor sets.

	Liquid			Grid
Level	Acc $\pm$ std	Tok	Lat (s)	Acc $\pm$ std	Tok	Lat (s)
K1	$0.253\pm 0.002$	346	3.5	$0.191\pm 0.016$	1,021	4.0
K2	$0.216\pm 0.011$	866	4.3	$0.197\pm 0.004$	2,012	4.5
K3	$0.581\pm 0.008$	11,617	13.7	$0.502\pm 0.020$	14,716	13.6
K4	$0.560\pm 0.013$	16,923	16.6	$0.566\pm 0.011$	18,179	12.8
K5	$0.680\pm 0.015$	17,794	50.2	$0.773\pm 0.019$	13,265	7.7

Appendix

Table 10. Full

K_{1}

–

K_{5}

sweep, all 120 tasks per domain, 3 runs.

	Liquid					Grid
Class	K1	K2	K3	K4	K5	K1	K2	K3	K4	K5
I-N-S	.00 $\pm$ .00	.00 $\pm$ .00	.90 $\pm$ .00	.88 $\pm$ .06	.87 $\pm$ .03	.03 $\pm$ .02	.03 $\pm$ .02	.78 $\pm$ .02	.90 $\pm$ .00	.88 $\pm$ .02
I-N-X	.02 $\pm$ .02	.04 $\pm$ .03	.59 $\pm$ .05	.53 $\pm$ .05	.46 $\pm$ .10	.00 $\pm$ .00	.00 $\pm$ .00	.23 $\pm$ .06	.63 $\pm$ .05	.90 $\pm$ .02
I-N-M	.36 $\pm$ .06	.29 $\pm$ .07	.90 $\pm$ .00	.87 $\pm$ .05	.85 $\pm$ .07	.23 $\pm$ .02	.26 $\pm$ .01	.93 $\pm$ .05	.93 $\pm$ .05	.93 $\pm$ .05
I-F-S	.15 $\pm$ .00	.15 $\pm$ .00	.70 $\pm$ .05	.65 $\pm$ .07	.71 $\pm$ .02	.10 $\pm$ .00	.12 $\pm$ .02	.42 $\pm$ .02	.45 $\pm$ .05	.44 $\pm$ .10
I-F-X	.20 $\pm$ .03	.14 $\pm$ .02	.30 $\pm$ .06	.33 $\pm$ .01	.36 $\pm$ .06	.15 $\pm$ .04	.10 $\pm$ .02	.26 $\pm$ .06	.32 $\pm$ .07	.36 $\pm$ .04

Appendix

Table 11. Per-class mean accuracy

\pm

std for every harness level (10 tasks/class, 3 runs).

Agent Harness Optimization

Agent harness optimization improves the prompts, tools, memory, workflows, and control code that shape how AI agents act.

See all 183 papers

Sep 30, 2026

Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives

Agent harnesses govern how large language models (LLMs) gather context, invoke tools, verify results, preserve state, and terminate, largely affecting agent performance. However, the value of each harness mechanism can differ across heterogeneous tasks: a mechanism that improves one task may impose overhead or context distraction on another, leading to the suboptimality of a global harness. We characterize this suboptimality as a mismatch induced by fixed mechanism choices, motivating task-specific harness construction. Nonetheless, generating harness code for each task introduces generation and debugging costs, with execution risks that can compound as more mechanisms are generated. To address those challenges, we introduce Harness Primitives, reusable harness mechanisms with clear application scope and composition contract mined from failed task trajectories. Based on Harness Primitives, we propose STITCH, a framework that Selects suitable primitives given Task Information and compiles them into Task-speCific Harnesses at test time. This separation enables task-specific harnesses without generating or repairing mechanism code at test time. Extensive experiments demonstrate that STITCH not only improves harness adaptability and robustness, but also scales with the primitive library size, boosting task success rates by up to 12 points over fixed harness baselines, surpassing human-designed harnesses like Codex CLI while maintaining a minimal test-time harness composition overhead of only 2.7%, 638 times more efficient than generating task-specific harnesses from scratch. Ultimately, our work demonstrates that building task-adaptive harnesses can be beneficial for completing diverse tasks and that building reusable primitives can be a promising path towards this goal.

Peng Kuang, Haibo Jin, Dehao Wu +4

University of Illinois Urbana-Champaign · University of Michigan, Ann Arbor

Agent Harness Optimization LLM Agent Harnesses

Ask PDF

Jun 14, 2026

From Question Answering to Task Completion: A Survey on Agent System and Harness Design

LLM-based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent-native training with co-evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM-based agents through a model-harness lens. We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model coupled with an execution harness. We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities: observation, context, control, action, state, and verification. Using this decomposition, we map task properties and domain pressures to harness configurations, review benchmark and evaluation practices, and synthesize model-harness evidence on how runtime design affects long-horizon task completion, efficiency, and reliability. Finally, we identify open challenges in value-aware evaluation, safety, harness generalization, and model-harness co-evolution. Rather than treating agents as models with auxiliary tools, this survey argues that agent quality -- including success, efficiency, safety, and generalization -- emerges from the interaction between model capability, runtime infrastructure, task structure, and evaluation design. A collection of papers discussed in this survey is provided in https://github.com/ggjy/Awesome-Agent-Engineering.

Jianyuan Guo, Zhiwei Hao, Chengcheng Wang +14

Department of Computer Science, City University of Hong Kong, HKSAR, China

LLM Agent Evaluation LLM Agents Long-Horizon LLM Agents LLM Agent Harnesses LLM Agent Reliability

Ask PDF

May 15, 2026

Harnesses for Inference-Time Alignment over Execution Trajectories

Harness engineering has emerged as an important inference-time technique for large language model (LLM) agents, aiming to improve long-term performance through task decomposition and guided execution. However, more elaborate harnesses are not uniformly better: increasing decomposition or guidance can sometimes improve execution, but can also reduce final task success. We study harness design through the lens of inference-time trajectory alignment. This perspective separates harness into two mechanisms: task decomposition, which structures a task into sub-goals, and guided execution, which reshapes local action distributions during execution. This decomposition allows us to quantify how workflow granularity, retry budgets, and guidance-induced action reweighting shape the performance limits of harness design. It further reveals concrete failure modes, including over-decomposition, over-pruning, and hallucinated execution. We validate these predictions through controlled synthetic experiments and real terminal agent benchmarks. Inspired by the theory, we further show that effective harnesses can be partial: specifying only the initial steps and leaving the remaining execution to agent can achieve higher pass rate than fully structured workflows.

Boyuan Wang, Bochao Li, Minghan Wang +2

Southern University of Science and Technology

LLM Planning Agent Harness Optimization LLM Agent Workflow Optimization Task Decomposition Long-Horizon LLM Agents LLM Agent Harnesses+1

Ask PDF

Abstract

Agent Harness Optimization

Explore similar work

Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives

From Question Answering to Task Completion: A Survey on Agent System and Harness Design

Harnesses for Inference-Time Alignment over Execution Trajectories