Phronix: The Capability-Centric Learning Paradigm for Agents

Hariharan Devarajan; Neeraj Rajesh; Aniruddha Marathe; Nikoli Dryden; Kin Ng; Michela Taufer

cs.AIOct 22, 2026

Phronix: The Capability-Centric Learning Paradigm for Agents

Authors: Hariharan Devarajan, Neeraj Rajesh, Aniruddha Marathe, Nikoli Dryden, Kin Ng, Michela Taufer

Organizations: Lawrence Livermore National Laboratory · Illinois Institute of Technology · University of Tennessee, Knoxville

Abstract

Scientific computing increasingly relies on large language model-based agents for performance engineering tasks such as instrumentation, analysis, and workflow porting. However, agent effectiveness depends heavily on the surrounding harness, which typically relies on manually written skills and curated memory. This limits autonomous learning, forcing agents to repeatedly rediscover solutions rather than retain and reuse knowledge from prior executions. We introduce Phronix, a capability-centric learning paradigm that enables agentic harnesses to acquire, validate, and refine reusable capabilities from their own executions. Phronix operates through a predictor-validator-corrector loop: MCP tools and associated skills propose solutions, tool executions validate them, and observed outcomes refine the stored capabilities. It distills every observed interaction into a capability store and researches a failing task rather than abandoning it. First, such learned capabilities solve a recurring task on the first attempt in 93-95% of cases across every model, significantly reducing solution non-determinism and improving reliability. Second, Phronix turns an agent's own execution into 7.6-30.8 durable capabilities per task without user instruction, and creation falls below two once the needed capability exists. Third, warm-start runs consume only 0.38-0.40 million tokens per task, which is three to four times less than a single baseline attempt, while a cold-start costs no more than the baseline. On the 1000 Genome Pegasus workflow, instrumenting a production application for I/O performance analysis, Phronix passes all 11 correctness checks against six for the baseline harness, and also lifts the task solving rates of two cheaper models from 29% and 27% of SWE-bench tasks solved to 91% and 87%, respectively, matching the performance of the most expensive evaluated model.

Figures & tables

Panel of Figure 1 — Fig. 1: Phronix adds three components to an existing harness, one per concept, over a capability store whose three layers travel with the framework, the user, and the repository. The numbered arrows follow one agentic task.

Panel of Figure 2 — Fig. 2: The Capability Builder through the 1000 Genomes run, where the tracer install becomes a tool because its link flag never changes and the annotation choice stays a skill because it does.

Panel of Figure 3 — Fig. 3: The Tracker through the same run, where the failed build and the one that followed are reported automatically, grouped by kind of operation rather than by workflow.

Panel of Figure 4 — Fig. 4: The Researcher through the annotation step, where the lookup finds a gap, the plan separates deterministic edits from judgment, and a failing attempt still reaches a later step.

			gpt-5.4			gpt-5.5			gpt-5.6-sol
Category	Repositories	Tasks	Base	Cold	Warm	Base	Cold	Warm	Base	Cold	Warm
Web & HTTP	django, flask, requests	9	13/45	13/18	20/22	18/63	21/27	32/36	51/72	24/27	40/45
Dev tooling	pytest, pylint, sphinx	6	7/30	9/12	15/15	11/42	12/18	23/24	36/48	18/18	30/30
Scientific	astropy, sympy	5	10/25	4/10	10/12	10/35	10/15	15/20	17/40	11/15	19/25
Data & ML	xarray, scikit-learn	5	3/25	6/10	11/14	7/35	9/15	14/20	18/40	11/15	20/25
Visualization	matplotlib, seaborn	5	10/25	8/10	14/14	11/35	15/15	20/20	33/40	15/15	25/25

TABLE I: Tasks solved over tasks attempted, pooled across runs, for the 30 SWE-bench Lite tasks in five categories.

Fig. 5: Every model pairs MCP tools with skills in every category, and a warm-start run solves 95% of previously solved tasks in one attempt.

Fig. 6: A cold-start run distills 7.6–30.8 capabilities per task unasked (bars, left), while a warm-start run keeps tracking (line, right) but creates under two.

Fig. 7: Research prices attempts by progress, a solved task costing 1.05–1.91 attempts against the baseline’s five and a stagnating one stopping near three. Whiskers show the standard deviation.

Fig. 8: A warm-start run consumes 0.38–0.40M tokens per task, three to four times less than one baseline attempt, and a cold-start run costs no more than the baseline while solving far more tasks. Whiskers show the standard deviation.

Fig. 9: Phronix instruments the 1000 Genomes workflow through capabilities it built, while every baseline call is shell the model composed. Red marks the six behind unsound decisions.

Stage	Decision	Baseline	Phronix
setup	Python interpreter	downgraded to 3.9.12	kept the session interpreter
	Tracer build toolchain	swapped the toolchain	added the missing link flag
	Tracer release version	2.1.1.post36.dev0	2.1.1.post36.dev0
annotate	Annotation placement	restructured the app	annotated in place
	Annotation checking	none	linted and validated
	Write-path coverage	2504 write events	2504 write events

TABLE II: Decisions behind the finished workflow, green where sound and red where not. Only 5 of the baseline’s 11 are sound.

LLM Agent Skill Learning

Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.

See all 398 papers

Sep 28, 2026

Harness Learning Enables Generalizable Test-Time Adaptation

A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer with reinforcement learning, using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, without performing any parameter-space update. Experiments on reasoning and multi-hop question answering show that harness learning improves revision quality and that the ability to adapt at test time transfers to unseen tasks. Policies trained on individual revisions can continue improving harnesses over multiple rounds, while the benefits of training on revision sequences vary across settings. These findings suggest a path towards continually learning agents that turn accumulated experience into generalizable improvements.

Alvin Zhang, Xuecheng Liu, Zixuan Wang +6

Test-Time Adaptation Meta-Learning Agent Harness Optimization RL for LLM Agents

Ask PDF

Sep 22, 2026

Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents

Large language model (LLM) agents often handle streams of related tasks, yet standard harnesses repeatedly ask the model to reconstruct the same control decisions inside each task's context. We study whether task feedback can instead turn recurring control into reusable executable code, while reserving LLM calls for task-specific semantic reasoning. We introduce Growing Harness, a failure-guided training paradigm that learns the agent harness itself from a strategy-free scaffold that exposes fixed model and tool interfaces but encodes no task-solving controller. Function-level execution traces localize each failure to a bounded code surface, an optimizer repairs a window of failures jointly, and a success-first held-out gate rolls back repair sequences that harm prior capability. Accepted edits accumulate in one shared harness, allowing its control structure to emerge from task feedback. Across BrowseComp-Plus and WebArena-Verified with three deployment models from 4B to 120B parameters, Growing Harness achieves the highest mean success in five of six benchmark-model settings and trails the best mean by 0.7 pp. in the sixth. Relative to a Tool-Calling agent, it reduces LLM calls by 76.0-91.8% and deployed-agent inference cost by 74.4-98.6%. On WebArena-Verified, its success remains 44.7-45.3% across model scales, whereas Tool-Calling falls to 6.7% with the 4B model. Ablations show that trace-local edits, joint repair, and gate-based rollback each improve final success. These results show that persistent program growth can move recurring control out of model context and into low-cost code, yielding reusable specialist agents that remain effective with smaller deployment models.

Laizhen Li, Jiarui Li, Juanjuan Zhao +4

Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Shenzhen University of Advanced Technology +1

Agent Harness Optimization LLM Agent Workflow Optimization LLM Agent Self-Improvement LLM Agent Training

Ask PDF

May 27, 2026

Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows

LLM agents are increasingly deployed as executable systems that use tools, modify workspaces, and produce concrete artifacts. In such workflows, performance depends not only on the base model, but also on the harness: the system layer that manages context, tools, state, constraints, permissions, tracing, and recovery. However, existing benchmarks typically abstract away execution, compare complete agent systems, or hold the harness fixed, making execution-layer variation difficult to study. We introduce Harness-Bench, a diagnostic benchmark for evaluating configuration-level harness effects in realistic agent workflows. Harness-Bench evaluates representative harness configurations across multiple model backends under shared task environments, budgets, and evaluation protocols, while preserving each harness's native execution behavior. The benchmark contains 106 sandboxed offline tasks constructed from practical agent-use patterns and manually reviewed for realism, solvability, oracle-checkability, and integrity. Each run records final artifacts, execution traces, usage statistics, and validator outputs, enabling analysis beyond final completion. Across 5,194 execution trajectories, we observe substantial variation in completion, process quality, efficiency, and failure behavior across model-harness pairings. These results suggest that agent capability should be reported at the model-harness configuration level rather than attributed to the base model alone. Our analysis further identifies recurring execution-alignment failures, where plausible reasoning becomes decoupled from tool feedback, workspace state, evidence, or verifiable output contracts. Harness-Bench provides a reproducible foundation for diagnosing and improving reliable, efficient, and auditable agent execution stacks.

Yilun Yao, Xinyu Tan, Chao-Hsuan Liu +10

Peking University · Qiyuan Tech

Computer-Use Agent Benchmarks AI Agent Reliability LLM Agent Evaluation AI Agent Benchmarks LLM Agent Harnesses LLM Agent Reliability

Ask PDF

Abstract

LLM Agent Skill Learning

Explore similar work

Harness Learning Enables Generalizable Test-Time Adaptation

Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents

Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows