LLM Agent Harnesses

LLM: Large Language Model

An LLM agent harness surrounds a language model with the mechanisms needed to act: tool interfaces, control loops, context and memory management, persistent state, and execution environments. Common approaches include ReAct-style loops, explicit workflow graphs, typed tool contracts, sandboxes, and verification or recovery steps; newer systems also adapt harnesses using execution traces, search, or reinforcement learning. Researchers evaluate these systems on coding, web, terminal, scientific, and interactive tasks, measuring not only success but also cost, reliability, and safety. A central tension is how much structure helps: added guidance can improve control and recovery, yet may constrain useful behavior, fail to transfer across models or tasks, or make comparisons hard to interpret without specifying the harness.

Latest papers

389 papers118 papers in the last four weeks, up 103% on the four weeks before. 0.7% of all new papers.

Practical Assessment of Agentic Systems in Acoustics
Oct 29, 2026Arne Hildenbrand, Philipp Diel, Zhuo Cao +3LLM Agent Harnesses LLM Agent Evaluation
Chair of Vibroacoustics of Vehicles and Machines, Department of Engineering Physics and Computation, TUM School of Engineering and Design, Technical University of Munich, Boltzmannstrasse 15, 85748 Garching, Germany · Chair of Thermodynamics of Mobile Energy Conversion Systems, Faculty of Mechanical Engineering, RWTH Aachen University, Forckenbeckstrasse 4, 52074 Aachen, Germany · IAS-8, Forschungszentrum Jülich, Wilhelm-Johnen-Straße, 52425 Jülich, Germany
VerifiedLab: Harnessing Knowns for Unknowns
Oct 27, 2026Yu Yao, Jiayi Dong, Yang Yang +2LLM Agent Harnesses AI Agents for Scientific Discovery
Massachusetts Institute of Technology, Cambridge, MA, USA. · University of California, Los Angeles, CA, USA. · Texas A&M University, College Station, TX, USA.
Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
Oct 27, 2026Jian Mu, Ruibo Shen, Yao ShuLLM Agent Harnesses Agent Harness Optimization
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
A Harness Reducing over-Defensive Writing in Agents by Isolating Objections
Oct 27, 2026Xiaochen ChangLLM Agent Harnesses LLM Agent Workflow Optimization
kw-bench: Breaking Frontier Harnesses on Real-World Tasks
Oct 26, 2026George Juraj SalapaAI Agent Benchmarks LLM Agent Harnesses
G.S. s.r.o. · PwC Austria
Context Is a Pipeline, Not a Window: Information Attrition in LLM Agent Harnesses
Oct 26, 2026Yuvraj Sehgal, Mahsa Panahandeh, Naser Ezzati-JivanLLM Context Management LLM Agent Harnesses
Department of Computer Science Brock University St. Catharines, Ontario, Canada · School of Electrical Engineering and Computer Science University of Ottawa Ottawa, Ontario, Canada
Work Agents: Systems, Capabilities, and Optimization
Oct 26, 2026Zishan Xu, Enjun Du, Mingqi Gao +13LLM Agent Reliability LLM Agent Workflow Optimization
Shanghai Jiao Tong University · The University of Hong Kong · The Hong Kong University of Science and Technology (Guangzhou) +5
Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy
Oct 26, 2026Yifei Yao, Yude Zou, Guanbo Jiao +12Language Model-Based Control Robotic Policy Evaluation
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Fudan University +1
Free to Look, Free to Reason: Simple LLM Harnesses Outdiagnose Hand-Crafted Root Cause Analysis Agents
Oct 26, 2026Tingzhu Bi, Xun Zhang, Jinglin Li +2Root Cause Analysis LLM Agent Harnesses
Peking University, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China
MarkovSim: an LLM Agent Harness with Bounded Context for Long-Horizon Computational Simulation
Oct 26, 2026Yeui-Lung Lei, Wen-Bo Chen, Pan ZhangLLM Agent Harnesses Long-Horizon LLM Agents
College of Civil and Transportation Engineering, Shenzhen University, Shenzhen, 518060, China · Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China.
RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization
Oct 26, 2026Jiawei Lu, Chen Zhang, Jinwei Lu +3LLM Agent Harnesses LLM Agent Self-Improvement
The Hong Kong Polytechnic University, Hong Kong, China · The Hong Kong University of Science and Technology, Hong Kong, China
How Do Coding-Agent Harnesses Self-Evolve? An Empirical Study of Retrospective Adaptation
Oct 26, 2026Christoph Treude, Sebastian BaltesLLM Agent Harnesses Self-Evolving Agents
Singapore Management University Singapore · Heidelberg University Heidelberg, Germany
Rethinking Verification in Agent Harnesses
Oct 25, 2026Zhengxin Zhang, Erkang Zhu, Yin Lin +3LLM Agent Verification LLM Agent Harnesses
Cornell University · Alibaba Group
Numina Fuse: Interactive Autoformalization in Lean 4
Oct 24, 2026Justin Asher, Yichuan Cao, Ruichen Qiu +2Autoformalization LLM Agent Harnesses
Project Numina · Academy of Mathematics and Systems Science, University of Chinese Academy of Sciences · University of Naples Federico II
Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI Testing
Oct 24, 2026Arsham Gholamzadeh Khoee, Ruyu Fu, Jiaqi Wei +4LLM Agent Harnesses Automated Software Testing
Department of Computer Science and Engineering, Chalmers University of Technology, Gothenburg, Sweden · Volvo Group, Gothenburg, Sweden
HarnessMM: Recursive Self-Improvement for Harness Token Efficiency
Oct 23, 2026Yuhua Jiang, Chang Xu, Feifei Gao +1Agent Harness Optimization LLM Agent Efficiency
Tsinghua University · Microsoft Research Asia (MSRA)
When Agents Fail: Model or Harness--Who's to Blame?
Oct 23, 2026Shao Zhang, Hangfan Zhang, Yuanyuan Shi +9LLM Agent Evaluation LLM Agent Harnesses
Shanghai Artificial Intelligence Laboratory
Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution
Oct 23, 2026Zhiyong Cao, Hengjie Zhu, Yongxin Wang +20LLM Agent Harnesses LLM Agent Reliability
Institute of Automation, CAS · Zhongguancun Academy · Institute of Information Engineering, CAS +2
RSIHub: A Composable Framework for Agent-Evolution Methods
Oct 23, 2026Zimu Wang, Xiaobo Wang, Siyu Ye +11Self-Evolving Agents LLM Agent Harnesses
University of California, Berkeley · Shanghai Jiao Tong University · University of Science and Technology of China +4
The LLENS Platform for On-Premises Large Language Model Agents in a Japanese University Hospital: Development and Evaluation Study
Oct 23, 2026Ken Enda, Jien Saito, Masami Mukai +12LLM Agents LLM Agent Harnesses
Division of AI Support for Medical Research, Faculty of Medicine, Hokkaido University, Sapporo, Japan · Department of Cancer Pathology, Faculty of Medicine, Hokkaido University, Sapporo, Japan · Technoplasm Inc., Sapporo, Japan +5
Do AI-for-Science Agents Preserve Model Safety?
Oct 23, 2026Michael Yew Chuan Lee, Manya Sriram, N. Benjamin ErichsonLLM Agent Safety LLM Safety Evaluation
DSO National Laboratories · UC Berkeley · ICSI and Berkeley Lab
Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents
Oct 22, 2026Prabal Vashisht, Zheng Hu, Ying YangPrompt Compression LLM Agent Evaluation
Mx ML, DoorDash Inc.
Phronix: The Capability-Centric Learning Paradigm for Agents
Oct 22, 2026Hariharan Devarajan, Neeraj Rajesh, Aniruddha Marathe +3LLM Agent Skill Learning LLM Agent Harnesses
Lawrence Livermore National Laboratory · Illinois Institute of Technology · University of Tennessee, Knoxville
Can LLMs Improve Their Own Harness? A Controlled Study on ARC-AGI-3
Oct 21, 2026Jens Lehmann, Sahar VahdatiLLM Agent Harnesses Agent Harness Optimization
Amazon∗ · Technische Universität Dresden · Universität Hannover +1
Agent Behavior Maps: Locating Agents by What They Would Do, Without Running Them
Oct 21, 2026Shota Fujikawa, Takuya Tamura, Kosuke Akimoto +2LLM Agent Evaluation LLM Agent Harnesses
The University of Tokyo · NEC Corporation
Harness Scaling for Generalizable Agentic Reinforcement Learning
Oct 21, 2026Fei Bai, Huatong Song, Chuan Hao +5Agentic RL LLM Agent Harnesses
Gaoling School of Artificial Intelligence, Renmin University of China · IQuest Research · Beihang University
Object-Oriented Harness: Evolving Agent Harnesses through Object-Oriented Design
Oct 20, 2026Gaotang Li, Bhavana Dalvi Mishra, Yifei Ming +9Agent Harness Optimization LLM Agent Harnesses
University of Illinois Urbana-Champaign · Google Cloud AI Research
ChartEvo: Self-Evolving Agent Harnesses for Long-Tail Chart Reasoning
Oct 20, 2026Qi Wang, Peijie Wang, Jun Cao +4LLM Agent Self-Improvement LLM Agent Harnesses
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · New Laboratory of Pattern Recognition, CASIA +1
MaD-RSI: Model-as-Data for Recursive Self-Improvement of Agent Harnesses
Oct 20, 2026Yu Cui, Hong He, Ruiqing Yue +8Agent Harness Optimization LLM Agent Harnesses
Beijing Institute of Technology · Tsinghua University · Chengdu Institute of Computer Applications, Chinese Academy of Sciences +4
Elicitation or Expansion? Demystifying Gains from Agent Harness Optimization
Oct 20, 2026Younghyun Kim, Wooseok Seo, Minseo Kim +6Agent Harness Optimization LLM Agent Harnesses
FuriosaAI · Yonsei University · University of California, Berkeley

Oct 29, 2026J/K move · Enter open · S save

Practical Assessment of Agentic Systems in Acoustics

Arne Hildenbrand, Philipp Diel, Zhuo Cao, Hanno Scharr+2

Chair of Vibroacoustics of Vehicles and Machines, Department of Engineering Physics and Computation, TUM School of Engineering and Design, Technical University of Munich, Boltzmannstrasse 15, 85748 Garching, Germany · Chair of Thermodynamics of Mobile Energy Conversion Systems, Faculty of Mechanical Engineering, RWTH Aachen University, Forckenbeckstrasse 4, 52074 Aachen, Germany · IAS-8, Forschungszentrum Jülich, Wilhelm-Johnen-Straße, 52425 Jülich, Germany

LLM Agent Harnesses LLM Agent Evaluation

Open paper AskPDF arXiv

In engineering, applications of artificial intelligence have so far been largely limited to surrogate models and optimization tasks. Large language models changed this, as they can write software from a task given in words. Placed inside a harness, a system of components that lets it search for information and execute software, such a model becomes an agent that can take over recurring engineering tasks. This work measures how the model and the elements of the harness shape the performance and behavior of the agent in two common acoustic applications. In the first, the agent detects and locates faults in audio recordings from a vehicle interior with near-perfect accuracy. A benchmark on this task shows that a weak model sets a limit that no harness design can overcome, and that design choices take effect only once the model is capable enough to use them. In the second, a case study on the design of a metamaterial unit cell, the agent reliably finds designs that achieve a specified insertion loss over a given frequency range. The results aim to help the reader understand what each element of an agent contributes and how to design an agent for acoustic tasks.