LLM Agent Harnesses

LLM: Large Language Model

An LLM agent harness surrounds a language model with the mechanisms needed to act: tool interfaces, control loops, context and memory management, persistent state, and execution environments. Common approaches include ReAct-style loops, explicit workflow graphs, typed tool contracts, sandboxes, and verification or recovery steps; newer systems also adapt harnesses using execution traces, search, or reinforcement learning. Researchers evaluate these systems on coding, web, terminal, scientific, and interactive tasks, measuring not only success but also cost, reliability, and safety. A central tension is how much structure helps: added guidance can improve control and recovery, yet may constrain useful behavior, fail to transfer across models or tasks, or make comparisons hard to interpret without specifying the harness.

Latest papers

389 papers118 papers in the last four weeks, up 103% on the four weeks before. 0.7% of all new papers.

Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agents
Jul 5, 2026Haiwen Yi, Xinyuan SongSoftware Engineering Agents LLM Agent Evaluation
University of Toronto · Emory University
HarnessMDP: Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
Jul 5, 2026Haiwen Yi, Xinyuan SongOffline RL LLM Agent Harnesses
University of Toronto · Emory University
MechMath Agent Team: LLM Driven Agents for Mathematical Research
Jul 5, 2026Yichuan Cao, Ruichen Qiu, Junqi Liu +5Automated Theorem Proving Multi-Agent Reasoning
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, CAS · School of Mathematical Sciences, University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences
Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions
Jul 4, 2026Haochen Luo, Yi Huang, Sichun Luo +5Agentic RL Agent Harness Optimization
School of Computing and Data Science, The University of Hong Kong · Grace Investment Machine · Jiutian Research, China Mobile
LLMoxie: Exploring Agentic AI for Scientific Software Development
Jul 2, 2026Landung Setiawan, Anant Mittal, Cordero Core +5Scientific Code Generation AI Agent Governance
eScience Institute University of Washington Seattle, Washington, USA · eScience Institute Dept of Chemical Engineering University of Washington Seattle, Washington, USA · eScience Institute Dept of Astronomy University of Washington Seattle, Washington, USA
COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflows
Jul 2, 2026Zongxia Li, Dawei Liu, Fuxiao Liu +6Image Generation Agentic Image Generation
University of Maryland · University of Pennsylvania · Nvidia +1
AgentLTL: A Trace-Verification Framework for Measuring, Enforcing, and Training Procedural Compliance in Tool-Using LLM Agents
Jul 1, 2026Laïla Elkoussy, Julien PerezLLM Agent Evaluation Formal Verification
LRE, EPITA · LRE · Bpifrance
Evaluating Agentic Harness Systems for Autonomous Computational Pathology
Jul 1, 2026Jie Lin, Zongyi Chen, Qiaoling Zheng +6Computational Pathology LLM Agent Evaluation
Department of Computer Science at School of Informatics, Xiamen University, Xiamen, China. · National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China. · Department of Pathology, Fujian Medical University Union Hospital, Fuzhou, China. +1
Self-GC: Self-Governing Context for Long-Horizon LLM Agents
Jul 1, 2026Xubin Hao, Hongjin Meng, Xin Yin +2Long-Horizon LLM Agents LLM Context Management
Xiaohongshu
ShopX: A Foundation Model for Intent-to-Item Fulfillment in Agentic Shopping
Jun 30, 2026Jiacheng Chen, Tao Zhang, Manxi Lin +26Large Language Model-Based Recommendation E-Commerce
Alibaba Taobao & Tmall Group
SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks
Jun 30, 2026Seongho Son, Sangwoong Yoon, Jiahua Tang +3Software Engineering Agents LLM Agent Workflow Optimization
University College London, United Kingdom · Ulsan National Institute of Science and Technology, South Korea · PSL Research University, France +1
Agentic Hardware Design as Repository-Level Code Evolution
Jun 26, 2026Cunxi Yu, Chenhui Deng, Nathaniel Pinckney +1Software Engineering Electronic Design Automation
NVIDIA Research
The Interplay of Harness Design and Post-Training in LLM Agents
Jun 24, 2026Kyungmin Kim, Youngbin Choi, Seoyeon Lee +3Agent Harness Optimization LLM Agents
Graduate School of Artificial Intelligence, POSTECH · Department of Computer Science and Engineering, POSTECH
Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents
Jun 20, 2026Wangxuan Fan, Xiaoyu Nie, Zhongxiang DaiPrivacy-Preserving Language Models LLM Agent Security
The Chinese University of Hong Kong, Shenzhen
Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents
Jun 19, 2026Han Chi, Jiaxin Qi, Yan Cui +2LLM Agent Evaluation AI Agent Benchmarks
Computer Network Information Center, Chinese Academy of Sciences · Hangzhou Institute for Advanced Study, Chinese Academy of Sciences
Building Agent Harnesses for Scientific Curation from Multimodal Sources
Jun 19, 2026Sheng Zhang, Qin Liu, Renqian Luo +9Document Information Extraction Scientific Workflow Automation
Microsoft Research · University of California, Davis · Merck & Co., Inc., Rahway, NJ, USA
Dissecting model behavior through agent trajectories
Jun 16, 2026Gaurav Gupta, Vatshank Chaturvedi, Jun Huan +1Software Engineering Agents LLM Agent Evaluation
AWS AI Labs
RHO: Your Coding Agent is Secretly a Roboticist
Jun 15, 2026Karim Elmaaroufi, Justin Svegliato, Sarunas Kalade +3LLM Agent Harnesses Robotic Manipulation
University of California, Berkeley · Embodied Science · AMD
From Question Answering to Task Completion: A Survey on Agent System and Harness Design
Jun 14, 2026Jianyuan Guo, Zhiwei Hao, Chengcheng Wang +14LLM Agent Evaluation LLM Agents
Department of Computer Science, City University of Hong Kong, HKSAR, China
HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
Jun 12, 2026Tingyang Chen, Shuo Lu, Kang Zhao +11LLM Agent Orchestration Agent Harness Optimization
Darwin Agent Team
Recursive Agent Harnesses
Jun 11, 2026Elias Lumer, Sahil Sen, Kevin Paul +1Tool-Augmented Language Model Agents LLM Agent Orchestration
PricewaterhouseCoopers, U.S.
HarnessBridge: Learnable Bidirectional Controller for LLM Agent Harness
Jun 11, 2026Xiaoxuan Wang, Haixin Wang, Alexander Taylor +3Agent Harness Optimization LLM Agent Training
University of California, Los Angeles
What makes a harness a harness: necessary and sufficient conditions for an agent harness
Jun 8, 2026Sanderson Oliveira de MacedoSoftware Engineering Agents AI Coding Agents
Federal Institute of Goiás
Self-Harness: Harnesses That Improve Themselves
Jun 8, 2026Hangfan Zhang, Shao Zhang, Kangcong Li +5Agent Harness Optimization LLM Agent Self-Improvement
Shanghai Artificial Intelligence Laboratory
Bayesian-Agent: Posterior-Guided Skill Evolution for LLM Agent Harnesses
Jun 6, 2026Xiaojun Wu, Cehao Yang, Honghao Liu +7Agent Harness Optimization LLM Agent Workflow Optimization
1IDEA Research · 2The Hong Kong University of Science and Technology (Guangzhou) · 3DataArcTech Ltd.
From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flaws
Jun 4, 2026Mengzhuo Chen, Junjie Wang, Zhe Liu +3Agent Failure Analysis Self-Improving Agents
State Key Laboratory of Complex System Modeling and Simulation Technology, Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China +1
Exploring Cross-Scenario Generality of Agentic Memory Systems: Diagnostics and a Strong Baseline
Jun 3, 2026Zhikai Chen, Jialiang Gu, Junyu Yin +6Agent Memory LLM Agent Memory
Michigan State University · George Mason University · Purdue University
MUSE: A Unified Agentic Harness for MLLMs
Jun 2, 2026Jianglin Lu, Hailing Wang, Xu Ma +4Multimodal Reasoning Multimodal Agents
Northeastern University
HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems
Jun 1, 2026Mingju Chen, Can Lv, Guibin Zhang +2Agent Harness Optimization LLM Agent Self-Improvement
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University · Tsinghua University
Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams
Jun 1, 2026Zewen Liu, Zhan Shi, Yisi Sang +7Continual Learning for LLM Agents Agent Harness Optimization
Emory University · Amazon · The Pennsylvania State University +2

Jul 5, 2026J/K move · Enter open · S save

Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agents

Haiwen Yi, Xinyuan Song

University of Toronto · Emory University

Software Engineering Agents LLM Agent Evaluation Long-Horizon Agent Evaluation LLM Agent Harnesses

Open paper AskPDF arXiv

Software agents are becoming increasingly important in modern coding workflows, where they solve complex tasks through multi-step interaction with code repositories. A central component of these systems is the execution harness, which governs how the underlying LLM observes, acts on, and verifies its environment. We show that the harness can in turn shape the agent's evolving beliefs about the environment, even when the task, environment, and base LLM remain fixed. To reveal this effect, we introduce a belief-rollout diagnostic that traces how an agent's anticipated world state and future decisions evolve under alternative harnesses. We further separate immediate divergence introduced by the interface from divergence that develops over longer horizons. Across controlled coding tasks include HIBench-Code, SWE-bench Verified and Terminal-Bench, we find substantial cross-harness belief divergence that is often invisible from terminal success alone and can propagate into different downstream actions. Building on this observation, we introduce Belief-Invariant World-Modeling (BIWM), which exposes evidence hidden or compressed by harness execution and aligns the resulting belief trajectories across different harnesses. Our findings reveal that harness design can systematically shape how software agents understand and act on their environments, providing practical guidance for building and evaluating more reliable coding agents. Our code is available at https://github.com/Hik289/Harness-induce-bias.git.