LLM Agent Harnesses

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

49 papers in the last four weeks, up 88% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 198

All topics

Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
Aug 5, 2026Jinyi Han, Yuanjian Xu, Ying Liao +6LLM Agent Evaluation LLM Agent Skill Retrieval
1East China Normal University · 2Hong Kong University of Science and Technology · 3Fudan University +1
OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents
Aug 4, 2026Jingsheng Zheng, Xinyuan Fang, Jintian Zhang +3LLM Agent Memory Long-Horizon Agent Evaluation
Zhejiang University · Ant Group · Independent Researcher
Formal Verification of Agentic Systems over Operational Data
Aug 4, 2026Alejandro J. Mercado, Alessio LomuscioFormal Verification LLM Agent Harnesses
Imperial College London · Safe Intelligence
Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
Aug 3, 2026Shuai Shao, Kangning Zhang, Qingyao Li +7Agent Harness Optimization LLM Agent Self-Improvement
Shanghai Jiao Tong University · Xiaohongshu Inc. · Southeast University
Homebot: A Personal AI Agent for Conversational Home Assistance and Automation
Aug 3, 2026Shengyuan Ye, Yixin Zhang, Han Liang +3Tool-Augmented Language Model Agents Tool-Using Agents
Power Dispatching and Controlling Center, Guangdong Power Grid Co., Ltd., Guangzhou, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China
LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
Aug 3, 2026Ziyu Ma, Hailang Huang, Shun Zou +5Long-Horizon Agent Evaluation Long-Horizon LLM Agents
DreamX Team, Alibaba Group
Towards the Harness of Embodied Agents
Aug 3, 2026Qi Wang, Tianyi Wang, Chengyang Li +6Tool-Using Agents Long-Horizon Agent Tasks
Eastern Institute of Technology, Ningbo
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Aug 3, 2026Luan Zhang, Ruochen Zhou, Dandan Song +9Agent Harness Optimization LLM Agents
Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China
Constructing Executable Analytical Knowledge Representations for Meta-Analysis Synthesis Using an Agentic Harness
Aug 3, 2026Lingbo Li, Anuradha Mathrani, Teo SusnjakAI-Assisted Scientific Research Knowledge Representation
School of Mathematical and Computational Sciences Massey University Auckland, New Zealand
GABench: A Comprehensive Benchmark for Evaluating LLM Agents on Graph Analysis Tasks
Aug 3, 2026Jiarui Tan, Zhongjian Zhang, YaBo Guo +5LLM Agent Evaluation AI Agent Benchmarks
Beijing University of Posts and Telecommunications · The Chinese University of Hong Kong · Peking University
RecSys Factory: Bounding LLM Agent Autonomy to Decision Points in the Industrial Recommender Lifecycle
Jul 31, 2026Dongyang Ao, Kaixiang Fang, Shijie XuLLM Agent Orchestration LLM Agents
FiT, Tencent Shenzhen, China
FinanceHarness: Autonomous Financial Deep Research Framework
Jul 30, 2026Yijia Xiao, Rujun Han, Yanfei Chen +8Deep Research Agents Financial Forecasting
Google Cloud AI Research · University of California, Los Angeles
Harness-G: A Graph-Structured Harness for Search Agents
Jul 30, 2026Yanning Hou, Haoyuan Chen, Sihang Zhou +7Graph Retrieval-Augmented Generation Credit Assignment in RL
National University of Defense Technology · Changsha, China
DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution
Jul 29, 2026Hanghui Guo, Weijie Shi, Zhangze Chen +6Agent Harness Optimization LLM Agent Harnesses
School of Computer Science and Engineering, Southeast University, China · Department of Computer Science and Engineering, Hong Kong University of Science and Technology, China · Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University, China +2
Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability
Jul 29, 2026Sizhe Zhou, Sheldon Yu, Hui Wei +8Agent Memory LLM Agent Memory
University of Illinois Urbana-Champaign · University of California, San Diego · University of California, Merced +2
Living-Harness Is an Interactive-Agent Evolver
Jul 29, 2026Yuetian Du, Yucheng Wang, He Xu +9AI Agent Reliability Agent Harness Optimization
Zhejiang University · The Hong Kong University of Science and Technology · Alibaba Group +2
Distributing Security Controls Through Harness Engineering
Jul 28, 2026William Robert GoreAI Agent Security LLM Agent Security
Georgia Institute of Technology
Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents
Jul 28, 2026Debjyoti PaulLanguage Model-Based Control LLM Context Management
COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution
Jul 28, 2026Jincheng Wang, Min Zheng, Tao WeiAgentic Workflows LLM Agent Harnesses
Ant Group
CogEEGAgent: Toward Autonomous Cognitive EEG Analysis with Grounded Execution and Selection-Aware Verification
Jul 27, 2026Dengzhe Hou, Lingyu Jiang, Fangzhou Lin +1Electroencephalography Selective Inference
1Graduate School of Information Sciences, Tohoku University · 2Unprecedented-scale Data Analytics Center, Tohoku University · 3Texas A&M University +1
SIREN: Towards End-to-End Extreme-Weather Early Warning with Experience-Grounded LLM Agents
Jul 27, 2026Hang Ni, Weijia Zhang, Fan Liu +2AI Agent Benchmarks LLM Agents
SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving
Jul 27, 2026Yihui Zhang, Tianyu Wo, Jinghao Wang +7LLM Inference Efficiency LLM Agent Serving
Beihang University · University of Leeds · The University of Sydney
JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents
Jul 26, 2026Yunlong Lin, Zixu Lin, Zhaohu Xing +23Multimodal Generation Multimodal Agents
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
Jul 24, 2026Shouren WangLLM Agent Memory LLM Agent Workflow Optimization
Case Western Reserve University
OpenForgeRL: Train Harness-native Agents in Any Environment
Jul 23, 2026Xiao Yu, Baolin Peng, Ruize Xu +7Computer-Use Agents LLM Agent Training
Columbia University · Microsoft Research · Dartmouth College
Delivery, Not Storage: Cue-Anchored Working Memory as a Harness Property for Coding Agents
Jul 23, 2026Swapnanil SahaAgent Memory Coding Agents
Independent
HARP: The Human--AI Research Platform
Jul 22, 2026Zeshu Zhu, Natalie Friedman, Kevin Weatherwax +1Human-in-the-Loop Evaluation Human-AI Interaction
BTPX Innovation Lab · BTPX User Assistance
NVIDIA-labs OO Agents: Native Python Object-Oriented Agents
Jul 22, 2026Paul Furgale, Severin Klingler, James Nolan +12LLM Agent Orchestration Tool-Using Agents
§ nvidia-nemo/labs-OO-Agents
FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
Jul 20, 2026Krish Agarwal, Zhuoming Chen, Yanyuan Qin +3Efficient Multimodal Inference LLM Inference Acceleration
Carnegie Mellon University · AMD · University at Buffalo
Agentic ERP: Multi-Agent Large Language Model Architecture for Autonomous Enterprise Resource Planning
Jul 19, 2026Zhihao Liu, Tianyu Wang, Xi Vincent Wang +1Multi-Agent Orchestration LLM Agent Evaluation
Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden

Aug 5, 2026, cs.CLJ/K move · Enter open · S save

Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?

Jinyi Han, Yuanjian Xu, Ying Liao, Xinyi Wang+5

1East China Normal University · 2Hong Kong University of Science and Technology · 3Fudan University+1

LLM Agent Evaluation LLM Agent Skill Retrieval LLM Agents LLM Agent Harnesses

Open paper AskPDF arXiv

Large language model (LLM) agents increasingly rely on skills, structured documents that specify when to act, which procedure to follow, and which tools are allowed. Existing evaluations mostly judge the quality of a skill or its contribution to task success, leaving unexamined whether an agent can recognize a relevant skill and apply it on its own. We introduce Skill-Use, a benchmark that evaluates skill use under progressive disclosure, where an agent sees only a skill's name and short description and must retrieve the full procedure before following it. Skill-Use separates three facets of skill use. Trigger measures whether the agent invokes the relevant skill, Compliance measures how faithfully it follows the prescribed procedure, and Boundary measures whether it avoids forbidden operations. A Skill-Use (SU) score combines the three and credits execution only after the skill is triggered. Skill-Use pairs 79 real skills with 177 executable tasks across nine domains, each grounded in real files, run in an isolated Docker sandbox, and scored by a trajectory-based rubric. Evaluating eight LLMs under two agent harnesses, we find that reliable skill use remains out of reach, as the strongest configuration reaches an SU of only 0.613. Triggering and procedural compliance fail as independent bottlenecks, and both scores and model rankings shift with the harness, so skill use behaves as a capability conditioned on the harness rather than a fixed property of the model.

Momentum

Related topics

Latest papers 198