LLM Agent Harnesses

LLM: Large Language Model

An LLM agent harness surrounds a language model with the mechanisms needed to act: tool interfaces, control loops, context and memory management, persistent state, and execution environments. Common approaches include ReAct-style loops, explicit workflow graphs, typed tool contracts, sandboxes, and verification or recovery steps; newer systems also adapt harnesses using execution traces, search, or reinforcement learning. Researchers evaluate these systems on coding, web, terminal, scientific, and interactive tasks, measuring not only success but also cost, reliability, and safety. A central tension is how much structure helps: added guidance can improve control and recovery, yet may constrain useful behavior, fail to transfer across models or tasks, or make comparisons hard to interpret without specifying the harness.

Latest papers

389 papers118 papers in the last four weeks, up 103% on the four weeks before. 0.7% of all new papers.

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
Jul 20, 2026Krish Agarwal, Zhuoming Chen, Yanyuan Qin +3Efficient Multimodal Inference LLM Inference Acceleration
Carnegie Mellon University · AMD · University at Buffalo
Agentic ERP: Multi-Agent Large Language Model Architecture for Autonomous Enterprise Resource Planning
Jul 19, 2026Zhihao Liu, Tianyu Wang, Xi Vincent Wang +1Multi-Agent Orchestration LLM Agent Evaluation
Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden
Where Does Agent Reliability Come From? A Cross-Benchmark Decomposition of Verification Loops, Specialist Models, and Scaffolding in a Production Enterprise Agent
Jul 19, 2026Arunabh DastidarAI Agent Reliability LLM Answer Verification
for the Leni Team · Leni Inc.
DataFlow-Harness: A Grounded Code-Agent Platform for Constructing Editable LLM Data Pipelines
Jul 18, 2026Runming He, Zhen Hao Wong, Hao Liang +4LLM Grounding Code Generation
Peking University · Institute for Advanced Algorithms Research, Shanghai · Zhongguancun Academy
Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents
Jul 17, 2026Zhengyu Chen, Teng Xiao, Huaisheng Zhu +3Agent Harness Optimization LLM Agent Workflow Optimization
Meituan · Allen Institute for AI · Independent
Recursive Harness Self-Improvement
Jul 17, 2026Hyunin Lee, Jinglue Xu, Jeffrey Seely +3Continual Learning for LLM Agents Agent Harness Optimization
1Sakana AI, 2UC Berkeley
Harnessing LLMs for Reliable Academic Supervision: A Comparative Study
Jul 16, 2026Akash RajLLM Reliability LLM Agent Harnesses
M.Tech, School of Artificial Intelligence and Data Science Indian Institute of Technology Jodhpur
RetroAgent: Harnessing LLMs to Search Over Structured Memory for Agentic Retrosynthesis Planning
Jul 16, 2026Yanqiao Zhu, Jingru Gan, Xiaoqi Sun +8LLM Agent Harnesses LLM Agent Planning
UCLA · MIT · Amazon +1
AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
Jul 15, 2026Kai Chen, Zichen Ding, Jiaye Ge +20LLM Agent Evaluation AI Agent Benchmarks
AgentCompass Team, Shanghai AI Laboratory
Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity
Jul 15, 2026Xiaotian Luo, Fengxingyu Wang, Chuanrui Hu +2LLM Agent Self-Improvement Quality-Diversity Optimization
EverMind AI · Shanda Group
Structured Feedback Improves Repair in an LLM Agent Loop
Jul 15, 2026Jaideep Ray, Ankit GoyalLLM Agent Workflow Optimization LLM Agent Evaluation
Independent Researcher
MyAG: A Graph-Based Framework for Designing and Analyzing Composable LLM Agent Systems
Jul 15, 2026Zhisong ZhangLLM Agent Orchestration LLM Agents
Department of Computer Science, City University of Hong Kong
Harness Handbook: Making Evolving Agent Harnesses Readable,Navigable, and Editable
Jul 14, 2026Ruhan Wang, Yucheng Shi, Zongxia Li +7Software Engineering Agents Code Localization
Tencent · Indiana University · University of Maryland, College Park +2
PalmClaw: A Native On-Device Agent Framework for Mobile Phones
Jul 14, 2026Hongru Cai, Yongqi Li, Ran Wei +1Tool-Using Agents LLM Agents
The Hong Kong Polytechnic University · Hangzhou Diagens Biotechnology Co., Ltd.
Rethinking the Evaluation of Harness Evolution for Agents
Jul 14, 2026Yike Wang, Huaisheng Zhu, Zhengyu Hu +8LLM Agent Evaluation Agent Harness Optimization
Allen Institute for AI · University of Washington · Independent
ToFu: A White-Box, Token-Efficient Agent Harness for Researchers
Jul 13, 2026Junhao Ruan, Yuan Ge, Bei Li +7Tool-Augmented Language Model Agents Software Engineering Agents
Northeastern University, China · LongCat RSI, Meituan · NiuTrans Research
Agentic Routing: The Harness-Native Data Flywheel
Jul 13, 2026Xinchen Liu, Hang Zhou, Yingjie Zong +12LLM Agent Training LLM Agent Harnesses
TokenRhythm Technologies
StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure
Jul 13, 2026Wenyi Wu, Sibo Zhu, Kun Zhou +3Computer-Use Agents Long-Horizon LLM Agents
University of California, San Diego · 2Aether AI Lab · §Work done during an internship at Aether AI.
Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime for Procedural LLM Agents
Jul 13, 2026Chenglin Yu, Li Yin, Ying Yu +4Instruction Following Runtime Enforcement for AI Agents
Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University · Department of Data and Systems Engineering, The University of Hong Kong · College of Economics and Management, Zhejiang Normal University +3
A Formal Hierarchical Architecture for Agentic Orchestration with Stack-Based Execution and Lazy Discovery
Jul 13, 2026Prashant Devadiga, Abhishek, Adithya Mishra +9LLM Agent Orchestration Tool-Using Language Agents
UPI Help
LOGOS: A Living Logic for AI Agent Teams That Evolve With Humans
Jul 12, 2026Yuma Ichikawa, Yamato Arai, Kosaku Kimura +2Multi-Agent LLM Systems Human-in-the-Loop AI
1Fujitsu Limited · 2RIKEN Center for AIP · 4Tokai University
Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes
Jul 11, 2026Hiroki TambaAI Agent Reliability LLM Agent Memory
Independent Researcher
MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation
Jul 11, 2026Chengguang Gan, Hanjun Wei, Yunhao Liang +3Computer-Use Agents Web Agent Benchmarks
Independent Researcher · University of Chinese Academy of Sciences · Monash University +2
Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents
Jul 10, 2026Izumi Takahara, Teruyasu MizoguchiAI Agents for Scientific Discovery LLM Auditing
Institute of Industrial Science, The University of Tokyo, Tokyo, Japan 153-8505
UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks
Jul 9, 2026Zhekai Chen, Chengqi Duan, Kaiyue Sun +4AI Agent Benchmarks Multimodal Agents
HKU MMLab · Meituan
TTHE: Test-Time Harness Evolution
Jul 9, 2026Jun Nie, Yonggang Zhang, Jun Song +5Test-Time Adaptation Agent Harness Optimization
Hong Kong Baptist University · University of Science and Technology of China · Hong Kong Generative AI Research & Development Center, The Hong Kong University of Science and Technology +1
From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
Jul 9, 2026Joongho Ahn, Moonsoo KimLLM Auditing LLM Guardrails
AI Leadership Research Center
The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI
Jul 8, 2026Muayad Sayed Ali, Aliaksandra Novik, Anji Boddupally +28LLM Inference Efficiency LLM Agent Orchestration
Writer, Inc.
Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
Jul 7, 2026Kabir Moghe, Peter ChinLLM-Based Program Synthesis Abstract Reasoning
Department of Computer Science Dartmouth College
Harnessing Code Agents for Automatic Software Verification
Jul 7, 2026Shuangxiang Kan, Shuanglong Kan, Sebastian ErtelSoftware Engineering Agents Automated Theorem Proving
Singapore Management University · Singapore · Barkhausen Institut +1

Jul 20, 2026J/K move · Enter open · S save

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu+2

Carnegie Mellon University · AMD · University at Buffalo

Efficient Multimodal Inference LLM Inference Acceleration LLM Agent Harnesses

Open paper AskPDF arXiv

Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.