AI Agent Benchmarks

Follow this topic Ask about this topic

AI agent benchmarks test systems that act over multiple steps, use tools, and respond to changing information, rather than only answer isolated questions. Researchers build executable environments for settings such as web search, coding, graphical interfaces, games, scientific workflows, and multi-agent coordination, then score outcomes with programmatic checks, expert rubrics, or simulated users. Increasingly, evaluations also inspect trajectories, evidence use, efficiency, safety, and recovery from errors. Agents can perform well on narrow tasks yet remain unreliable on long-horizon work, and results often depend strongly on the model, harness, environment, and grading method.

Latest papers

1,503 papers271 papers in the last four weeks, up 91% on the four weeks before. 1.6% of all new papers.

EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents
Apr 7, 2026Xuan Dong, Huanyang Zheng, Tianhao Niu +7AI Agent Benchmarks AI Agents for Scientific Discovery
Harbin Institute of Technology
Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition
Apr 7, 2026Yushuo Zheng, Huiyu Duan, Zicheng Zhang +3LLM Agent Evaluation AI Agent Benchmarks
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
Apr 6, 2026Yiwen Song, Yale Song, Tomas Pfister +1AI-Assisted Scientific Research AI Agent Benchmarks
Google
Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation
Apr 6, 2026Alhasan Mahmood, Samir Abdaljalil, Hasan KurbanMultilingual Language Model Evaluation LLM-as-a-Judge
Hamad Bin Khalifa University, Doha, Qatar · Texas A&M University, College Station, TX USA
GUIDE: Interpretable GUI Agent Evaluation via Hierarchical Diagnosis
Apr 6, 2026Yuwen Zhai, Runze Li, Liang Wang +6AI Agent Evaluation GUI Agents
Alibaba Group · School of Computer Science and Technology, East China Normal University · Shanghai Innovation Institute
RoboPhD: Evolving Diverse Complex Agents Under Tight Evaluation Budgets
Apr 6, 2026Andrew Borthwick, Stephen Ash, Anthony GalczakLLMs for Evolutionary Optimization Large Language Model-Guided Optimization
Independent Researchers
ClawArena: Benchmarking AI Agents in Evolving Information Environments
Apr 5, 2026Haonian Ji, Kaiwen Xiong, Siwei Han +9AI Agent Evaluation AI Agent Benchmarks
UNC-Chapel Hill · University of California, Berkeley · University of California, Santa Cruz
Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
Apr 3, 2026Qianshan Wei, Yishan Yang, Siyi Wang +12Multimodal Agent Evaluation Agent Evaluation
CASIA · UCAS · SEU +5
OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments
Apr 3, 2026Felix Henry, Lihan Lin, Jiangyou Zhu +4Mobile GUI Automation GUI Agents
XPeng Motors
ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents
Apr 3, 2026Chao Li, Cailiang Liu, Ang Gao +7Synthetic Clinical Data Generation AI Agent Benchmarks
Shanda Group
NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration
Apr 3, 2026Ahmed Twabi, Yepeng Ding, Tohru KondoFinite-State Machine LLM Agent Evaluation
Informatics & Data Science, Hiroshima University, Hiroshima, Japan
GBQA: A Game Benchmark for Evaluating LLMs as Quality Assurance Engineers
Apr 3, 2026Shufan Jiang, Chios Chen, Zhiyang ChenAutomated Software Engineering Software Engineering Agents
The University of Hong Kong · Datawhale Org. · Independent Researcher +1
Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
Apr 2, 2026Dat Tran, Douwe KielaMulti-Agent LLM Systems Multi-Agent Systems
Stanford University
GeoAI Agency Primitives
Apr 2, 2026Akram Zaytar, Rohan Sawahn, Caleb Robinson +5Agentic Workflows AI Agent Benchmarks
Microsoft AI for Good Lab · NASA Harvest
MolClaw: An Autonomous Agent with Hierarchical Skills for Drug Molecule Evaluation, Screening, and Optimization
Apr 2, 2026Lisheng Zhang, Lilong Wang, Xiangyu Sun +14Drug Discovery Autonomous Research Agents
Peking University Health Science Center, Peking University, Beijing, China · Shanghai AI Laboratory, Shanghai, China · Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China
PHMForge: Evaluating LLM Agents on Industrial Prognostics through MCP-Native, Algorithm-Grounded Tools
Apr 2, 2026Yusheng Li, Tianjun Feng, Yunfeng Chen +6LLM Agent Evaluation AI Agent Benchmarks
Columbia University · Georgia Institute of Technology · IBM, New York
REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
Apr 2, 2026Smriti Jha, Matteo Paltenghi, Chandra Maddila +3Coding Agent Benchmarks Benchmark Construction
Meta, USA
ToolMisuseBench: An Offline Deterministic Benchmark for Tool Misuse and Recovery in Agentic Systems
Apr 2, 2026Akshey Sigdel, Rista BaralTool-Use Evaluation Tool-Using Agents
Independent Researcher
HippoCamp: Benchmarking Contextual Agents on Personal Computers
Apr 1, 2026Zhe Yang, Shulin Tian, Kairui Hu +9AI Agent Benchmarks Multimodal Search Agents
S-Lab, Nanyang Technological University, Singapore · Synvo AI
$\texttt{YC-Bench}$ : Benchmarking AI Agents for Long-Term Planning and Consistent Execution
Apr 1, 2026Muyu He, Adit Jain, Anand Kumar +4Long-Horizon Agent Evaluation AI Agent Benchmarks
Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants
Apr 1, 2026Deepak Nathani, Cheng Zhang, Chang Huan +7Proactive Agents Proactive Assistance
University of California, Santa Barbara · Independent Researcher, USA · Apple, USA +1
Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems
Apr 1, 2026Dongzhe Fan, Zheyi Xue, Siyuan Liu +1Graph Retrieval-Augmented Generation RAG Evaluation
New York University Shanghai, Shanghai, China
Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents
Apr 1, 2026Alibek Kaliyev, Artem MaryanskyyAI Agent Reliability Tool-Use Evaluation
The University of Texas at Austin · Austin, Texas, USA · Uber +1
In harmony with gpt-oss
Apr 1, 2026Borislav MavrinAI Agent Benchmarks Tool-Use Evaluation
Improvisational Games as a Benchmark for Social Intelligence of AI Agents: The Case of Connections
Mar 31, 2026Gaurav Rajesh Parikh, Angikar GhosalAI Agent Benchmarks LLM Agent Evaluation
Graduate School of Business, Stanford University, 655 Knight Way, Stanford, CA 94305
Benchmarking Interaction, Beyond Policy: a Reproducible Benchmark for Collaborative Instance Object Navigation
Mar 31, 2026Edoardo Zorzi, Francesco Taioli, Yiming Wang +4AI Agent Benchmarks Embodied Navigation
Sapienza University of Rome, Rome, Italy · University of Verona, Verona, Italy · Polytechnic of Turin +2
ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation
Mar 31, 2026Yinuo Liu, Zi Qian, Heng Zhou +7Tool Use in VLMs Multimodal Tool Use
Qwen Business Unit, Alibaba Group, China · Huazhong University of Science and Technology, China · Zhejiang University, China
6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management
Mar 31, 2026Jiao Chen, Jianhua Tang, Xiaotong Yang +1AI Agent Benchmarks 6G Networks
Shenzhen Smart City Technology Development Group Company, Ltd. · Shien-Ming Wu School of Intelligent Engineering, South China University of Technology · China Unicom Group Co., Ltd.
ELT-Bench-Verified: Benchmark Quality Issues Underestimate AI Agent Capabilities
Mar 31, 2026Christopher Zanoli, Andrea Giovannini, Tengjun Jin +2AI Agent Benchmarks Benchmark Auditing
IBM Research · ETH Zurich · University of Illinois (UIUC)
AEC-Bench: A Multimodal Benchmark for Agentic Systems in Architecture, Engineering, and Construction
Mar 31, 2026Harsh Mankodiya, Chase Gallik, Theodoros Galanos +1AI Agent Benchmarks Multimodal Agents

Apr 7, 2026J/K move · Enter open · S save

EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agents

Xuan Dong, Huanyang Zheng, Tianhao Niu, Zhe Han+6

Harbin Institute of Technology

AI Agent Benchmarks AI Agents for Scientific Discovery Evidence Retrieval Multimodal Reasoning

Open paper AskPDF arXiv

Scientific research follows multi-turn, multi-step workflows that require proactively searching the literature, consulting figures and tables, and integrating evidence across papers to align experimental settings and support reproducible conclusions. This joint capability is not systematically assessed in existing benchmarks, which largely under-evaluate proactive search, multi-evidence integration and sustained evidence use over time. In this work, we introduce EpiBench, an episodic multi-turn multimodal benchmark that instantiates short research workflows. Given a research task, agents must navigate across papers over multiple turns, align evidence from figures and tables, and use the accumulated evidence in the memory to answer objective questions that require cross paper comparisons and multi-figure integration. EpiBench introduces a process-level evaluation framework for fine-grained testing and diagnosis of research agents. Our experiments show that even the leading model achieves an accuracy of only 29.23% on the hard split, indicating substantial room for improvement in multi-turn, multi-evidence research workflows, providing an evaluation platform for verifiable and reproducible research agents.