AI Agent Benchmarks

Ask about this topic

Overview

AI agent benchmarks test systems that act over multiple steps, use tools, and respond to changing information, rather than only answer isolated questions. Researchers build executable environments for settings such as web search, coding, graphical interfaces, games, scientific workflows, and multi-agent coordination, then score outcomes with programmatic checks, expert rubrics, or simulated users. Increasingly, evaluations also inspect trajectories, evidence use, efficiency, safety, and recovery from errors. Agents can perform well on narrow tasks yet remain unreliable on long-horizon work, and results often depend strongly on the model, harness, environment, and grading method.

Latest papers

CardsList

1,503 papers271 papers in the last four weeks, up 91% on the four weeks before. 1.6% of all new papers.

  1. Test-Driven AI Agent Definition (TDAD): Compiling Tool-Using Agents from Behavioral Specifications

    Mar 9, 2026Tzafrir RehanTool-Using AgentsAI Agent Evaluation

    Fiverr Labs

  2. OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning

    Mar 9, 2026Krista Opsahl-Ong, Arnav Singhvi, Jasmine Collins +10Multi-Document QALLM Agent Evaluation

    Databricks

  3. PostTrainBench: Can LLM Agents Automate LLM Post-Training?

    Mar 9, 2026Ben Rank, Hardik Bhatnagar, Ameya Prabhu +4LLM Post-TrainingAutonomous Research Agents

    ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +2

  4. FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use

    Mar 9, 2026Jiaxuan Lu, Kong Wang, Yemin Wang +9LLM Agent EvaluationTool-Using Language Agents

    Shanghai AI Laboratory · Hunan University · Xiamen University +3

  5. UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking

    Mar 9, 2026Chang Liu, Chuqiao Kuang, Tianyi Zhuang +4Agentic SearchWeb Search Agents

    Huawei Technologies Ltd. · The University of Hong Kong · University College London

  6. PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents

    Mar 9, 2026Yuxiang Chai, Shunye Tang, Han Xiao +2GUI AgentsAI Agent Benchmarks

    Nankai University · Huawei Research

  7. $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Mar 9, 2026Yang Liu, Jiaqi Li, Jun Bai +20LLM Agent EvaluationAI Agent Benchmarks

    BIGAI · Humanlaya · xbench +1

  8. ResearchEnvBench: Benchmarking Agents on Environment Synthesis for Research Code Execution

    Mar 6, 2026Yubang Wang, Chenxi Zhang, Bowen Chen +7AI Agent BenchmarksAutonomous Research Agents

    Shanghai Innovation Institution · College of Computer Science and Artificial Intelligence, Fudan University · OpenMOSS +5

  9. DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality

    Mar 6, 2026Yukun Huang, Leonardo F. R. Ribeiro, Momchil Hardalov +3Automated Fact-CheckingAI Agent Benchmarks

    Duke University · Amazon AGI · Boston University

  10. The World Won't Stay Still: Programmable Evolution for Agent Benchmarks

    Mar 6, 2026Guangrui Li, Yaochen Xie, Yi Liu +11AI Agent BenchmarksAgent Evaluation

    Amazon · UC Berkeley

  11. TML-Bench: Benchmark for Data Science Agents on Tabular ML Tasks

    Mar 5, 2026Mykola PinchukAI Agent BenchmarksAgent Benchmarks

    Independent Researcher, San Jose, USA

  12. Tool-Genesis: A Task-Driven Tool Creation Benchmark for Self-Evolving Language Agent

    Mar 5, 2026Bowei Xia, Mengkang Hu, Shijian Wang +5Tool-Using AgentsAI Agent Benchmarks

    UESTC · The University of Hong Kong · Xiaohongshu Inc.

  13. EigenData: A Self-Evolving Multi-Agent Platform for Function-Calling Data Synthesis, Auditing, and Repair

    Mar 5, 2026Jiaao Chen, Jingyuan Qi, Mingye Gao +3Synthetic Data Generation for Language ModelsAI Agent Benchmarks

    Eigen AI

  14. Evaluating the Search Agent in a Parallel World

    Mar 5, 2026Jiawei Chen, Xintian Shen, Lihao Zheng +7LLM Agent EvaluationWeb Search Agents

  15. iAgentBench: Benchmarking Sensemaking Capabilities of Information-Seeking Agents on High-Traffic Topics

    Mar 4, 2026Preetam Prabhu Srikar Dammu, Arnav Palkhiwala, Tanya Roosta +1AI Agent BenchmarksLLM Agent Evaluation

    University of Washington Seattle, Washington, USA · UC Berkeley Berkeley, California, USA

  16. Benchmarking Emergent Coordination in Large-Scale LLM Populations: An Evaluation Framework on the MoltBook Archive

    Mar 3, 2026Brandon Yee, Pairie KohMulti-Agent CoordinationLLM Agent Evaluation

    Management Lab, Yee Collins Research Group

  17. Proact-VL: A Proactive VideoLLM for Real-Time AI Companions

    Mar 3, 2026Weicai Yan, Yuhong Dai, Qi Ran +6AI CompanionsVision-Language Agents

    Zhejiang University · Shenzhen University · South China University of Technology +2

  18. Valet: A Standardized Testbed of Traditional Imperfect-Information Card Games

    Mar 3, 2026Mark Goadrich, Achille Morenville, Éric PietteImperfect-Information GamesAI Agent Benchmarks

    Hendrix College, Conway AR 72032, USA · ICTEAM, UCLouvain, Louvain-la-Neuve, Belgium

  19. Agentified Assessment of Logical Reasoning Agents

    Mar 3, 2026Zhiyu Ni, Yifeng Xiao, Zheng LiangLLM Agent EvaluationAI Agent Benchmarks

    University of California, Berkeley

  20. SorryDB: Can AI Provers Complete Real-World Lean Theorems?

    Mar 3, 2026Austin Letson, Leopoldo Sarra, Auguste Poiroux +9Automated Theorem ProvingLean Theorem Proving

    Axiomatic AI · Math, Inc. · EPFL +8

  21. LiveAgentBench: Comprehensive Benchmarking of Agentic Systems Across 104 Real-World Challenges

    Mar 3, 2026Hao Li, Huan Wang, Jinjie Gu +3AI Agent EvaluationAI Agent Benchmarks

    Ant Group

  22. TritonDFT: Automating DFT with a Multi-Agent Framework

    Mar 2, 2026Zhengding Hu, Kuntal Talit, Zhen Wang +8Scientific Workflow AutomationDensity Functional Theory

    University of California, San Diego · University of California, Merced · Los Alamos National Laboratory

  23. Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

    Mar 2, 2026Haochuan WangLLM Inference EfficiencyLLM Agent Evaluation

    Massachusetts Institute of Technology

  24. Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scale

    Mar 2, 2026Hao Li, Chunjiang Mu, Jianhao Chen +5LLM Agent OrchestrationLLM Agent Skill Retrieval

    Shanghai Artificial Intelligence Laboratory

  25. Pencil Puzzle Bench: A Benchmark for Multi-Step Verifiable Reasoning

    Mar 2, 2026Justin WaughLLM EvaluationLLM Reasoning

    Approximate Labs

  26. LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulations

    Mar 2, 2026Viet-Thanh Pham, Lizhen Qu, Thuy-Trang Vu +2AI Agent BenchmarksCross-Cultural Language Model Evaluation

    Department of Data Science & AI, Monash University

  27. FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents

    Mar 2, 2026Qizheng Li, Yifei Zhang, Xiao Yang +4LLM Fine-TuningAI Agent Benchmarks

    Microsoft Research Asia · Peking University · Nanjing University +1

  28. CeProAgents: A Hierarchical Agents System for Automated Chemical Process Development

    Mar 2, 2026Yuhang Yang, Ruikang Li, Jifei Ma +8Multi-Agent LLM SystemsLLM Agent Orchestration

    State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China, Hefei, China · Dalian Institute of Chemical Physics, Chinese Academy of Sciences, Dalian, China

  29. ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Context

    Mar 2, 2026Zidi Xiu, David Q. Sun, Kevin Cheng +9Tool-Use EvaluationAI Agent Benchmarks

    Apple

  30. How Well Does Agent Development Reflect Real-World Work?

    Mar 1, 2026Zora Zhiruo Wang, Sanidhya Vijayvargiya, Aspen Chen +7AI Agent BenchmarksAI and Labor