AI Agent Benchmarks

Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,013

All topics
CardsList

  1. SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systems

    Apr 17, 2026Hikaru Shindo, Hanzhao Lin, Lukas Helff +2LLM Agent EvaluationAI Agent Benchmarks

    Technical University of Darmstadt · German Research Center for Artificial Intelligence · CERTAIN +2

  2. GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows

    Apr 17, 2026Jize Wang, Xuanxuan Liu, Yining Li +7Tool-Use EvaluationLong-Horizon Agent Evaluation

    1Shanghai Jiao Tong University · 2Shanghai AI Laboratory · 3Tencent +1

  3. Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints

    Apr 17, 2026Xinge Liu, Terry Jingchen Zhang, Bernhard Schölkopf +2AI Agents for Scientific DiscoveryAI Agent Benchmarks

    University of Toronto · 2Vector Institute · 3Max Planck Institute for Intelligent Systems, Tübingen, Germany +1

  4. PolicyBank: Evolving Policy Understanding for LLM Agents

    Apr 16, 2026Jihye Choi, Jinsung Yoon, Long T. Le +2AI Agent BenchmarksAI Agent Governance

    Google Cloud · University of Wisconsin-Madison

  5. CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas

    Apr 16, 2026Emanuel Tewolde, Xiao Zhang, David Guzman Piedrahita +2Social DilemmasCooperative Game Theory

    Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL) · Jinesis Lab, University of Toronto & Vector Institute +3

  6. PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research

    Apr 16, 2026Tingjia Miao, Wenkai Jin, Muhua Zhang +19LLM EvaluationAI Agent Benchmarks

    School of Artificial Intelligence, Shanghai Jiao Tong University · Zhiyuan College, Shanghai Jiao Tong University · School of Physics and Astronomy, Shanghai Jiao Tong University +7

  7. From Reactive to Proactive: Assessing the Proactivity of Voice Agents via ProVoice-Bench

    Apr 16, 2026Ke Xu, Yuhao Wang, Yu WangLLM Agent EvaluationAgent Evaluation

    Shanghai Jiao Tong University

  8. Benchmarks for Trajectory Safety Evaluation and Diagnosis in OpenClaw and Codex: ATBench-Claw and ATBench-Codex

    Apr 16, 2026Zhonghao Yang, Yu Li, Yanxu Zhu +6Benchmark DesignSoftware Engineering Benchmarks

    Shanghai Artificial Intelligence Laboratory

  9. MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror

    Apr 16, 2026Shengyu Guo, Tongrui Ye, Jianbo Zhang +3Multimodal Large Language ModelsAI Agent Benchmarks

    Shanghai AI Lab

  10. HWE-Bench: Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks

    Apr 16, 2026Fan Cui, Hongyuan Hou, Zizhang Luo +2Automated Program RepairElectronic Design Automation

    Peking University · Beijing, China

  11. DR3^{3}-Eval: Towards Realistic and Reproducible Deep Research Evaluation

    Apr 16, 2026Qianqian Xie, Qingheng Xiong, He Zhu +16LLM Agent EvaluationLong-Horizon Agent Evaluation

    Nanjing University · M-A-P · Nanjing University of Science and Technology +2

  12. Mind DeepResearch Technical Report

    Apr 16, 2026MindDR Team, Li Auto IncMulti-Agent LLM SystemsDeep Research Agents

    MindDR Team, Li Auto Inc

  13. HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

    Apr 15, 2026Jiacheng Wang, Jinchang Hou, Fabian Wang +3AI Agent AuditingLong-Horizon Agent Evaluation

    Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China

  14. UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

    Apr 13, 2026Yijuan Liang, Xinghao Chen, Yifan Ge +5LLM Agent EvaluationAI Agent Benchmarks

    University of Science and Technology of China · Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo · Department of Computing, The Hong Kong Polytechnic University

  15. NetForge RL: A Multi-Agent Simulation Environment for Cyber Defense with Durative Actions

    Apr 10, 2026Igor JankowskiAutonomous Cyber DefenseCybersecurity

  16. Riemann-Bench: A Benchmark for Moonshot Mathematics

    Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Mathematical Reasoning BenchmarksBenchmark Design

    Surge AI

  17. Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation

    Apr 6, 2026Alhasan Mahmood, Samir Abdaljalil, Hasan KurbanMultilingual Language Model EvaluationLLM-as-a-Judge

    Hamad Bin Khalifa University, Doha, Qatar · Texas A&M University, College Station, TX USA

  18. OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments

    Apr 3, 2026Felix Henry, Lihan Lin, Jiangyou Zhu +4Mobile GUI AutomationGUI Agents

    XPeng Motors

  19. Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents

    Apr 1, 2026Alibek Kaliyev, Artem MaryanskyyAI Agent ReliabilityTool-Use Evaluation

    The University of Texas at Austin · Austin, Texas, USA · Uber +1

  20. SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

    Mar 31, 2026Kuangshi Ai, Haichao Miao, Kaiyuan Tang +13Scientific VisualizationData Visualization

    University of Notre Dame · Lawrence Livermore National Laboratory · University of Utah +4

  21. LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks

    Mar 20, 2026Xiang Long, Li Du, Yilong Xu +11Computer-Use Agent BenchmarksTool-Augmented Language Model Agents

    Samsung Research, Beijing, China · HKUST (Guangzhou) · City University of Hong Kong +1

  22. ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

    Mar 19, 2026Thomas De Min, Subhankar Roy, Stéphane Lathuilière +2Multimodal Large Language ModelsAI Agent Benchmarks

    University of Trento · University of Bergamo · Inria, Univ. Grenoble Alpes, CNRS, LJK +1

  23. ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs

    Mar 19, 2026Wanjia Zhao, Ludwig Schmidt, Yejin Choi +3LLM EvaluationAI Agent Benchmarks

    Stanford University · Microsoft Research

  24. RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments

    Mar 17, 2026Linghua Zhang, Jun Wang, Jingtong Wu +1Long-Horizon Agent EvaluationAI Agent Benchmarks

    Ant Group · City University of Hong Kong

  25. CCTU: A Benchmark for Tool Use under Complex Constraints

    Mar 16, 2026Junjie Ye, Guoqiang Zhang, Wenjie Fu +4Tool-Use EvaluationAI Agent Benchmarks

    College of Computer Science and Artificial Intelligence, Fudan University

  26. $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Mar 9, 2026Yang Liu, Jiaqi Li, Jun Bai +20LLM Agent EvaluationAI Agent Benchmarks

    BIGAI · Humanlaya · xbench +1

  27. Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

    Mar 2, 2026Haochuan WangLLM Inference EfficiencyLLM Agent Evaluation

    Massachusetts Institute of Technology

  28. HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosis

    Mar 1, 2026Shuo Tang, Jiadong Zhang, Gengxian Zhou +11Multi-Agent LLM SystemsAI Agent Benchmarks

    MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy, Beijing +3

  29. OmniGAIA: Towards Native Omni-Modal AI Agents

    Feb 26, 2026Xiaoxi Li, Wenxiang Jiao, Jiarui Jin +10Multimodal ReasoningAI Agent Benchmarks

    1Renmin University of China · 2Xiaohongshu Inc. · 3Peking University +2

  30. Agentic AI for Scalable and Robust Optical Systems Control

    Feb 23, 2026Zehao Wang, Mingzhe Han, Wei Cheng +12LLM Agent OrchestrationAI Agent Benchmarks

    Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA · NEC Laboratories America, Princeton, NJ 08540, USA · Duke Quantum Center and Department of Physics, Duke University, Durham, NC, USA 27708 +4