AI Agent Benchmarks

Ask about this topic

Momentum

209 papers in the last four weeks, up 61% on the four weeks before. 1.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,159

All topics
CardsList

  1. Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints

    Apr 17, 2026Xinge Liu, Terry Jingchen Zhang, Bernhard Schölkopf +2AI Agents for Scientific DiscoveryAI Agent Benchmarks

    University of Toronto · 2Vector Institute · 3Max Planck Institute for Intelligent Systems, Tübingen, Germany +1

  2. PolicyBank: Evolving Policy Understanding for LLM Agents

    Apr 16, 2026Jihye Choi, Jinsung Yoon, Long T. Le +2AI Agent BenchmarksAI Agent Governance

    Google Cloud · University of Wisconsin-Madison

  3. CoopEval: Benchmarking Cooperation-Sustaining Mechanisms and LLM Agents in Social Dilemmas

    Apr 16, 2026Emanuel Tewolde, Xiao Zhang, David Guzman Piedrahita +2Social DilemmasCooperative Game Theory

    Carnegie Mellon University · Foundations of Cooperative AI Lab (FOCAL) · Jinesis Lab, University of Toronto & Vector Institute +3

  4. PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research

    Apr 16, 2026Tingjia Miao, Wenkai Jin, Muhua Zhang +19LLM EvaluationAI Agent Benchmarks

    School of Artificial Intelligence, Shanghai Jiao Tong University · Zhiyuan College, Shanghai Jiao Tong University · School of Physics and Astronomy, Shanghai Jiao Tong University +7

  5. From Reactive to Proactive: Assessing the Proactivity of Voice Agents via ProVoice-Bench

    Apr 16, 2026Ke Xu, Yuhao Wang, Yu WangLLM Agent EvaluationAgent Evaluation

    Shanghai Jiao Tong University

  6. Benchmarks for Trajectory Safety Evaluation and Diagnosis in OpenClaw and Codex: ATBench-Claw and ATBench-Codex

    Apr 16, 2026Zhonghao Yang, Yu Li, Yanxu Zhu +6Benchmark DesignSoftware Engineering Benchmarks

    Shanghai Artificial Intelligence Laboratory

  7. MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror

    Apr 16, 2026Shengyu Guo, Tongrui Ye, Jianbo Zhang +3Multimodal Large Language ModelsAI Agent Benchmarks

    Shanghai AI Lab

  8. HWE-Bench: Benchmarking LLM Agents on Real-World Hardware Bug Repair Tasks

    Apr 16, 2026Fan Cui, Hongyuan Hou, Zizhang Luo +2Automated Program RepairElectronic Design Automation

    Peking University · Beijing, China

  9. DR3^{3}-Eval: Towards Realistic and Reproducible Deep Research Evaluation

    Apr 16, 2026Qianqian Xie, Qingheng Xiong, He Zhu +16LLM Agent EvaluationLong-Horizon Agent Evaluation

    Nanjing University · M-A-P · Nanjing University of Science and Technology +2

  10. Mind DeepResearch Technical Report

    Apr 16, 2026MindDR Team, Li Auto IncMulti-Agent LLM SystemsDeep Research Agents

    MindDR Team, Li Auto Inc

  11. HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

    Apr 15, 2026Jiacheng Wang, Jinchang Hou, Fabian Wang +3AI Agent AuditingLong-Horizon Agent Evaluation

    Baidu Inc., Beijing, China · Beijing Institute of Technology, Beijing, China

  12. UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents

    Apr 13, 2026Yijuan Liang, Xinghao Chen, Yifan Ge +5LLM Agent EvaluationAI Agent Benchmarks

    University of Science and Technology of China · Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology, Ningbo · Department of Computing, The Hong Kong Polytechnic University

  13. NetForge RL: A Multi-Agent Simulation Environment for Cyber Defense with Durative Actions

    Apr 10, 2026Igor JankowskiAutonomous Cyber DefenseCybersecurity

  14. Riemann-Bench: A Benchmark for Moonshot Mathematics

    Apr 8, 2026Suhaas Garre, Erik Knutsen, Sushant Mehta +1Mathematical Reasoning BenchmarksBenchmark Design

    Surge AI

  15. Multilingual Prompt Localization for Agent-as-a-Judge: Language and Backbone Sensitivity in Requirement-Level Evaluation

    Apr 6, 2026Alhasan Mahmood, Samir Abdaljalil, Hasan KurbanMultilingual Language Model EvaluationLLM-as-a-Judge

    Hamad Bin Khalifa University, Doha, Qatar · Texas A&M University, College Station, TX USA

  16. OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments

    Apr 3, 2026Felix Henry, Lihan Lin, Jiangyou Zhu +4Mobile GUI AutomationGUI Agents

    XPeng Motors

  17. Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents

    Apr 1, 2026Alibek Kaliyev, Artem MaryanskyyAI Agent ReliabilityTool-Use Evaluation

    The University of Texas at Austin · Austin, Texas, USA · Uber +1

  18. SciVisAgentBench: A Benchmark for Evaluating Scientific Data Analysis and Visualization Agents

    Mar 31, 2026Kuangshi Ai, Haichao Miao, Kaiyuan Tang +13Scientific VisualizationData Visualization

    University of Notre Dame · Lawrence Livermore National Laboratory · University of Utah +4

  19. LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks

    Mar 20, 2026Xiang Long, Li Du, Yilong Xu +11Computer-Use Agent BenchmarksTool-Augmented Language Model Agents

    Samsung Research, Beijing, China · HKUST (Guangzhou) · City University of Hong Kong +1

  20. ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

    Mar 19, 2026Thomas De Min, Subhankar Roy, Stéphane Lathuilière +2Multimodal Large Language ModelsAI Agent Benchmarks

    University of Trento · University of Bergamo · Inria, Univ. Grenoble Alpes, CNRS, LJK +1

  21. ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs

    Mar 19, 2026Wanjia Zhao, Ludwig Schmidt, Yejin Choi +3LLM EvaluationAI Agent Benchmarks

    Stanford University · Microsoft Research

  22. RetailBench: Evaluating Long-Horizon Autonomous Decision-Making and Strategy Stability of LLM Agents in Realistic Retail Environments

    Mar 17, 2026Linghua Zhang, Jun Wang, Jingtong Wu +1Long-Horizon Agent EvaluationAI Agent Benchmarks

    Ant Group · City University of Hong Kong

  23. CCTU: A Benchmark for Tool Use under Complex Constraints

    Mar 16, 2026Junjie Ye, Guoqiang Zhang, Wenjie Fu +4Tool-Use EvaluationAI Agent Benchmarks

    College of Computer Science and Artificial Intelligence, Fudan University

  24. $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Mar 9, 2026Yang Liu, Jiaqi Li, Jun Bai +20LLM Agent EvaluationAI Agent Benchmarks

    BIGAI · Humanlaya · xbench +1

  25. Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

    Mar 2, 2026Haochuan WangLLM Inference EfficiencyLLM Agent Evaluation

    Massachusetts Institute of Technology

  26. HVR-Met: A Hypothesis-Verification-Replanning Agentic System for Extreme Weather Diagnosis

    Mar 1, 2026Shuo Tang, Jiadong Zhang, Gengxian Zhou +11Multi-Agent LLM SystemsAI Agent Benchmarks

    MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy, Beijing +3

  27. OmniGAIA: Towards Native Omni-Modal AI Agents

    Feb 26, 2026Xiaoxi Li, Wenxiang Jiao, Jiarui Jin +10Multimodal ReasoningAI Agent Benchmarks

    1Renmin University of China · 2Xiaohongshu Inc. · 3Peking University +2

  28. Agentic AI for Scalable and Robust Optical Systems Control

    Feb 23, 2026Zehao Wang, Mingzhe Han, Wei Cheng +12LLM Agent OrchestrationAI Agent Benchmarks

    Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708, USA · NEC Laboratories America, Princeton, NJ 08540, USA · Duke Quantum Center and Department of Physics, Duke University, Durham, NC, USA 27708 +4

  29. AgenticRAGTracer: A Hop-Aware Benchmark for Diagnosing Multi-Step Retrieval Reasoning in Agentic RAG

    Feb 22, 2026Qijie You, Wenkai Yu, Wentao ZhangMulti-Hop QALLM Agent Evaluation

    University of Science and Technology Beijing · 2Peking University · 3Zhongguancun Academy +1

  30. Evaluating Test-Time Scaling of General LLM Agents

    Feb 22, 2026Xiaochuan Li, Ryan Ming, Pranav Setlur +6LLM Agent EvaluationTest-Time Scaling

    Language Technologies Institute, School of Computer Science, Carnegie Mellon University · Meta