AI Agent Benchmarks

Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,013

All topics
CardsList

  1. WhatWorkedBench: Benchmarking Experimental Understanding in AI Agents

    Sep 23, 2026Jingjie Ning, Xueqi Li, Yibo Kong +1AI Agent EvaluationActive Learning

    Carnegie Mellon University · Tsinghua University

  2. MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

    Sep 23, 2026Yongjun Jeong, Hanbum Ko, Ye Rin Kim +6Molecular OptimizationLLM Agent Evaluation

    Department of Artificial Intelligence, Korea University · LG AI Research · Department of Statistics, Korea University

  3. CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments

    Sep 23, 2026Yuxuan Li, Will Epperson, Wesley Deng +1Computer-Use AgentsAgent Harness Optimization

    Carnegie Mellon University · Microsoft Research

  4. SWE-Serve: Benchmarking Agentic Engineering For Production Inference Serving

    Sep 22, 2026Jennifer Williams, Dave Farris, Jeff Farris +1Software EngineeringLLM Inference Serving

    NVIDIA · University of California, Berkeley

  5. The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

    Sep 22, 2026Wenbo Pan, Zhichao Liu, Shujie Liu +6Software Engineering BenchmarksLong-Horizon Agent Evaluation

    City University of Hong Kong · Microsoft · Independent Researcher

  6. ShowTellArena: Evaluating Business Workflow Understanding from Demonstrations

    Sep 21, 2026David Garg, Ritobrata Sarkar, Ehsan Azarnasab +1AI Agent EvaluationAI Agent Benchmarks

    Brackett Labs

  7. Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development

    Sep 21, 2026Haocheng Xia, Eugene Wu, Yongjoo ParkLLM Agent EvaluationAI Agent Benchmarks

    University of Illinois Urbana-Champaign Champaign, USA · Columbia University New York, USA

  8. The AI Neuroscientist: An Interactive Agentic Interface for Neuroimaging Analysis

    Sep 21, 2026Aakash Patel, Panos Ketonis, Shreya Saxena +2AI Agents for Scientific DiscoveryLLM Agent Evaluation

    Department of Computer Science, Yale University · Wu Tsai Institute, Yale University · Department of Biomedical Engineering, Yale University +2

  9. Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers

    Sep 21, 2026Weihang Ding, Junfei ZhanLLM Agent EvaluationAI Agent Benchmarks

    Department of Industrial Engineering and Operations Research University of California, Berkeley · Department of Computing Imperial College London, United Kingdom

  10. FinFIRST: Benchmarking Search Agents for Financial Information Retrieval, Sourcing and Traceability

    Sep 21, 2026Wenqing Wang, Haitao Xiang, Xinyi Zhao +8Financial QAAI Agent Benchmarks

    Ling Team, Inclusion AI · Peking University · China International Capital Corporation Limited

  11. MSI-Bench: Evaluating Multi-Speaker Voice Interaction for Collaborative AI Agents

    Sep 21, 2026Chenxu Xiong, Dongming Shen, Yuzhi Tang +3Audio-Language Model EvaluationVoice Agent Evaluation

    Boson AI Santa Clara, CA 95054

  12. MCP-GRANITE Benchmark: GRANularity Interface TEsting for MCP-Based LLM Agents

    Sep 21, 2026Demetris Paschalides, Moysis Symeonides, George Pallis +1AI Agent BenchmarksTool-Using Agents

    Department of Computer Science University of Cyprus Nicosia, Cyprus

  13. XYEval: Agents say yes to bad advice

    Sep 20, 2026Zhengxuan Wu, Yuxuan Li, Oyvind Tafjord +1LLM SycophancyAI Agent Evaluation

    Google DeepMind

  14. BabelArena: A Large-Scale Multilingual Benchmark for LLM Agents

    Sep 20, 2026Peng Kuang, Yuchun Fan, Jiangnan Li +7Multilingual Language Model EvaluationLLM Agent Evaluation

    Alibaba Token Hub, Alibaba Group · University of Illinois Urbana-Champaign · Northeastern University +2

  15. MTVA-Bench: Evaluating the Language Model Inside Cascaded Voice Agents

    Sep 17, 2026Pritish Mishra, Ishaan Kumar, Akshat Mandoli +1Voice Agent EvaluationMultilingual Language Model Evaluation

    Smallest AI

  16. F2^{2}DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows

    Sep 17, 2026Bojian Xiong, Wentao Ding, Yujing Lu +11Reward ModelingLLM Evaluation

    Tianjin University · TJUNLP Lab, Tianjin University, Tianjin, China · Baidu Inc. +1

  17. SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership

    Sep 17, 2026Run Peng, Zinnia Nie, Jing Ding +7Human Behavior SimulationLong-Horizon Agent Evaluation

    University of Michigan · ByteDance · Osaka University +1

  18. Beyond Outcomes: Dual-View Relational Learning for Efficient Agent Benchmarking

    Sep 16, 2026Xinshuai Guo, Junjie Wu, Dolly Deng +4AI Agent EvaluationAI Agent Benchmarks

    Hunyuan Team, Tencent · Tsinghua University

  19. ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts

    Sep 16, 2026Liyang Fan, Chi Wei, Yitai Li +8Scientific VisualizationAI Agent Benchmarks

    SZU · SIAT, CAS · SUAT +2

  20. Clueing up LLMs with Tool-Augmented Deductive Reasoning

    Sep 16, 2026Rebecca Ansell, Autumn Toney-WailsMulti-Agent LLM SystemsAI Agent Benchmarks

    Georgetown University · Syntheos, Corp · UNU-MERIT

  21. AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines

    Sep 16, 2026Li ChenGPU Kernel OptimizationBenchmark Design

  22. RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents

    Sep 16, 2026Qingnuan Han, Boli Fang, Mingzhi Hou +1Tool-Use EvaluationAI Agent Benchmarks

    Didi Global

  23. Locating Hidden Failures Makes Long-Horizon Agents More Reliable

    Sep 15, 2026Salman Rahman, Yubin Kim, Mihir Parmar +15Agent Failure AnalysisAI Agent Reliability

    University of California, Los Angeles · New York University · Google Research +1

  24. GraphEcho: Structural Redundancy and Evidence Provenance in LLM Graph Agents

    Sep 15, 2026Sikun Wang, Yixi Zhou, Lei Fan +1LLM Reasoning with GraphsLLM Agent Evaluation

    Tokyo University of Science · Hong Kong Baptist University · University of Michigan +1

  25. ToMAS: A Pilot Failure-Grounded Theory-of-Mind Benchmark from Multi-Agent LLM Failures

    Sep 15, 2026Muhammad Ashar Ishfaq, Glaucia MeloTheory of MindLLM Agent Evaluation

    The Islamia University of Bahawalpur, Pakistan · Toronto Metropolitan University, Toronto, Canada

  26. Atria Dawn: The Dawn of Agentic Superintelligence

    Sep 14, 2026Honglin Guo, Tao Gui, Kun Cai +140AI Agents for Scientific DiscoveryAI Agent Benchmarks

  27. V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments

    Sep 14, 2026Ziqian Fan, Shibo Xu, Junjie Li +8Multimodal ICLAI Agent Benchmarks

    South China University of Technology · Shanghai Jiao Tong University · Fudan University +4

  28. BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

    Sep 14, 2026Yolo Y. Tang, Daiki Shimada, Jiayue Meng +14AI Agent BenchmarksTemporal Video Understanding

    University of Rochester · Sony Group Corporation · Carnegie Mellon University +1

  29. InterSocialBench: Benchmarking Human and LLM Preferences for Companion-Robot Social Behavior

    Sep 14, 2026Yaodan Xu, Boyang Guo, Yuqing Gu +4Human Preference ModelingAI Agent Benchmarks

    Xdream Robotics

  30. When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis

    Sep 14, 2026Kaiyuan Liu, Qiuyang Mang, Bo Peng +6LLM Agent EvaluationTest-Time Scaling

    UC Berkeley · University of Washington · Princeton University +1