AI Agent Benchmarks

Ask about this topic

Momentum

209 papers in the last four weeks, up 61% on the four weeks before. 1.4% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,159

All topics
CardsList

  1. Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows

    Apr 30, 2026Chenxin Li, Zhengyang Tang, Mingxin Huang +8LLM Agent EvaluationAI Agent Benchmarks

    1The Chinese University of Hong Kong · 2The Chinese University of Hong Kong, Shenzhen · 3South China University of Technology +4

  2. What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design

    Apr 30, 2026Ivan BercovichComputer-Use Agent BenchmarksReward Hacking

  3. D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery

    Apr 30, 2026Hanane Nour Moussa, Yifei Li, Zhuoyang Li +7AI Agents for Scientific DiscoveryAI Agent Benchmarks

    1The Ohio State University · 2Cisco Research

  4. KellyBench: A Benchmark for Long-Horizon Sequential Decision Making

    Apr 30, 2026Thomas Grady, Kip Parker, Iliyan Zarov +3Sequential Decision MakingLong-Horizon Agent Evaluation

    General Reasoning, Inc.

  5. WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments

    Apr 30, 2026Jinchao Li, Yunxin Li, Chenrui Zhao +3Computer-Use Agent BenchmarksComputer-Use Agents

    Harbin Institute of Technology, Shenzhen · Shenzhen Loop Area Institute

  6. RoadMapper: A Multi-Agent System for Roadmap Generation of Solving Complex Research Problems

    Apr 30, 2026Jiacheng Liu, Zichen Tang, Zhongjun Yang +8Multi-Agent LLM SystemsLLM Planning

    Beijing University of Posts and Telecommunications · The Hong Kong University of Science and Technology (Guangzhou) · IDEA Research +1

  7. ClawGym: A Scalable Framework for Building Effective Claw Agents

    Apr 29, 2026Fei Bai, Huatong Song, Shuang Sun +11Synthetic Data GenerationAI Agent Benchmarks

    Gaoling School of Artificial Intelligence, Renmin University of China · IQuest Research · Beihang University

  8. StratMem-Bench: Evaluating Strategic Memory Use in Virtual Character Conversation Beyond Factual Recall

    Apr 29, 2026Yerong Wu, Tianxing Wu, Minghao Zhu +2Conversational MemoryAI Agent Benchmarks

    School of Computer Science and Engineering, Southeast University, China · 2Key Laboratory of New Generation Artificial Intelligence Technology and its Interdisciplinary Applications (Southeast University), Ministry of Education, China · 3Independent Researcher +1

  9. LATTICE: Evaluating Decision Support Utility of Crypto Agents

    Apr 29, 2026Aaron Chan, Tengfei Li, Tianyi Xiao +3AI-Assisted Decision MakingAI Agent Benchmarks

    Sahara AI · University of Southern California

  10. DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios

    Apr 28, 2026Jinxiang Meng, Shaoping Huang, Fangyu Lei +17Data VisualizationAI Agent Benchmarks

    Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · National University of Singapore +1

  11. AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery

    Apr 28, 2026Lei Xiong, Kun Luo, Ziyi Xia +15Deep Research AgentsScientific Discovery

    AutoResearchBench Team*

  12. ValueBlindBench: Agreement-Gated Stress Testing of LLM-Judged Investment Rationales Before Returns Are Observable

    Apr 28, 2026Sidi Chang, Peiying Zhu, Yuxiao ChenLLM-as-a-JudgeAI Agent Evaluation

    Blossom AI Labs, Tokyo, Japan · Blossom AI, San Francisco, USA

  13. Cooperate to Compete: Strategic Coordination in Multi-Agent Conquest

    Apr 28, 2026Abigail O'Neill, Alan Zhu, Mihran Miroyan +2Multi-Agent CoordinationMulti-Agent Negotiation

    University of California, Berkeley

  14. Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver

    Apr 27, 2026Joshua Sherwood, Ben Aybar, Benjamin KaplanAI Coding AgentsAI Agent Evaluation

    Department of Computer Science University of Chicago Chicago, IL 60637 · Independent Researcher

  15. BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks

    Apr 27, 2026Xinming Tu, Tianze Wang, Yingzhou +4Benchmark AuditingAI Agent Benchmarks

    Minta · Allen School, University of Washington, Seattle, WA, USA · Phylo, Inc., South San Francisco, CA, USA +1

  16. GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation

    Apr 27, 2026Yunsu Kim, Kaden Uhlig, Joern WuebkerAI Agent EvaluationAI Agent Benchmarks

    LILT

  17. Can Current Agents Close the Discovery-to-Application Gap? A Case Study in Minecraft

    Apr 27, 2026Zhou Ziheng, Huacong Tang, Jinyuan Zhang +9AI Agents for Scientific DiscoveryCircuit Discovery

    University of California, Los Angeles · Peking University · Amazon

  18. NeuroClaw Technical Report

    Apr 27, 2026Cheng Wang, Zhibin He, Zhihao Peng +7Multi-Agent LLM SystemsAI Agent Benchmarks

    Electronic Engineering Department, The Chinese University of Hong Kong, China · School of Electronic Information, Northwest University, China · Department of Computer Science, Emory University, Atlanta, GA, USA +3

  19. Skill Retrieval Augmentation for Agentic AI

    Apr 27, 2026Weihang Su, Jianming Long, Qingyao Ai +6AI Agent BenchmarksLLM Agent Skill Retrieval

    Department of Computer Science and Technology, Tsinghua University · ByteDance Inc.

  20. AutoGUI-v2: A Comprehensive Multi-Modal GUI Functionality Understanding Benchmark

    Apr 27, 2026Hongxin Li, Xiping Wang, Jingran Su +4VLM EvaluationGUI Agents

    University of Chinese Academy of Sciences (UCAS) · 2New Laboratory of Pattern Recognition (NLPR), CASIA · 3State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), CASIA +3

  21. OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents

    Apr 27, 2026Zheng Wu, Yi Hua, Zhaoyuan Huang +8Computer-Use AgentsAI Agent Evaluation

    School of Computer Science, Shanghai Jiao Tong University

  22. AgentPulse: A Continuous Multi-Signal Framework for Evaluating AI Agents in Deployment

    Apr 27, 2026Yuxuan Gao, Megan Wang, Yi Ling YuAI Agent EvaluationAI Agent Benchmarks

    University of Pennsylvania · 2Columbia University · 3OpenMesh AI

  23. MarketBench: Evaluating AI Agents as Market Participants

    Apr 26, 2026Andrey Fradkin, Rohit KrishnanMulti-Agent CoordinationLLM Agent Evaluation

    Boston University and the MIT Initiative on the Digital Economy. · Independent Researcher.

  24. ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents

    Apr 26, 2026Fanqing Meng, Lingxiao Du, Zijian Wu +46Computer-Use Agent BenchmarksLLM Agent Evaluation

  25. SoccerRef-Agents: Multi-Agent System for Automated Soccer Refereeing

    Apr 25, 2026Zi Meng, Wanli Song, Yi Hu +2Multi-Agent LLM SystemsSports Video Analysis

    University of Michigan, Michigan, USA · Shanghai Jiao Tong University, Shanghai, China

  26. Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems

    Apr 24, 2026Mengzhuo Chen, Junjie Wang, Fangwen Mu +4Agent Failure AnalysisMulti-Agent LLM Systems

    State Key Laboratory of Complex System Modeling and Simulation Technology · Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

  27. Superminds Test: Actively Evaluating Collective Intelligence of Agent Society via Probing Agents

    Apr 24, 2026Xirui Li, Ming Li, Yunze Xiao +4Multi-Agent CoordinationAI Agent Evaluation

    University of Maryland · 2Mohamed bin Zayed University of Artificial Intelligence · 3Carnegie Mellon University

  28. AgentSearchBench: A Benchmark for AI Agent Search in the Wild

    Apr 24, 2026Bin Wu, Arastun Mammadli, Xiaoyu Zhang +1Agentic SearchAI Agent Evaluation

    Centre for Artificial Intelligence, University College London

  29. When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

    Apr 23, 2026Chenghao Yang, Yuning Zhang, Zhoufutu Wen +4Tool-Augmented Language Model AgentsLLM Agent Evaluation

    School of Cyber Science and Technology, USTC · Anhui Province Key Laboratory of Digital Security · 3M-A-P +1

  30. How VLAs (Really) Work In Open-World Environments

    Apr 23, 2026Amir Rasouli, Yangzheng Wu, Zhiyuan Li +4VLM RobustnessLong-Horizon Robotic Manipulation

    Noahs Ark Laboratory, Huawei Technologies Canada. · while at Huawei Canada.