AI Agent Benchmarks

Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,013

All topics
CardsList

  1. RECON: Benchmarking Agent Memory for Compositional Reasoning over Long Contexts

    Jul 18, 2026Mihir Shriniwas AryaCompositional ReasoningLLM Agent Memory

    Department of Computer Science and Engineering RV College of Engineering

  2. CLOSER-Bench: Evaluating Budgeted Cross-Stage Design Closure for Hardware Agents

    Jul 18, 2026Peilong Zhou, Zhirong Chen, Cangyuan Li +4Benchmark DesignElectronic Design Automation

    1SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences

  3. Just A Rather Very Intelligent Spoken Agent

    Jul 18, 2026Chen Chen, Zhehuai ChenLong-Horizon Agent EvaluationHuman-AI Interaction

    NVIDIA

  4. Interactive Task Alignment as a POMDP

    Jul 17, 2026Andy Dai, Zexue He, Zhenyu Zhang +2Pragmatic Reasoning in Language ModelsAI Agent Benchmarks

    1Stanford University

  5. Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents

    Jul 17, 2026Zedong Yu, Qianxing Li, Zhi Gao +8Computer-Use Agent BenchmarksGUI Agents

    Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China · China University of Geosciences (Beijing), Beijing, China +1

  6. PersonaMem-v3: Toward Omni-Platform Personal Intelligence for Holistic User Understanding, Recommendation, and Agentic Tasks

    Jul 16, 2026Bowen Jiang, Yuan Yuan, Zhuoqun Hao +12Personal AI AgentsLLM Personalization

    University of Pennsylvania · Meta Recommendation Systems · MIT

  7. BrainPilot: Automating Brain Discovery with Agentic Research

    Jul 16, 2026Haoxuan Li, Tianci Gao, Jianhe Li +13AI for ScienceLLM Agent Orchestration

    College of AI, Tsinghua University · 2Shanghai Qizhi Institute · 3Business School, Renmin University of China +8

  8. OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios

    Jul 16, 2026Chengyu Shen, Yujie Fu, Gangtao Xin +13Computer-Use Agent BenchmarksAI Agent Evaluation

    Huawei Cloud Post-Training Team

  9. StructureClaw: Traceable LLM Agents and an Executable Benchmark for Structural Engineering Workflows

    Jul 16, 2026Sizhong Qin, Yi Gu, Yao Jiang +13LLM Agent EvaluationAI Agent Benchmarks

  10. MCPEvol-Bench: Benchmarking LLM Agent Performance Across Dynamic Evolutions of MCP Servers

    Jul 16, 2026Huanxi Liu, Kun Hu, Jiaqi Liao +6Tool-Use EvaluationAI Agent Benchmarks

    College of Computer Science and Technology, National University of Defense Technology · State Key Laboratory of Complex & Critical Software Environment · National Key Laboratory of Parallel and Distributed Computing

  11. DeepStress: Stress-Testing Deep Search Agents

    Jul 15, 2026Ismael Rousseau, Geraldine Damnati, Frederic BechetAI Agent ReliabilityAI Agent Evaluation

    Orange Research, Lannion, France · Aix-Marseille Univ., CNRS, LIS UMR 7020, 13000 Marseille, France

  12. EgoProceVQA: A Novel Egocentric Procedural Understanding Task with Self-Skill-Exploration Agent

    Jul 15, 2026Junlong Li, Junxi Li, Yuxiang Yang +3Egocentric Video QAVideo QA

    The Hong Kong Polytechnic University Hong Kong, China

  13. AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities

    Jul 15, 2026Kai Chen, Zichen Ding, Jiaye Ge +20LLM Agent EvaluationAI Agent Benchmarks

    AgentCompass Team, Shanghai AI Laboratory

  14. DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments

    Jul 15, 2026Huatao Li, Xinwei Geng, Yuheng Wang +9Computer-Use Agent BenchmarksLLM Agent Evaluation

    School of Artificial Intelligence, Shanghai Jiao Tong University · 2Honor Device Co., Ltd

  15. Set-shifting Behavioral Test for Harnessed Agents

    Jul 15, 2026Ziwei YeLLM Agent EvaluationAI Agent Benchmarks

  16. Fin-Analyst at FinMMEval 2026 Task 3: A Live Hybrid Trading Agent with LLM Specialists and Rule-Based Signals

    Jul 14, 2026Mohotarema Rashid, Lingzi Hong, Junhua Ding +1Multi-Agent LLM SystemsLLM Agent Evaluation

    Department of Information Science, University of North Texas, Denton, TX, United States · Department of Data Science, University of North Texas, Denton, TX, United States

  17. RCWT: Measuring Task-Budget Displacement from Coordination Content in LLM Calls

    Jul 13, 2026Brenda Lelis, Rodrigo Cabral-CarvalhoMulti-Agent CoordinationAI Agent Benchmarks

    CloudWalk, Inc. · São Paulo, Brazil

  18. The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomy

    Jul 13, 2026Chunzheng Zhu, Lei Tian, Bohan Tan +15HealthcareLLM Agent Self-Improvement

    1Hunan University · 2ByteDance · 3Duke University +6

  19. The Hidden Footprint: Making Storage a First-Class Metric for LLM Agent Evaluation

    Jul 13, 2026Chenglin Yu, Hongquan Gui, Ying Yu +3LLM Agent EvaluationAI Agent Benchmarks

    Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University · College of Economics and Management, Zhejiang Normal University · Department of Computing, The Hong Kong Polytechnic University +2

  20. NextFund: A Unified Performance Tracking Platform for Agentic Portfolio Management

    Jul 13, 2026Changlun Li, Peixian Ma, Qiqi Duan +2Quantitative FinanceLLM Agent Evaluation

    Paradoox AI Research

  21. BackendForge: Benchmarking Agentic End-to-End Code Generation with Backend Services

    Jul 13, 2026Yuzhe Guo, Mengzhou Wu, Yuan Cao +4Tool-Augmented Language Model AgentsLanguage Model Generation Evaluation

    1Key Lab of HCST (PKU), MOE; SCS, Peking University · 2Beijing Tongming Lake Information Technology Application Innovation Center (TLAIC) · 3Fudan University Institute of Systems for Advanced Computing +1

  22. Opti-Agent-Bench: Benchmarking End-to-End Optimization R&D Agents on Real-World Business Problems

    Jul 12, 2026Yongchang Fu, Xinjie Huang, Chengjun Dai +3Large Language Model-Guided OptimizationLLM Agent Evaluation

    Ding Talk, Alibaba Group, Hangzhou, China · Zhejiang University, Hangzhou, China

  23. MafiaScope: Non-Invasive, Time-Resolved Belief Probing for LLM Agents in Social Deduction Games

    Jul 12, 2026Ilia KarpovSocial Deduction GamesTheory of Mind

    HSE University

  24. The Compliance Trap: Diagnosing How AI Agents Consume Conflicting Memory

    Jul 12, 2026Yixiong Chen, Xinyi Bai, Alan YuilleWeb Agent BenchmarksLong-Horizon Agent Evaluation

    1Johns Hopkins University · 2Cornell University

  25. UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp

    Jul 12, 2026Xiyu Wei, Qingwei Zong, Zhuocheng Yu +1Multimodal QAAI Agent Benchmarks

    Key Laboratory of Computational Linguistics, MOE, Peking University · School of Software and Microelectronics, Peking University · School of Computer Science, Peking University

  26. RideGym: A Standardized Interface for Real-World Large-Scale Ride-Sharing System

    Jul 11, 2026Zijian Zhao, Yulong Hu, Sen LiVehicle RoutingAI Agent Benchmarks

    The Hong Kong University of Science and Technology Hong Kong, China

  27. Quantum Circuit Vision: Cost-Aware Evaluation of Visual AI Agents for Quantum Code Generation

    Jul 11, 2026Dongping Liu, Aoyu Zhang, Luyao ZhangAI Agent EvaluationAI Agent Benchmarks

    Amazon Web Services Hong Kong, China · Amazon Web Services Beijing, China · Digital Innovation Research Center and Social Science Division, Duke Kunshan University Address: Duke Avenue No.8, Kunshan, Suzhou, Jiangsu, China, 215316

  28. Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

    Jul 10, 2026Jiale Liu, Huajun Xi, Shaokun Zhang +6Agent Failure AnalysisLLM Agent Evaluation

    Penn State University · AG2ai, Inc. · Mohamed bin Zayed University of Artificial Intelligence +2

  29. DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers

    Jul 10, 2026Jerzy Kamiński, Ilya Galyukshev, Artem Kuznetsov +4Computer-Use Agent BenchmarksLLM Agent Evaluation

    ITMO University · Central University

  30. Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading

    Jul 9, 2026Zongxia Li, Zhongzhi Li, Yucheng Shi +10Long-Horizon Agent EvaluationAI Agent Benchmarks

    Tencent HY LLM Frontier · University of Maryland, College Park · University of Georgia +5