AI Agent Benchmarks

Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 1,013

All topics
CardsList

  1. Automated Transcript Analysis for Detecting Flaws in Agentic Benchmarks

    Jul 29, 2026Jeff Mohl, Nelson Gardner-Challis, Magda Dubois +6Benchmark AuditingAI Agent Benchmarks

    Independent · UK AI Security Institute · MARS +4

  2. Skill Use or Skill Theater? Evaluating the Reasoning Backroom in Skill-Augmented Language Agents

    Jul 29, 2026Jinwei Hu, Yi Qi, Xinmiao Huang +3LLM Agent EvaluationAI Agent Benchmarks

    School of Computer Science and Informatics, University of Liverpool, UK · Department of Computer Science, University of Leeds, UK · Department of Computer Science, Mohamed bin Zayed University of Artificial Intelligence, UAE

  3. VAmoS Bench: Voice Agent Simulation Bench

    Jul 29, 2026Joshua Meyer, Sahar Shayegan, Ritiz Tambi +5Voice Agent EvaluationCustomer Support Automation

    Veris AI

  4. PAUSE: A User-Centric Benchmark for Personal AI Assistants in Unified Service Environments

    Jul 29, 2026Haoyu Chen, Xirui Shi, Yuyao Wang +2Long-Horizon Agent EvaluationAI Agent Benchmarks

    Electrical and Computer Engineering

  5. LayerRAG-Bench: A Cross-Layer Reliability Benchmark for Agentic Retrieval-Augmented Generation

    Jul 29, 2026Musa ShamsAI Agent ReliabilityRetrieval-Augmented Generation

    Independent Researcher

  6. APEX-Accounting

    Jul 29, 2026Julien Benchek, Austin Bennett, Jasmin Kern +8LLM EvaluationBusiness Process Automation

    1Mercor · 2Ramp

  7. Benchmarking the Residual: What Long-Horizon Evaluations Add Beyond Matched Short-Task Performance

    Jul 29, 2026Chao Peng, Zhiheng Lyu, Peijie Dong +2Long-Horizon Agent EvaluationAI Agent Benchmarks

    Tencent

  8. Flat Score, Amplified Failures: How the Error Budget Masks Damage in Quantized LLM Agents

    Jul 29, 2026Jiwon Jang, Kisu Yang, Heuiseok Lim +1AI Agent ReliabilityBenchmark Design

    VAIV Company · Seoul National University · Korea University

  9. A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities

    Jul 29, 2026Wenhao Yang, Runzhi He, Minghui ZhouAI Coding AgentsAI Agent Benchmarks

    Peking University, Beijing, China

  10. UrbanDS: A Graph-Guided LLM Multi-Agent System for Data-Intensive Urban Tasks

    Jul 29, 2026Zhilun Zhou, Jianghao Yu, Yuming Lin +4Multi-Agent LLM SystemsAI Agent Benchmarks

  11. SciDataSailor: Deep Scientific Data Exploring

    Jul 29, 2026Jiyong Rao, Yicheng Qiu, Chi Zhang +2Scientific QAAI Agent Benchmarks

    Shanghai Artificial Intelligence Laboratory · Tongji University

  12. Exploring Structures in Physics Problems: Can AI Agents Discover Statistical Mechanical Mappings?

    Jul 29, 2026Wanyu Zhao, Wanbing ZhaoAI Agents for Scientific DiscoveryLLM Agent Evaluation

    Department of Physics and Astronomy Rice University

  13. ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science

    Jul 28, 2026Yuan Zhu, Ethan B. Liu, Frank Nie +1Long-Horizon Agent EvaluationAI Agent Benchmarks

    Shandong University, China

  14. Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation

    Jul 28, 2026Stefan Krsteski, Charlotte Meyer, Guillaume Allegre +2AI Agent EvaluationAgent Evaluation

    Andromede AI

  15. RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement

    Jul 28, 2026Fanqing Meng, Lingxiao Du, Qiguang Chen +4LLM Agent EvaluationLLM Agent Self-Improvement

    1Evolvent AI · 2National University of Singapore

  16. WorkSurface-Bench: Benchmarking Enterprise Agents on Multi-Surface Knowledge Routing

    Jul 28, 2026Hao Liang, Meiyi Qiang, Sizhe Qiu +2Computer-Use Agent BenchmarksAI Agent Benchmarks

    Peking University · Zhongguancun Academy · University of Chinese Academy of Sciences

  17. OrchBench: Evaluating Multi-Agent Orchestration Plans in Isolation via Deterministic Simulation

    Jul 28, 2026Zhenzhen Ren, Jiyan He, Xinpeng Zhang +5Multi-Agent CoordinationMulti-Agent Orchestration

    1Fudan University · 2Zhongguancun Academy · 3Queen Mary University of London

  18. PatientAgentBench: A Benchmark Framework for Evaluating Patient-Facing Health AI Agents

    Jul 28, 2026Korosh Vatanparvar, Ashutosh Joshi, Maria Xenochristou +11AI Agent EvaluationAI Agent Safety

    Amazon Health AI

  19. HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following

    Jul 28, 2026Liudas Panavas, Sebastian Minus, Bradley Monton +4Computer-Use Agent BenchmarksAI Agent Reliability

    Surge AI*

  20. Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments

    Jul 27, 2026Takuya Isogawa, Ryotaro Okabe, Nutdech Phadetsuwannukun +2AI Agent BenchmarksLaboratory Automation

    Department of Nuclear Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Chemistry, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Physics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA

  21. SIREN: Towards End-to-End Extreme-Weather Early Warning with Experience-Grounded LLM Agents

    Jul 27, 2026Hang Ni, Weijia Zhang, Fan Liu +2AI Agent BenchmarksLLM Agents

  22. GAUGE: Grading Agent-Built Financial Models Without a Golden Answer

    Jul 27, 2026Jiacheng Lu, Sinuo Wang, Wentao Zhao +12Quantitative FinanceFinancial Forecasting

    Shanghai Jiao Tong University Shanghai, China · StepFun Shanghai, China · University of Adelaide Adelaide, Australia +5

  23. Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents

    Jul 27, 2026Bowen Qin, Yi XieSoftware Engineering AgentsAI Agent Benchmarks

    1National University of Singapore (NUS) · 2Peking University

  24. The Cost of Knowing: A Resource-Aware Protocol for Benchmarking Hallucination Beyond Static Leaderboards

    Jul 27, 2026Keyu Li, Jin Gao, Dequan WangFactual Consistency EvaluationAI Agent Benchmarks

    Shanghai Jiao Tong University Shanghai, China

  25. Success Is Not Self-Explanatory: Auditing Success Provenance in Agent Evaluation

    Jul 27, 2026Jingkun Luo, Da-Tian PengData ProvenanceBenchmark Auditing

    Northwestern Polytechnical University

  26. E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios

    Jul 26, 2026Weihuang Zheng, Tianyuan Zou, Eileen Ye +5Computer-Use Agent BenchmarksTool-Use Evaluation

    Hunyuan Team, Tencent · Institute for AI Industry Research, Tsinghua University · School of Computer Science and Engineering, Southeast University

  27. DualityCert: Verifier-Gated Language-Model Repair of Broken Duality Claims in Quantum Field Theory

    Jul 26, 2026Xingyang YuLLM Self-CorrectionClaim Verification

    Department of Physics Virginia Tech Blacksburg, VA 24061, USA

  28. Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis

    Jul 26, 2026Xinhao Yao, Yuanzhuo Liu, Changhao Wang +6Deep Research AgentsLLM Agent Evaluation

    Tencent · Renmin University of China · UCAS +1

  29. AllocBench: Measuring Online Tool Allocation Capability in LLM Agents

    Jul 25, 2026Daniel Wang, Andrew XuTool-Augmented Language Model AgentsAI Coding Agents

    Yale University

  30. AgentOmnia: Scaling Agentic Models for Full-Scenario Applications

    Jul 25, 2026Hao Jiang, Gangtao Xin, Yingdi Huang +35Tool-Augmented Language Model AgentsAgentic RL

    Huawei Cloud Post-Training Team