Overview

AI agent benchmarks test systems that act over multiple steps, use tools, and respond to changing information, rather than only answer isolated questions. Researchers build executable environments for settings such as web search, coding, graphical interfaces, games, scientific workflows, and multi-agent coordination, then score outcomes with programmatic checks, expert rubrics, or simulated users. Increasingly, evaluations also inspect trajectories, evidence use, efficiency, safety, and recovery from errors. Agents can perform well on narrow tasks yet remain unreliable on long-horizon work, and results often depend strongly on the model, harness, environment, and grading method.

Latest papers

CardsList

1,598 papers130 papers in the last four weeks, down 53% on the four weeks before. 1.0% of all new papers.

  1. ClosureBench: A Benchmark for Reasoning under Closure Assumptions

    Nov 19, 2026Francesco Belardinelli, Flavio Corradini, Matteo LeonesiAI Agent BenchmarksLanguage Model Reasoning Evaluation

  2. CompAct-Bench: A Benchmark for Working-Context Compaction in Long-Horizon Agent Tasks

    Nov 19, 2026Jiayi Chen, Lizhe Chen, Yujia Chen +8Context Compaction for LLM AgentsAI Agent Benchmarks

  3. AutoRAGBench: A Benchmark for LLM Agents Automating RAG System Development

    Nov 19, 2026Hang Zhang, Yuzhi Pan, Ziyao Zhong +12LLM AgentsAI Agent Benchmarks

  4. AppSci-Bench: Can LLMs Build AI Agents and Make Them Better?

    Nov 19, 2026Sanjari Srivastava, Manpreet Kaur, Cheng Chang +7AI Agent BenchmarksLLM Agents

  5. Moving Generality from Model to Modeler: LLM Agents for PHM on a Deployment-Style Benchmark

    Nov 18, 2026Zhenling Chen, Zhiguo Zeng, Anne BarrosLLM Agent EvaluationAI Agent Benchmarks

  6. GridAnchor: Specialized Multi-Agent AI for Safeguarding Autonomous Energy Systems Against Cyber-Physical Hallucinations

    Nov 18, 2026Abderaouf Bahi, Hasan MasrurCPS SecurityPower Systems

  7. EverydayBench: Evaluating Satisfactory Completion of Live User-Facing Digital Work

    Nov 18, 2026Fangrui Huang, Bingyang Wang, Chuhan Li +14Web Agent BenchmarksAI Agent Benchmarks

  8. Ground, Iterate, Trace: Decomposing AI Research Ideation

    Nov 18, 2026Samson Yu, Qiran Zou, Jing Yu Lim +3Research Idea GenerationAI Agents for Scientific Discovery

  9. RSIBench-Infra: Can Agents Optimize Their Own Training and Inference Infrastructure at Production Scale?

    Nov 18, 2026Hanlin Wu, Yifei Wang, Zhicheng Cai +5AI Agent BenchmarksEfficient Neural Network Inference

  10. Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development

    Nov 18, 2026Xirui Jiang, Yuxi Chen, Rui Yang +11Game-Playing AgentsAI Agent Benchmarks

  11. A Capability Framework for Evaluating Agentic AI Platforms in Regulated Enterprises

    Nov 18, 2026Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona ZahidAI Agent EvaluationAI Agent Benchmarks

  12. Verifying Agents in Rubric-Graded Environments

    Nov 17, 2026Markus Dücker, Vaibhav Kumar, Yi Liu +4LLM Agent VerificationAI Agent Evaluation

  13. Harness Leverage Across Model Size: A Ladder of Five Local Agent Harnesses from 0.75B to 4.7B

    Nov 16, 2026Hao Wang, Ting HuangLLM Agent HarnessesAI Agent Benchmarks

  14. CAE-Bench: Benchmarking AI Agents for Reusable CAE Solver Development

    Nov 16, 2026Zhiyu Zhou, Tianyou Liu, Xinping Yang +8AI Agent BenchmarksCoding Agent Benchmarks

  15. DetourBench: Trajectory-Level Evaluation of Interactive Agents with Exact Minimum-Action Bounds

    Nov 16, 2026Hanlin Tian, Sihan Zhu, Zhao Yang +5Agent EvaluationProcess-Based Agent Evaluation

  16. VAHBench: A Harness Benchmark for Agentic Long-Video Generation

    Nov 16, 2026Hongbo Zhang, Liuyang Song, Hang Zhou +4Agentic Video GenerationVideo Generation Evaluation

  17. Does a completed simulation run mean a usable urban wind result for AI agents?

    Nov 14, 2026Zhongke Qu, Yunwei Zhang, Junwei Su +5LLM AgentsProcess-Based Agent Evaluation

  18. OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination

    Nov 13, 2026Jie Huang, Yuecheng Hong, Yao Chen +3Long-Horizon Agent EvaluationLLM Agents

  19. Visual Cipher Bench: Benchmarking LLM Agents on Long-Horizon Visual Reasoning

    Nov 12, 2026Deyao Hong, Jiayi Jiang, Yuchun Chen +12Visual ReasoningAI Agent Benchmarks

  20. Singularity-Bench: Can Agents Build Agents?

    Nov 12, 2026Yang Zhou, Krish Agarwal, Zhuoming Chen +16AI Agent BenchmarksLong-Horizon Agent Evaluation

  21. AstroMemBench: A Benchmark for Memory Systems in Embodied Spacecraft Agents

    Nov 12, 2026You Wu, Guhong Zhang, Hongwei Li +2Memory Agent BenchmarksLong-Term Memory for Embodied Agents