Overview

AI agent benchmarks test systems that act over multiple steps, use tools, and respond to changing information, rather than only answer isolated questions. Researchers build executable environments for settings such as web search, coding, graphical interfaces, games, scientific workflows, and multi-agent coordination, then score outcomes with programmatic checks, expert rubrics, or simulated users. Increasingly, evaluations also inspect trajectories, evidence use, efficiency, safety, and recovery from errors. Agents can perform well on narrow tasks yet remain unreliable on long-horizon work, and results often depend strongly on the model, harness, environment, and grading method.

Latest papers

CardsList

1,598 papers130 papers in the last four weeks, down 53% on the four weeks before. 1.0% of all new papers.

  1. AI GameStore: Scalable, Open-Ended Evaluation of Machine General Intelligence with Human Games

    Feb 19, 2026Lance Ying, Ryan Truong, Prafull Sharma +9Artificial General IntelligenceAI Agent Benchmarks

  2. Hybrid-Gym: Training Coding Agents to Generalize Across Tasks

    Feb 18, 2026Yiqing Xie, Emmy Liu, Gaokai Zhang +7AI Coding AgentsSoftware Engineering Agents

  3. Towards a Science of AI Agent Reliability

    Feb 18, 2026Stephan Rabanser, Sayash Kapoor, Peter Kirgis +3AI Agent ReliabilityAI Agent Evaluation

  4. MMA: Multimodal Memory Agent

    Feb 18, 2026Yihao Lu, Wanru Cheng, Zeyu Zhang +1Multimodal AgentsMemory-Augmented Agents

  5. Learning Personalized Agents from Human Feedback

    Feb 18, 2026Kaiqu Liang, Julia Kruk, Shengyi Qian +9Personal AI AgentsUser Preference Modeling

  6. OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction

    Feb 16, 2026Skyler Hallinan, Thejas Venkatesh, Xiang Ren +4LLM Tool UseLLM Agents

  7. ResearchGym: Evaluating Language Model Agents on Real-World AI Research

    Feb 16, 2026Aniketh Garikaparthi, Manasi Patwardhan, Arman CohanAI Agent BenchmarksAutonomous Research Agents

  8. Hunt Globally: Wide Search AI Agents for Drug Asset Scouting in Investing, Business Development, and Competitive Intelligence

    Feb 16, 2026Vlad Vinogradov, Alisa Vinogradova, Luba Greenwood +7Deep Research AgentsWeb Search Agents

  9. Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systems

    Feb 16, 2026Pramit Saha, Joshua Strong, Mohammad Alsharid +2Tool SelectionModel Selection

  10. Engineering Reasoning and Instruction (ERI) Benchmark: A Large Taxonomy-driven Dataset for Foundation Models and Agents

    Feb 16, 2026MZ Naser, Ahmad Bani Awwad, Zoie McCreery +8LLM EvaluationAI Agent Benchmarks

  11. BASIS: Evaluating Belief-Aware Solutions in Interactive Settings beyond Instructions

    Feb 14, 2026Minyuan Ruan, Ziyue Wang, Kaiming Liu +3LLM Agent EvaluationAI Agent Benchmarks

  12. ReplicatorBench: Benchmarking LLM Agents for Replicability in Social and Behavioral Sciences

    Feb 11, 2026Bang Nguyen, Dominik Soós, Qian Ma +8LLM Agent EvaluationAI Agent Benchmarks

  13. GameDevBench: Evaluating Agentic Capabilities Through Game Development

    Feb 11, 2026Wayne Chi, Yixiong Fang, Arnav Yayavaram +8Multimodal Agent EvaluationSoftware Engineering Agents

  14. Evaluating Memory Structure in LLM Agents

    Feb 11, 2026Alina Shutova, Alexandra Olenina, Ivan Vinogradov +1LLM Agent MemoryLLM Agent Evaluation

  15. OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions

    Feb 5, 2026Fangzhi Xu, Hang Yan, Qiushi Sun +14AI Agent BenchmarksLong-Horizon Agent Evaluation

  16. PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios

    Feb 5, 2026Chris Zhu, Sasha Cui, Will Sanok Dufallo +4LLM Agent EvaluationMulti-Agent Negotiation

  17. ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response

    Feb 3, 2026Xiaomeng Zhu, Fengming Zhu, Weijie Zhou +8Proactive AgentsAI Agent Benchmarks

  18. Declarative by Design, Assistable Only by Convention: Benchmarking Multi-Agent Frameworks for AI-Assistability

    Feb 3, 2026Shafiuddin Rehan Ahmed, Sourabh DeshpandeAI Agent BenchmarksCode Generation Evaluation

  19. AgentRx: Diagnosing AI Agent Failures from Execution Trajectories

    Feb 2, 2026Shraddha Barke, Arnav Goyal, Alind Khare +3Fault DiagnosisAI Agent Reliability

  20. Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory

    Jan 30, 2026Yuhao Zhan, Tianyu Fan, Linxuan Huang +2Deep Research AgentsHallucination Detection

  21. EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents

    Jan 23, 2026Xinze Li, Ziyue Zhu, Siyuan Liu +4Episodic MemoryLong-Horizon Agent Evaluation

  22. Toward Efficient Agents: Memory, Tool learning, and Planning

    Jan 20, 2026Xiaofang Yang, Lijun Li, Heng Zhou +12LLM Agent MemoryAI Agent Benchmarks

  23. LifeAgentBench: Benchmarking LLMs for Long-Horizon, Cross-Dimensional Lifestyle Health Reasoning

    Jan 20, 2026Ye Tian, Zihao Wang, Onat Gungor +2HealthcareLong-Horizon Agent Evaluation

  24. DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

    Jan 20, 2026Maojun Sun, Yifei Xie, Yue Wu +5Multimodal Agent EvaluationLLM Agent Evaluation