AI Agent Benchmarks

Ask about this topic

Momentum

96 papers in the last four weeks, up 30% on the four weeks before. 1.2% of all new papers.

Jul 6Week of Sep 21
View on map

Latest papers 1,013

All topics
CardsList

  1. MemSyco-Bench: Benchmarking Sycophancy in Agent Memory

    Jul 1, 2026Zhishang Xiang, Zerui Chen, Yunbo Tang +5Agent MemoryLLM Sycophancy

    1Xiamen University · 2Jilin University

  2. LitReview Arena: Evaluating Literature Review Agents with Battle-Style Peer Review Platform

    Jul 1, 2026Ruotong Zhao, Zhiyu Chen, Xurui Liu +7Human Preference EvaluationLLM-as-a-Judge

    Tsinghua University, Beijing, China · Zhongguancun Academy, Beijing, China · Zhongguancun Institute of AI, Beijing, China +2

  3. Evaluating Agentic Harness Systems for Autonomous Computational Pathology

    Jul 1, 2026Jie Lin, Zongyi Chen, Qiaoling Zheng +6Computational PathologyLLM Agent Evaluation

    Department of Computer Science at School of Informatics, Xiamen University, Xiamen, China. · National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China. · Department of Pathology, Fujian Medical University Union Hospital, Fuzhou, China. +1

  4. AGI Maze as a Benchmark Framework for World-Modeling Agents

    Jul 1, 2026Alexey PotapovAgent MemoryAI Agent Benchmarks

    SingularityNET Foundation

  5. EgoGapBench: Benchmarking Egocentric Action Selection in Multi-Agent Scenes

    Jul 1, 2026Jihyeok Jung, Jeewu Lee, Sanghyeop Kim +2Egocentric Video UnderstandingAI Agent Benchmarks

    KAIST AI · Sogang University · Ministry of Science and ICT

  6. PHREEQC-MCQ-200: A Diagnostic Benchmark for Tool-Augmented Scientific Simulator Agents

    Jul 1, 2026Ke Zhang, Sahchit Chundur, Mohammad Javad Qomi +1Computer-Use Agent BenchmarksAI for Science

    University of California, Riverside · University of California, Irvine

  7. EPC: A Standardized Protocol for Measuring Evaluator Preference Dynamics in LLM Agent Systems

    Jul 1, 2026Zewen LiuLLM Agent EvaluationAI Agent Benchmarks

    Anonymous authors Paper under double-blind review

  8. QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents

    Jun 30, 2026Sergio Hernández-Gutiérrez, Matteo Merler, Ilze Amanda Auzina +3Long-Horizon Agent EvaluationAI Agent Benchmarks

    Tübingen AI Center, University of Tübingen · Fondazione Bruno Kessler

  9. DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

    Jun 30, 2026Meng Chen, Anya Ji, Tsung-Han Wu +4Computer-Use AgentsAI in Education

    University of California, Berkeley · 2Technical University of Munich

  10. MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments

    Jun 30, 2026Qingyun Liu, Jiwen Zhang, Jingyi Hu +2Multi-Agent CollaborationAI Agent Benchmarks

    1Fudan University · 3The Chinese University of Hong Kong · 2Shanghai Innovation Institute

  11. FinPersona-Bench: A Benchmark for Longitudinal Psychometric Stability of Autonomous Financial Agents

    Jun 30, 2026Muhammad Usman Safder, Ayesha Gull, Rania Elbadry +7Personality Modeling in Language ModelsLong-Horizon Agent Evaluation

    Steve · MBZUAI · The University of Tokyo +2

  12. Xiaomi-GUI-0 Technical Report

    Jun 30, 2026Wanxia Cao, Chengzhen Duan, Pei Fu +29Mobile GUI AutomationRL for GUI Agents

  13. HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents

    Jun 30, 2026Qianchu Liu, Sheng Zhang, Guanghui Qin +16Long-Horizon Agent EvaluationAI Agent Benchmarks

    Microsoft Research

  14. ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents

    Jun 30, 2026Kaiwen Xiong, Haonian Ji, Shi Qiu +4LLM Agent OrchestrationLLM Agent Evaluation

    1UNC-Chapel Hill · University of California, Berkeley · University of California, Santa Cruz

  15. AhaBench: Do Agents Turn Experience into Reusable Insights? A Long-Horizon Benchmark for Continual Learning

    Jun 30, 2026Zerui Cheng, Jiawei Xu, Huacan Chai +3Continual Learning for LLM AgentsLifelong Learning Agents

    Princeton University · Tencent Hy · Tsinghua University +2

  16. MultiUAV-Plat: An LLM-Oriented Platform, Benchmark and Framework for Multi-UAV Collaborative Task Planning

    Jun 30, 2026Sheng Zhang, Qinglin Li, Yuechao Zang +3Aerial RoboticsLarge Language Model-Based Robot Planning

    National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Deya Road, Changsha, 410073, Hunan, China

  17. Beyond expert users: agents should help users construct preferences, not just elicit them

    Jun 29, 2026Irena Saracay, Ludwig Schmidt, Carlos GuestrinAI Agent BenchmarksRecommender Systems

    Stanford University

  18. What Drives Interactive Improvement from Feedback?

    Jun 29, 2026Bartłomiej Cupiał, Jan Łojek, Mikołaj Garstecki +3LLM Self-RefinementLLM Agent Evaluation

    University of Warsaw · AKCES NCBR · Princeton University +2

  19. MirrorCode: AI can rebuild entire programs from behavior alone

    Jun 29, 2026Tom Adamczewski, David Owen, David Rein +4Software Engineering AgentsAI Agent Benchmarks

    1Epoch AI · 2METR · 3Prime Intellect +2

  20. SWE-Together: Evaluating Coding Agents in Interactive User Sessions

    Jun 29, 2026Yifan Wu, Zhuokai Zhao, Songlin Li +8Coding AgentsAI Agent Evaluation

  21. Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios?

    Jun 29, 2026Yangda Peng, Yunjia Qi, Haotian Xia +8LLM-as-a-JudgeAI Agent Benchmarks

    Department of Computer Science and Technology, Tsinghua University · Tencent Hunyuan

  22. CLQT: A Closed-Loop, Cost-Aware, Strategy-Consistent Benchmark for Diagnostic Evaluation of LLM Portfolio-Management Agents

    Jun 29, 2026Bo Qu, Mingguang ChenQuantitative FinanceLLM Agent Evaluation

    Illinois Institute of Technology (IIT). · University of California, Riverside (UCR).

  23. ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit

    Jun 29, 2026Abhijnan Nath, Nikhil KrishnaswamyBelief-Space PlanningCredit Assignment in RL

    Situated Grounding and Natural Language (SIGNAL) Lab Department of Computer Science, Colorado State University Fort Collins, CO 80523 USA

  24. Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction

    Jun 28, 2026Sunqi Fan, Qingle Liu, Runqi Yin +2GUI AgentsVideo QA

    Tsinghua University, Beijing, China

  25. Hierarchical Experimentalist Agents

    Jun 28, 2026Abhranil Chandra, Sankaran Vaidyanathan, Utsav Dhanuka +2LLM Agent Skill LearningAI Agent Benchmarks

    University of Massachusetts Amherst

  26. A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis

    Jun 28, 2026Yuanhong Cai, Xiaohui Nie, Kanglin Yin +8Software EngineeringLLM Agent Evaluation

    Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Key Laboratory for Satellite Digitalization Technology, Chinese Academy of Sciences, Shanghai, China · Nankai University, Tianjin, China +2

  27. Multi-Agent Routing as Set-Valued Prediction: A WildChat Benchmark and Cost-Aware Evaluation

    Jun 27, 2026Ananto Nayan Bala, Faisal Muhammad ShahMulti-Label ClassificationAI Agent Benchmarks

    Ahsanullah University of Science and Technology Dhaka, Bangladesh

  28. MedEvoEval: Evaluating Continual Evolution of Doctor Agents through Simulated Clinical Episodes

    Jun 27, 2026Hui ZhangContinual Learning for LLM AgentsLong-Horizon Agent Evaluation

    Beijing Institute of Technology (BIT), China

  29. SEATauBench: Adapting Tool-Agent-User Evaluation Into Low-Resource Southeast Asian Languages

    Jun 27, 2026My Chiffon Nguyen, Aulia Adila, Saksorn Ruangtanusak +4LLM Agent EvaluationAI Agent Evaluation

    SEACrowd · SCB DataX, SCBX Group · Chulalongkorn University +2

  30. GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes

    Jun 26, 2026Amit Parekh, Sabrina McCallum, Kareem Al-Hasan +3Game-Playing AgentsMulti-Agent Collaboration

    Heriot-Watt University · University of Edinburgh