AI Agent Benchmarks

Follow this topic Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,013

All topics

VISTA: A Generative Egocentric Video Framework for Daily Assistance
May 11, 2026Yu-Hsiang Liu, Yu-Chien Tang, An-Zi YenSynthetic Data Generation AI Agent Benchmarks
Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan
Can Agent Benchmarks Support Their Scores? Evidence-Supported Bounds for Interactive-Agent Evaluation
May 11, 2026Shanshan Gao, Liyi ZhouComputer-Use Agent Benchmarks Benchmark Design
The University of Sydney
Agentic Performance at the Edge: Insights from Benchmarking
May 11, 2026Shiqiang Wang, Herbert WoisetschlägerLLM Agent Evaluation AI Agent Evaluation
University of Exeter · Exeter, United Kingdom · Technical University of Munich +1
Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values
May 11, 2026Haonan Dong, Qiguan Feng, Kehan Jiang +3AI Alignment AI Agent Evaluation
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Key Laboratory of Machine Perception (Ministry of Education), Peking University · School of Software and Microelectronics, Peking University +1
AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks
May 11, 2026Baraa Al Jorf, Farah E. ShamoutMulti-Agent LLM Systems LLM Agent Evaluation
New York University Abu Dhabi, UAE
SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems
May 11, 2026Zonglin Yang, Xingtong Liu, Xinyan XuAI Agents for Scientific Discovery AI Agent Evaluation
Readraft Lab · Peking University · Tongji University +1
Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse
May 11, 2026Kuan Zhang, Dongchen Liu, Qiyue Zhao +12Game-Playing Agents AI Agent Benchmarks
College of AI, Tsinghua University · MMLab, The University of Hong Kong · University of Chinese Academy of Sciences
EnactToM: An Evolving Benchmark for Functional Theory of Mind in Embodied Agents
May 11, 2026Gurusha Juneja, Dylan Lu, Saaket Agashe +7Multi-Agent Coordination Theory of Mind
University of California, Santa Barbara · 2King’s College London · 3Cisco Research
CalBench: Evaluating Coordination-Privacy Trade-offs in Multi-Agent LLMs
May 10, 2026Chelsea Zou, Yiheng Yao, Selena She +2Multi-Agent LLM Systems Multi-Agent Coordination
Stanford University
An Executable Benchmarking Suite for Tool-Using Agents
May 10, 2026Zhiqing Zhong, Zhijing Ye, Jiamin Wang +1Computer-Use Agent Benchmarks Tool-Use Evaluation
Stevens Institute of Technology
MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding
May 10, 2026Xiaoyu Yuan, Niklas Heikkala, Tiina Törmänen +3Theory of Mind AI Agent Benchmarks
University of Oulu
Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents
May 10, 2026Josefa Lia Stoisser, Marc Boubnovski Martell, Sidsel Boldsen +2LLM Agent Evaluation AI Agent Benchmarks
Novo Nordisk London, UK
DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agents
May 10, 2026Yixiong Chen, Wenjie Xiao, Pedro R. A. S. Bassi +7VLM Evaluation Tool Use in VLMs
Johns Hopkins University · University of Bologna · Center for Biomolecular Nanotechnologies, Istituto Italiano di Tecnologia +3
The Trap of Trajectory: Towards Understanding and Mitigating Spurious Correlations in Agentic Memory
May 10, 2026Luoxi Tang, Rupali Rajendra Vaje, Yuqiao Meng +5AI Agent Benchmarks LLM Agent Reliability
Binghamton University, State University of New York, Binghamton, NY, USA · Oakland University, Rochester, MI, USA · University of Pennsylvania, Philadelphia, PA, USA
MiroBench: Benchmarking Realism in Agentic Simulation of Real-world Discussions
May 10, 2026Yaoning Yu, Ye Yu, Haojing Luo +1Human Behavior Simulation LLM Agent Evaluation
University of Illinois at Urbana-Champaign
LLM Agents Already Know When to Call Tools -- Even Without Reasoning
May 10, 2026Chung-En Sun, Linbo Liu, Ge Yan +2Tool-Augmented Language Model Agents AI Agent Benchmarks
University of California, San Diego
MDGYM: Benchmarking AI Agents on Molecular Simulations
May 9, 2026Vinay Kumar, Satyendra Rajput, Mausam +1AI Agents for Scientific Discovery AI Agent Evaluation
Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi · Department of Computer Science and Engineering, Indian Institute of Technology Delhi · Department of Civil and Environmental Engineering, Indian Institute of Technology Delhi +1
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces
May 9, 2026Xiaozhe Li, Jixuan Chen, Xinyu Fang +4Large Language Model-Guided Optimization LLM Agent Self-Improvement
Tongji University1 · Shanghai AI Lab2 · University of California San Diego5 +2
When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents
May 9, 2026Strick Sheng, Ziyue Wang, Liyi ZhouAI Agent Reliability LLM Agent Evaluation
The University of Sydney · Nanjing University
Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows
May 9, 2026Tao Yu, Hao Wang, Changyu Li +15LLM Agent Evaluation Business Process Automation
BAAI · CASIA · UCAS +2
Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents
May 9, 2026Ying Chen, Lihuang Fang, Rui Jiang +4AI Agent Evaluation Agent Evaluation
XPENG ROBOTICS
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
May 9, 2026Bohan Lyu, Yucheng Yang, Siqiao Huang +25AI Agents for Scientific Discovery Automated Algorithm Discovery
UC Berkeley · Tsinghua University · Purdue University +6
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
May 9, 2026Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10AI Agent Reliability Multi-Agent Collaboration
University of Utah · University of Maryland, Baltimore County · University of Dhaka +7
Log analysis is necessary for credible evaluation of AI agents
May 8, 2026Peter Kirgis, Sayash Kapoor, Stephan Rabanser +8Agent Failure Analysis AI Agent Evaluation
Princeton University · Independent · UK AISI +4
Results and Retrospective Analysis of the CODS 2025 AssetOpsBench Challenge
May 8, 2026Dhaval Patel, Chathurangi Shyalika, Suryanarayana Reddy Yarrabothula +4Competitive Analysis Multi-Agent Orchestration
1IBM · 2Artificial Intelligence Institute at University of South Carolina · 3Steel Authority of India Limited +2
Learning CLI Agents with Structured Action Credit under Selective Observation
May 8, 2026Haoyang Su, Ying WenComputer-Use Agents Credit Assignment in RL
Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University
AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents
May 8, 2026Zhengkang Guo, Yiyang Li, Lin Qiu +7LLM Agent Evaluation Long-Horizon Agent Evaluation
1Fudan University · 2Meituan Longcat Team
SCENE: Recognizing Social Norms and Sanctioning in Group Chats
May 8, 2026Mateusz Jacniacki, Maksymilian BilskiLLM Agent Evaluation AI Agent Benchmarks
Humalike
DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain
May 8, 2026Hsuvas Borkakoty, Sebastian Pohl, Cheng Wang +2LLM Agent Evaluation AI Agent Benchmarks
Interdisciplinary Transformation University, Austria · Amazon, Berlin, Germany
MMTB: Evaluating Terminal Agents on Multimedia-File Tasks
May 8, 2026Chiyeong Heo, Jaechang Kim, Junhyuk Kwon +4Computer-Use Agent Benchmarks Audio-Visual Understanding
GSAI, POSTECH · National AI Research Lab · Krafton AI +1

May 11, 2026, cs.CLJ/K move · Enter open · S save

VISTA: A Generative Egocentric Video Framework for Daily Assistance

Yu-Hsiang Liu, Yu-Chien Tang, An-Zi Yen

Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan

Synthetic Data Generation AI Agent Benchmarks Egocentric Vision Controllable Video Generation

Open paper AskPDF arXiv

Training AI agents to proactively assist humans in daily activities, from routine household tasks to urgent safety situations, requires large-scale visual data. However, capturing such scenarios in the real world is often difficult, costly, or unsafe, and physics-based simulators lack the visual fidelity needed to transfer learned behaviors to real settings. Therefore, we introduce VISTA, a video synthesis system that produces high-fidelity egocentric videos as training and evaluation data for AI agents. VISTA employs a 5-step script generation pipeline with causal reverse reasoning to create diverse, logically grounded intervention modes. These scenarios span two levels of agent autonomy: reactive and proactive. In reactive modes, the user explicitly asks the agent for help. In proactive modes, the agent offers help without receiving a direct request. We further divide proactive modes into explicit and implicit types. In explicit proactive scenarios, the user is aware of needing help but does not directly address the agent. In implicit proactive scenarios, the agent intervenes before the user even realizes that help is needed. VISTA allows users to customize and refine scenarios to generate video benchmarks for daily tasks, offering a scalable and controllable alternative to real-world data collection for training and evaluating AI agents in realistic environments.

Momentum

Related topics

Latest papers 1,013