AI Agent Benchmarks

Follow this topic Ask about this topic

Momentum

148 papers in the last four weeks, up 185% on the four weeks before. 1.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 1,013

All topics

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework
Aug 7, 2026Jing Chen, Yang Sun, Li Zhang +2Long-Horizon Agent Evaluation AI Agent Benchmarks
Huawei Technologies Ltd.
TRACE: A Multi-Layer Benchmark for Human AI Controller Coordination Under Drift and Failure
Aug 7, 2026Joshua Zuniga, Srinivasan Subramanian, Ramya Madhuri Narapureddy +1Agent Failure Analysis AI Agent Benchmarks
College of Computing and Software Engineering Kennesaw State University Marietta, GA, USA
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
Aug 6, 2026Noam Koren, Roy Bar-Haim, Abigail GoldsteenAI Agent Benchmarks Task-Oriented Dialogue Systems
IBM Research
HarnessOpt-Bench: Evaluating LLMs at Harness Optimization
Aug 6, 2026Varun Ursekar, Apaar Shanker, Yash Maurya +4Large Language Model-Guided Optimization LLM Agent Evaluation
Scale AI
FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
Aug 6, 2026Bo Deng, Kang Zhou, Lifan Guo +6Long-Horizon Agent Evaluation AI Agent Benchmarks
Beihang University · Qwen DianJin Team, Alibaba Cloud Computing
Evaluating Investment Logic in Large Language Models: A Real-World Benchmark Towards Personalzied Financial Agents
Aug 6, 2026Yuanhong Jiang, Jingjie Zou, Zhenghong Lin +4AI-Assisted Decision Making LLM Agent Evaluation
Predicting Task Difficulty Without Rollouts
Aug 6, 2026Stefan Krsteski, Charlotte MeyerLong-Horizon Agent Evaluation AI Agent Benchmarks
Andromede AI
Unified Agent: Managing Interactions across Devices
Aug 6, 2026Xinshuang Liu, Runfa Blark Li, Shaoxiu Wei +2AI Agent Benchmarks State Tracking
University of California, San Diego San Diego, CA, USA
StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding
Aug 6, 2026Xichen Zhang, Guankai Li, Yinghao Zhu +6Memory-Augmented VLMs Streaming Video Understanding
The Hong Kong University of Science and Technology · Xiaohongshu Inc. · The University of Hong Kong +1
SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution
Aug 6, 2026Zhi Han, Chenxi Zeng, Liuhaichen Yang +3LLM-as-a-Judge Long-Horizon Agent Evaluation
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · University College London +1
EcoAgent-Bench: Evaluating Economic Decision-Making in Budget-Constrained LLM Agents
Aug 6, 2026Jie Wu, Ming Gong, Feixiang Cheng +1Computer-Use Agent Benchmarks Tool-Use Evaluation
Atlassian
RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough
Aug 5, 2026Anchen Sun, Kaiqi YangMulti-Agent LLM Systems AI Agent Benchmarks
Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL 33146 USA · Department of Mathematics, University of Miami, Coral Gables, FL 33146 USA
Strategic Evaluation of Planning Strategies for LLM Agents in Cyber-Physical Systems
Aug 4, 2026J. de Curtò, I. de ZarzàCyber-Physical Systems LLM Planning
Department of Computer Applications in Science and Engineering, BARCELONA Supercomputing Center, Barcelona, Spain · Escuela T´ecnica Superior de Ingenier´ıa (ICAI), Universidad Pontificia Comillas, Madrid, Spain · Human-Centered AI, Data and Software, LUXEMBOURG Institute of Science and Technology, Esch-sur-Alzette, Luxembourg
FinProBench: Evaluating Financial AI Agents with Role-Grounded Rubrics Derived from Professional Deliverables
Aug 4, 2026Ben Wang, Kang Zhou, Lifan Guo +2AI Agent Evaluation AI Agent Benchmarks
Qwen DianJin Team, Alibaba Cloud Computing
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
Aug 4, 2026Zhen Fang, Yu Zeng, Wenxuan Huang +17Deep Research Agents Video QA
ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?
Aug 4, 2026Tianyi Guan, Yiding Wang, Haotong Yang +5Continual Learning for LLM Agents LLM Agent Evaluation
Institute for Artificial Intelligence, Peking University · 2Beijing Institute for General Artificial Intelligence
GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks
Aug 4, 2026Leijun Zhou, Zhihao Liu, Xiang Qu +9LLM Agent Evaluation AI Agent Benchmarks
PrismShadow · New York University
Adversarial Fast-Moving Real-World Domains as Test Beds for Benchmarking AI Scientist Capabilities
Aug 4, 2026William Bolton, Philip TorrAI Agents for Scientific Discovery Benchmark Design
Department of Engineering Science, University of Oxford, Oxford, United Kingdom.
Can LLM design high-quality experiments? A Comprehensive and Systematic Benchmark on Autonomous Experimental Design
Aug 4, 2026Zejun Liu, Jian Wu, Ru Peng +4AI for Science Benchmark Design
Westlake University · Zhejiang University · Nanjing University of Science and Technology +1
MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
Aug 4, 2026Qiming Li, Shujie Hu, Haohan Liu +3Code Generation AI Agent Benchmarks
1Harbin Institute of Technology · 2Meituan, Longcat Team · 3Peng Cheng Laboratory
EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulated Learners
Aug 4, 2026Unggi Lee, Sookbun Lee, Yeil Jeong +3Intelligent Tutoring Systems AI in Education
Korea University Sejong Campus · Opentutorials · Indiana University +2
Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls
Aug 4, 2026Guoyao Yu, Xiaoqing Sun, Ziqi Huang +13Tool-Augmented Language Model Agents AI Agent Benchmarks
UrbanAgent: A Tool-Augmented Agent for Cross-System Urban Tasks
Aug 4, 2026Jiayu Cao, Xingyuan Zeng, feiyu Li +6LLM Agent Evaluation AI Agent Benchmarks
ParEvalLayer: When Partial LLM-Agent Evaluations Support a Decision
Aug 3, 2026Wei-Jung Huang, Bonan ShenPairwise Comparison LLM Agent Evaluation
Independent Researcher · United States
Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce
Aug 3, 2026Shicheng Fan, Mingdai Yang, Duohao Wang +9AI Agent Evaluation Agent Evaluation
University of Illinois at Chicago · 2Springbrand · 3Northwestern University +6
PredAct-Bench: Benchmarking Tool-Augmented Dialogue under Controlled Tool Noise
Aug 3, 2026Abdulrahman AlRabah, Xiaocheng Yang, Dilek Hakkani-Tür +1AI Agent Reliability AI-Assisted Decision Making
University of Illinois Urbana-Champaign
ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step
Aug 3, 2026Vernon Toh, Navonil Majumder, Zhengyuan Liu +2LLM Agent Evaluation AI Agent Benchmarks
DeCLaRe Lab, Nanyang Technological University, Singapore · Agency for Science, Technology, and Research (A*STAR), Singapore
From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents
Aug 3, 2026Jiajia Song, Bobo Li, Haiwen Yi +6LLM Personalization LLM Agent Evaluation
National University of Singapore · University of Toronto · University of Minnesota Twin Cities +2
CompanionBench: A Theory-Anchored, Real-World-Grounded Benchmark for AI Emotional Companionship
Aug 3, 2026Yao Liu, Guangjia Chai, Yuming Huang +3Emotional Support Conversation AI Companions
BulkPR-Bench: Benchmarking Queue-Level Governance of Interacting Pull Requests
Aug 3, 2026Zetong Xiong, Qiao Zhao, Jun Zhang +20Software Engineering Agents Benchmark Construction
Baidu Beijing, China

Aug 7, 2026, cs.AIJ/K move · Enter open · S save

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

Jing Chen, Yang Sun, Li Zhang, Lin Xu+1

Huawei Technologies Ltd.

Long-Horizon Agent Evaluation AI Agent Benchmarks Agent Trajectory Analysis

Open paper AskPDF arXiv

Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory. Existing benchmarks primarily evaluate behavioral outcomes but provide limited support for fine-grained attribution analysis. We introduce trajectory attribution and develop a benchmark and annotation framework for this task. The benchmark organizes heterogeneous trajectories under a unified component schema and provides annotations of the primary attribution component, together with attack and execution chains where applicable. Instantiating the benchmark with trajectories from AgentDojo and the Stage and Canary settings of Agent3Sigma yields more than 1,300 annotated trajectories covering task-aligned actions, unsafe actions, and safety refusals. The benchmark defines two evaluation tasks, primary attribution localization and attribution-chain recovery, and provides reference baselines based on incremental trajectory contribution and component-level leave-one-out perturbation. It captures diverse attribution settings, including local and long-range attribution as well as structured attribution chains. Reference baseline results exhibit substantial performance differences across these settings, providing an initial characterization of the benchmark's attribution challenges. Beyond this initial instantiation, we release a reusable annotation skill that enables trajectories generated by new agent models to be standardized, annotated, and evaluated under the same framework. Project resources and future releases are available at https://github.com/chenjing-2024/agent-trajectory-attribution.

Momentum

Related topics

Latest papers 1,013