Computer-Use Agent Benchmarks

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 91

All topics

JobBench: Aligning Agent Work With Human Will
May 25, 2026Yuetai Li, Yichen Feng, Zhangchen Xu +21Computer-Use Agent Benchmarks LLM Agent Evaluation
University of Washington · 10Bake AI · University of California, Santa Barbara +10
Anchor: Mitigating Artifact Drift in Agent Benchmark Generation
May 25, 2026Maksim Ivanov, Abhijay RanaComputer-Use Agent Benchmarks Long-Horizon Agent Evaluation
Agentic Labs New York, United States
Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World
May 25, 2026Yusong Lin, Xinyuan Liang, Haiyang Wang +8Computer-Use Agent Benchmarks AI Agent Benchmarks
Beijing Institute of Technology · Huawei Technologies Co., Ltd · Peking University +1
AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions
May 25, 2026Jingwei Sun, Jianing Zhu, Yuanyi Li +3Computer-Use Agent Benchmarks Computer-Use Agents
TMLR Group, Hong Kong Baptist University · The University of Texas at Austin · Sydney AI Centre, The University of Sydney +1
Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety
May 21, 2026Piercosma Bisconti, Matteo Prandi, Federico Pierucci +11LLM Safety Benchmarks Computer-Use Agent Benchmarks
Icaro Foundation · Sapienza University of Rome · Sant’Anna School of Advanced Studies +6
TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks
May 21, 2026Zhaoyang Chu, Jiarui Hu, Xingyu Jiang +8Computer-Use Agent Benchmarks Terminal Agents
University College London · Nanjing University · Tencent
AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
May 19, 2026Parsa Mazaheri, Kasra MazaheriAgent Failure Analysis Computer-Use Agent Benchmarks
University of California, Santa Cruz · Massachusetts Institute of Technology
OpenComputer: Verifiable Software Worlds for Computer-Use Agents
May 19, 2026Jinbiao Wei, Qianran Ma, Yilun Zhao +4Computer-Use Agent Benchmarks Computer-Use Agents
Yale NLP Lab · University of Pennsylvania · University of North Carolina at Chapel Hill
Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks
May 18, 2026Yubin Qu, Ying Zhang, Yanjun Zhang +4Computer-Use Agent Benchmarks Coding Agents
Griffith University · Wake Forest University · Nanyang Technological University +2
SaaS-Bench: Can Computer-Use Agents Leverage Real-World SaaS to Solve Professional Workflows?
May 15, 2026Kean Shi, Zihang Li, Tianyi Ma +13Computer-Use Agent Benchmarks Computer-Use Agents
1UniPat AI · 2PKU · 3HKU +2
ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents
May 13, 2026Yuxiang Lai, Peng Xia, Haonian Ji +8Computer-Use Agent Benchmarks LLM Agent Evaluation
University of North Carolina at Chapel Hill · Stanford University · University of Southern California +2
Do Androids Dream of Breaking the Game? Systematically Auditing AI Agent Benchmarks with BenchJack
May 12, 2026Hao Wang, Hanchen Li, Qiuyang Mang +3Computer-Use Agent Benchmarks Reward Hacking
UC Berkeley
Covering Human Action Space for Computer Use: Data Synthesis and Benchmark
May 12, 2026Miaosen Zhang, Xiaohan Zhao, Zhihong Tan +14Computer-Use Agent Benchmarks Computer-Use Agents
1Southeast University · 2Mohamed bin Zayed University of Artificial Intelligence · 3Wuhan University +2
LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
May 12, 2026Di Wu, Zixiang Ji, Asmi Kawatkar +4Computer-Use Agent Benchmarks Retrieval-Augmented Generation
University of California, Los Angeles
No More, No Less: Task Alignment in Terminal Agents
May 12, 2026Sina Mavali, David Pape, Jonathan Evertz +5Computer-Use Agent Benchmarks AI Alignment
CISPA Helmholtz Center for Information Security · ELLIS Institute Tübingen and MPI for Intelligent Systems · Tübingen AI Center
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation
May 11, 2026Shuangrui Ding, Xuanlang Dai, Long Xing +14Computer-Use Agent Benchmarks Tool-Augmented Language Model Agents
1Shanghai AI Laboratory, 2The Chinese University of Hong Kong · 1Shanghai AI Laboratory, 3Fudan University · 1Shanghai AI Laboratory, 4University of Science and Technology of China +8
Can Agent Benchmarks Support Their Scores? Evidence-Supported Bounds for Interactive-Agent Evaluation
May 11, 2026Shanshan Gao, Liyi ZhouComputer-Use Agent Benchmarks Benchmark Design
The University of Sydney
An Executable Benchmarking Suite for Tool-Using Agents
May 10, 2026Zhiqing Zhong, Zhijing Ye, Jiamin Wang +1Computer-Use Agent Benchmarks Tool-Use Evaluation
Stevens Institute of Technology
MMTB: Evaluating Terminal Agents on Multimedia-File Tasks
May 8, 2026Chiyeong Heo, Jaechang Kim, Junhyuk Kwon +4Computer-Use Agent Benchmarks Audio-Visual Understanding
GSAI, POSTECH · National AI Research Lab · Krafton AI +1
Computer Use at the Edge of the Statistical Precipice
May 7, 2026Pierluca D'Oro, Sneha Silwal, William Wong +6Computer-Use Agent Benchmarks Computer-Use Agents
Meta Superintelligence Labs
MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
May 7, 2026Ashwani Anand, Ivi Chatzi, Ritam Raha +1Computer-Use Agent Benchmarks AI Agent Reliability
Max Planck Institute for Software Systems, Kaiserslautern, Germany
Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
May 5, 2026Zirui Tang, Xuanhe Zhou, Yumou Liu +19Computer-Use Agent Benchmarks AI Agent Evaluation
1Shanghai Jiao Tong University · 2ByteDance · 5Independent Researher +2
AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?
May 1, 2026Ranit Karmakar, Jayita ChatterjeeComputer-Use Agent Benchmarks LLM Agent Evaluation
Harvard University, Boston, MA 02115 · Boston, MA 02115
What Makes a Good Terminal-Agent Benchmark Task: A Guideline for Adversarial, Difficult, and Legible Evaluation Design
Apr 30, 2026Ivan BercovichComputer-Use Agent Benchmarks Reward Hacking
WindowsWorld: A Process-Centric Benchmark of Autonomous GUI Agents in Professional Cross-Application Environments
Apr 30, 2026Jinchao Li, Yunxin Li, Chenrui Zhao +3Computer-Use Agent Benchmarks Computer-Use Agents
Harbin Institute of Technology, Shenzhen · Shenzhen Loop Area Institute
Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks
Apr 27, 2026Lawrence Keunho Jang, Jing Yu Koh, Daniel Fried +1Computer-Use Agent Benchmarks Web Agent Benchmarks
Carnegie Mellon University
ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents
Apr 26, 2026Fanqing Meng, Lingxiao Du, Zijian Wu +46Computer-Use Agent Benchmarks LLM Agent Evaluation
AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents
Apr 3, 2026Yunhao Feng, Yifan Ding, Yingshui Tan +6LLM Safety Benchmarks Computer-Use Agent Benchmarks
Alibaba Group · Fudan University · Hunan Institute of Advanced Technology +2
LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
Mar 20, 2026Xiang Long, Li Du, Yilong Xu +11Computer-Use Agent Benchmarks Tool-Augmented Language Model Agents
Samsung Research, Beijing, China · HKUST (Guangzhou) · City University of Hong Kong +1
TimeWarp: Evaluating Web Agents by Revisiting the Past
Mar 5, 2026Md Farhan Ishmam, Kenneth MarinoComputer-Use Agent Benchmarks Web Agent Benchmarks
University of Utah, USA

May 25, 2026, cs.AIJ/K move · Enter open · S save

JobBench: Aligning Agent Work With Human Will

Yuetai Li, Yichen Feng, Zhangchen Xu, Zixian Ma+20

University of Washington · 10Bake AI · University of California, Santa Barbara+10

Computer-Use Agent Benchmarks LLM Agent Evaluation AI Agent Benchmarks Human-AI Collaboration AI and Labor

Open paper AskPDF arXiv

Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluates AI agents on the workflows that experts identify as high-priority for delegation, empowering humans based on their needs instead of replacing them with GDP value. JobBench covers 130 agentic tasks across 35 occupations. Each task is packaged as a workspace of heterogeneous reference files, requiring the agent to reason through the cluttered information streams of real professional work. Outputs are graded by a fact-anchored chain of rubrics, averaging 35.6 binary criteria per task. We evaluate 36 models; the strongest, Claude Opus~4.7 under Claude Code, reaches only 45.9 %. We hope JobBench shifts the community's target labour-market effect from replacement to enhancement: building agents that do what humans actually want delegated, not only what is most economically valuable.

Momentum

Related topics

Latest papers 91