Computer-Use Agent Benchmarks
Overview
These benchmarks measure whether agents can complete tasks through desktop, mobile, and web interfaces, often across multiple apps and long sequences of actions. They use sandboxed or live environments, with deterministic checks of application state and saved artifacts alongside trajectory rubrics and human or model judgments. Evaluation increasingly covers professional workflows, changing interfaces, privacy and safety, and efficiency, while testing how performance depends on the agent harness and available tools. A central challenge is balancing realistic, varied tasks with reproducible, tamper-resistant scoring; headline success rates can hide partial progress, process errors, and failures to deliver usable results.
Latest papers
121 papers15 papers in the last four weeks, up 36% on the four weeks before. 0.1% of all new papers.