Agent Benchmarks

Agent benchmarks test whether AI systems can complete tasks through sequences of decisions and actions, from web browsing and coding to desktop use, games, and professional workflows. They use simulated or real-world environments, tool-use tasks, long-horizon scenarios, and memory or multimodal challenges, with scoring based on verified outcomes, intermediate steps, costs, and reliability across repeated runs. Common approaches include executable state-based checks, trajectory review, human or rubric-based judging, and controlled comparisons of models, tools, and agent harnesses. Results show substantial gaps on complex tasks, while realism, reproducibility, verifier quality, and the influence of the harness make scores difficult to compare or generalize to deployment.