Web Agent Benchmarks

Ask about this topic

Overview

Web agent benchmarks measure whether AI systems can find information and complete tasks through browser interfaces, from web research and form filling to shopping and website development. They use simulated websites for repeatable tests, live sites for realistic workflows, and controlled changes to interfaces or instructions to probe robustness. Evaluation combines task success with measures such as evidence quality, interaction traces, efficiency, and human or automated judgments. A central challenge is balancing realistic, varied settings with reproducible scoring; agents still often struggle with long tasks, changing interfaces, and verifying that their actions produced the intended result.

Latest papers

CardsList

64 papers12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

  1. Back to Basics: A Skill-Centric Approach to Generalist Web Agents

    Oct 26, 2026Minh Pham-Dinh, Md Farhan Ishmam, Hernan Yupanqui +2Web AgentsLLM Agent Skill Learning

    University of Utah · Islamic University of Technology

  2. The Agent in the Middle: How AI-Mediated Access Reshapes Web Vulnerability Exploitation

    Oct 26, 2026Ruozhao Yang, Jiongchi Yu, Ming Hu +1AI Agent SecurityAI Agent Security Benchmarks

    Singapore Management University, Singapore · Nanyang Technological University, Singapore · East China Normal University, China

  3. AgentProofBench: Benchmarking Reusable Verifiers for Web Agents

    Oct 25, 2026Jingjie Ning, Xueqi Li, Yi Lu +1LLM Agent VerificationWeb Agent Benchmarks

  4. SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents

    Oct 24, 2026Bo-Yu Chen, Zhi Zhou, Yu-Feng LiAgent Skill LearningWeb Agent Benchmarks

    School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University

  5. WebUniverse: Scaling Verified Training Data for Agentic Web Development

    Oct 23, 2026Yaorui Ding, Fengqing Jiang, Yichen Feng +8Web AgentsWeb Application Generation

    University of Washington · Western Washington University · King Abdulaziz City for Science and Technology +1

  6. LoWAgent: A Low-Resource Multilingual Benchmark for Web Agents

    Oct 19, 2026Ocean Monjur, Daud Ahmad Nisar, Shahriar Kabir Nahin +4Web Agent BenchmarksWeb Agents

    University of South Florida, USA · Instituto Politécnico Nacional, Mexico · Independent Researcher +1

  7. Get In, Decide, Prove: A Merchant-Side Readiness Benchmark for Agentic Commerce

    Oct 12, 2026Adrian de Valois-Franklin, Alex Bogdan, Anas Ibrahim +2Agentic CommerceWeb Agent Benchmarks

    Evolutionairy AI Toronto, Canada

  8. HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks

    Oct 12, 2026Zijian Cai, Haowei Liu, Kailai Chen +7Web AgentsWeb Search Agents

    Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, CASIA · State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences +5

  9. A Baseline Assessment for the Agentic Web Technology Stack

    Oct 9, 2026Vagner Figueredo de Santana, Diogo CortizWeb AgentsMulti-Agent LLM Systems

    NIC.br, Brazil · NIC.br / PUC-SP, Brazil

  10. When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks

    Oct 9, 2026Yisheng Li, Dongming Han, Hongkun Pan +5Web AgentsAI Agent Reliability

    State Key Laboratory of Novel Software Technology, Nanjing University · HiThink Research · Zhejiang University

  11. HyperBrowseComp: A Multilingual and Multimodal Stress Test for Web-Browsing Agents

    Oct 2, 2026Alham Fikri Aji, Faiz Rizki Ramadhan, Zayd M. K. Zuhri +14Web Agent BenchmarksAgent Benchmarks

    Mohamed bin Zayed University of Artificial Intelligence · Mila – Quebec Artificial Intelligence Institute · Inception AI +2

  12. WebFovea: When the Model Is Right but the Click Is Wrong -- Reliable Round Trips for Vision-Based Web Agents on Live Websites

    Oct 2, 2026Jiangang HanWeb AgentsTool-Using Vision-Language Agents

    Independent Researcher

  13. Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and Trajectories

    Oct 1, 2026Chengguang Gan, Zimeng He, Yoshihiro Tsujii +3AI Agent AuditingWeb Agent Benchmarks

    Techtouch, Inc. · Institute of Science Tokyo

  14. WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents

    Sep 28, 2026Anton Emelyanov, Maria Tikhonova, Zaven Martirosian +2Web Agent BenchmarksGUI Agents

    DAIMLD · HSE University · NUST MISIS

  15. The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

    Sep 24, 2026Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen +6Computer-Use Agent EvaluationComputer-Use Agents

    University of Utah

  16. Guides That Cause Actions: An Offline Study of Guide-Action Mutual Reinforcement in Multimodal Web Agents

    Sep 23, 2026Chengguang Gan, Yunhao Liang, QingHao Zhang +1Web Agent BenchmarksAI Agent Evaluation

    Independent Researcher

  17. Constructing Challenging Browser-Use Tasks by Controlled Environment Interventions

    Sep 20, 2026Xunjian Yin, Tianchen Guan, Jinao Wang +8Web AgentsWeb Agent Benchmarks

    Duke University · Amazon

  18. RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation

    Sep 15, 2026ZhuoXin Liu, Zhiming Ma, Ying Zhang +10Web Agent BenchmarksAI Agent Safety Benchmarks

    Baidu · SmartFlowAI · JD Technology +3

  19. Signal or Noise? A Benchmark Study of Agent Skills in Web Development

    Aug 24, 2026Ziyue Yang, Fan DingLLM Agent Skill EvaluationSoftware Engineering Agents

    Baidu, NLP Beijing, China

  20. Wuying-Browser-Agent: Real-World Centric Fundamental Long-Horizon Browser Agents

    Aug 18, 2026AIMAE Team, Tianxiang Chen, Yan Cheng +39Web AgentsLong-Horizon LLM Agents

  21. CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception

    Aug 9, 2026Zejun Xu, Taiyi Chen, Jin Li +13Computer-Use AgentsWeb Agent Benchmarks

    Macau University of Science and Technology · FellouAI · Tsinghua University +3

  22. Routing Is Least Learnable Where It Is Most Valuable: Bounds on Representation Routing for Web Agents

    Aug 6, 2026Jiaming Wei, Zekun Wu, Adriano Koshiyama +1Web Agent BenchmarksWeb Agents

    University College London / Holistic AI · Holistic AI / University College London · Holistic AI +1

  23. The Compliance Trap: Diagnosing How AI Agents Consume Conflicting Memory

    Jul 12, 2026Yixiong Chen, Xinyi Bai, Alan YuilleWeb Agent BenchmarksLong-Horizon Agent Evaluation

    1Johns Hopkins University · 2Cornell University

  24. MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation

    Jul 11, 2026Chengguang Gan, Hanjun Wei, Yunhao Liang +3Computer-Use AgentsWeb Agent Benchmarks

    Independent Researcher · University of Chinese Academy of Sciences · Monash University +2

  25. WebRetriever: A Large-Scale Comprehensive Benchmark for Efficient Web Agent Evaluation

    Jul 7, 2026Wei Dong, Tianyu Fu, Zhe Yu +9LLM-as-a-JudgeWeb Agent Benchmarks

    Mininglamp Technology · School of Artificial Intelligence, University of Chinese Academy of Sciences · MAIS, Institute of Automation, Chinese Academy of Sciences

  26. Ko-WideSearch: A Korean Breadth-Search Benchmark for Exhaustive Set Enumeration by Web Agents

    Jun 25, 2026Minbyul JeongComputer-Use Agent BenchmarksWeb Agent Benchmarks

    Upstage AI

  27. When Web Agents Finish but Still Fail: Reproducible Triggers and Trace Diagnostics for Parallel Web Exploration

    Jun 16, 2026Aagam Sogani, Botao Rui, Swetha Vaidyanathan +3Agent Failure AnalysisWeb Agent Benchmarks

    University of Wisconsin–Madison, Madison, WI, USA.

  28. LongWebBench: Evaluating Structural and Functional Webpage Generation in Long-Horizon Settings

    Jun 16, 2026Yi Zhao, Zhen Yang, Mengpan Chen +5VLM EvaluationWeb Application Generation

    Tsinghua University · Yanshan University · University of Waterloo +1

  29. MyPCBench: A Benchmark for Personally Intelligent Computer-Use Agents

    Jun 15, 2026Lawrence Keunho Jang, Andrew Keunwoo Jang, Jing Yu Koh +1Personal AI AgentsTool-Augmented Language Model Agents

    Carnegie Mellon University

  30. FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection

    Jun 15, 2026Y. H. Zhou, Z. M. Ma, Y. J. Zhou +12Web Agent BenchmarksLLM Agent Evaluation