cs.CROct 14, 2025

HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities

Authors: Xiaoxue Ren, Penghao Jiang, Kaixin Li, Zhiyong Huang, Xiaoning Du, Jiaojiao Jiang, Zhenchang Xing, Jiamou Sun, 

Organizations: Zhejiang University · University of New South Wales · National University of Singapore · Monash University · 

Abstract

Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making it difficult to scale with the growing web ecosystem. While language model agents show promise in cybersecurity, modern web applications demand visual understanding, dynamic content handling, and multi-step interactions that only computer-use agents (CUAs) can perform. Yet, their ability to discover and exploit vulnerabilities through graphical interfaces remains largely unexplored. We present HackWorld, the first framework for systematically evaluating CUAs' capabilities to exploit web application vulnerabilities via visual interaction. Unlike sanitized benchmarks, HackWorld includes 36 real-world applications across 11 frameworks and 7 languages, featuring realistic flaws such as injection vulnerabilities, authentication bypasses, and unsafe input handling. Using a Capture-the-Flag (CTF) setup, it tests CUAs' capacity to identify and exploit these weaknesses while navigating complex web interfaces. Evaluation of state-of-the-art CUAs reveals concerning trends: exploitation rates below 12% and low cybersecurity awareness. CUAs often fail at multi-step attack planning and misuse security tools. These results expose the current limitations of CUAs in web security contexts and highlight opportunities for developing more security-aware agents capable of effective vulnerability detection and exploitation.

Figures & tables

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

    May 11, 2026Zhun Wang, Nico Schiller, Hongwei Li +13AI Agent SecurityAI Agent Security Benchmarks

  2. Harness Matters: Measuring and Dissecting Security Resilience in Web Agents

    Nov 9, 2026Xinyi Wu, Jiagui Chen, JiaYi Dong +5LLM Agent SecurityLLM Agent Harnesses

  3. Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing

    Jun 24, 2026Liwei Yu, Shuo Li, Ming Zhou +2LLM Agent EvaluationAI Agent Security Benchmarks