cs.CRMar 1, 2026

AWE: Adaptive Agents for Dynamic Web Penetration Testing

Authors: Akshat Singh Jaswal, Ashish Baghel

Organizations: Stux Labs

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern web applications are increasingly produced through AI-assisted development and rapid no-code deployment pipelines, widening the gap between accelerating software velocity and the limited adaptability of existing security tooling. Pattern-driven scanners fail to reason about novel contexts, while emerging LLM-based penetration testers rely on unconstrained exploration, yielding high cost, unstable behavior, and poor reproducibility. We introduce AWE, a memory-augmented multi-agent framework for autonomous web penetration testing that embeds structured, vulnerability-specific analysis pipelines within a lightweight LLM orchestration layer. Unlike general-purpose agents, AWE couples context aware payload mutations and generations with persistent memory and browser-backed verification to produce deterministic, exploitation-driven results. Evaluated on the 104-challenge XBOW benchmark, AWE achieves substantial gains on injection-class vulnerabilities - 87% XSS success (+30.5% over MAPTA) and 66.7% blind SQL injection success (+33.3%) - while being much faster, cheaper, and more token-efficient than MAPTA, despite using a midtier model (Claude Sonnet 4) versus MAPTA's GPT-5. MAPTA retains higher overall coverage due to broader exploratory capabilities, underscoring the complementary strengths of specialized and general-purpose architectures. Our results demonstrate that architecture matters as much as model reasoning capabilities: integrating LLMs into principled, vulnerability-aware pipelines yields substantial gains in accuracy, efficiency, and determinism for injection-class exploits. The source code for AWE is available at: https://github.com/stuxlabs/AWE

Figures & tables

Fig. 1: AWE system architecture.
Fig. 2: Five-phase XSS detection pipeline.
Fig. 3: Comparative performance of Claude Sonnet 4, GPT-4o, and Gemini 2.0 Flash across five vulnerability categories.
SystemSolve RateAvg. Time (s)Model
AWE51.9% (54/104)53.1Claude Sonnet 4
MAPTA76.9% (80/104)190.8GPT-5
TABLE I: Overall performance on the XBOW benchmark.
Fig. 4: Average number of payload iterations required for successful exploitation by each model. Claude Sonnet 4 converges in the fewest attempts (10–40), followed by GPT-4o with about 20% more iterations and Gemini 2.0 Flash with about 40% more. This demonstrates Claude’s superior efficiency and reasoning stability.
SystemTotal CostCost/SolveTokensTokens/Solve
AWE$7.73$0.1131.12M20.7K
MAPTA$21.38$0.26754.87M685.9K
TABLE II: Cost and token efficiency comparison.
VulnerabilityTotalMAPTAAWE
  Count%Count%
XSS231357%2087%
Blind SQLi3133%267%
SQLi66100%6100%
XXE33100%3100%
SSRF33100%3100%
TABLE III: Category-wise performance comparison on XBOW for Injection Vulnerablities.
More in this topic

Automated Penetration Testing

Automated penetration testing uses software agents to find, verify, and assess exploitable weaknesses in systems.

See all 48 papers

Explore similar work

CardsList
Jun 24, 2026

Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing

Large Language Models (LLMs) have shown promise for automated penetration testing, yet existing end-to-end black-box evaluations are highly susceptible to error cascading: failures in early reconnaissance can mask an agent's actual ability to exploit vulnerabilities. To more accurately characterize these capabilities, we propose a two-stage decoupled evaluation framework that separates exploit execution from reconnaissance. Using ground-truth injection and knowledge-driven ablation across 70 high-fidelity web vulnerability testbeds, our framework isolates exploitation performance from reconnaissance noise. We empirically evaluate five open-source penetration-testing agents, covering multiagent, monolithic, and graph-driven architectures, on a strictly aligned subset of 50 representative vulnerabilities. The results reveal a substantial capability gap. With accurate vulnerability context, agents achieve a functional success rate of up to 90.0%, whereas autonomous reconnaissance, measured by targeted vulnerability recall, plateaus at approximately 50.0%, primarily due to failures in parsing unstructured telemetry. Cross-architectural analysis further reveals distinct capability niches: multi-agent isolation is more effective for long-sequence interactions such as de-serialization, while monolithic and graph-driven designs perform better on short-chain injections and cross-session access-control vulnerabilities, respectively. This decoupled evaluation work provides a fine-grained benchmarking protocol and an empirical basis for designing next-generation automated offensive security agents.

Liwei Yu, Shuo Li, Ming Zhou +2

University of Science and Technology of China, Hefei, China · Nanjing University of Science and Technology, Nanjing, China · Research Institute, Runjian Co., Ltd, China +1

LLM Agent EvaluationAI Agent Security BenchmarksAutomated Penetration TestingActive Information Gathering
AskPDF
May 24, 2026

APT-Agent: Automated Penetration Testing using Large Language Models

Penetration testing is essential to securing modern web infrastructures, yet traditional manual methods struggle to keep pace with their scale and complexity. Large Language Models (LLMs) offer new opportunities for automating these tasks, but existing approaches face two persistent challenges: hallucination of technical entities and insufficient long-term contextual memory. To address these issues, we present APT-Agent, a fully automated LLM-driven penetration testing framework that systematically orchestrates reconnaissance, exploitation, and exfiltration. APT-Agent introduces a hybrid rectification module to recover hallucinated commands and a command-specific memory architecture to preserve operational context across multi-step attack sequences. We evaluate our APT-Agent on Metasploitable 2 against seven vulnerable services spanning web, database, and network protocols. APT-Agent achieves an 84.29% end-to-end exploitation success rate, compared to 48.57% (Script Kiddie) and 18.57% (PentestGPT) under matched conditions. By reducing cognitive burden and minimizing reliance on human intervention, APT-Agent represents a step toward scalable, reliable, and cognitively efficient automation for penetration testing.

William Guanting Li, Alsharif Abuadbba, Kristen Moore +1

University of Queensland · University of Queensland Brisbane, Australia · CSIRO Data61 +2

LLM Hallucination MitigationAutomated Penetration TestingLLMs for CybersecurityMemory-Augmented Language Model Agents
AskPDF
Jul 22, 2026

Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents

Traditional pentesting uses reconnaissance at each step to uncover unseen weaknesses, build stronger attacks, and advance the objective; we argue that AI agents require the same treatment. We formalize agent reconnaissance by modeling the process and identifying the knowledge assets it seeks to extract: what they are, how they are used, and which agent weaknesses they exploit to give adversaries leverage in indirect prompt injection attacks. We instantiate these insights in Know Your Agent (KYA), a framework that automates black-box, reconnaissance-driven pentesting by probing agents, building target profiles, and using those profiles to craft stronger attacks. We evaluate KYA on agent-security benchmarks and a real-world coding agent, and release KYA, its benchmarks, and baseline implementations for reproducibility.

Or Zion Eliav, Eyal Lenga, Shir Bernstien +1

Faculty of Computer and Information Science Ben-Gurion University of the Negev Beer-Sheva, Israel

AI Agent SecurityAI Agent Security BenchmarksIndirect Prompt InjectionAutomated Penetration TestingPrompt Injection Attacks on LLMsPrompt Injection Attacks on AI Agents
AskPDF