Environment-Grounded Multi-Agent Workflow for Autonomous Penetration Testing

Michael Somma; Markus Großpointner; Paul Zabalegui; Eppu Heilimo; Branka Stojanović

cs.ROMar 25, 2026

Environment-Grounded Multi-Agent Workflow for Autonomous Penetration Testing

Authors: Michael Somma, Markus Großpointner, Paul Zabalegui, Eppu Heilimo, Branka Stojanović

Organizations: JOANNEUM RESEARCH Forschungsgesellschaft mbH, DIGITAL – Institute for Digital Technologies, Steyrergasse 17, 8010 Graz, Austria · TU Graz, Institute for Technical Informatics, Inffeldgasse 16/I, 8010 Graz, Austria · Alias Robotics S.L., Calle Venta de la Estrella, 6 Pab. 130, 01006 Vitoria-Gasteiz, Álava, Spain · Jamk University of Applied Sciences, Institute of Information Technology, 40101 Jyväskylä, Finland

Abstract

The increasing complexity and interconnectivity of digital infrastructures make scalable and reliable security assessment methods essential. Robotic systems represent a particularly important class of operational technology, as modern robots are highly networked cyber-physical systems deployed in domains such as industrial automation, logistics, and autonomous services. This paper explores the use of large language models for automated penetration testing in robotic environments. We propose an environment-grounded multi-agent architecture tailored to Robotics-based systems. The approach dynamically constructs a shared graph-based memory during execution that captures the observable system state, including network topology, communication channels, vulnerabilities, and attempted exploits. This enables structured automation while maintaining traceability and effective context management throughout the testing process. Evaluated across multiple iterations within a specialized robotics Capture-the-Flag scenario (ROS/ROS2), the system demonstrated high reliability, successfully completing the challenge in 100% of test runs (n=5). This performance significantly exceeds literature benchmarks while maintaining the traceability and human oversight required by frameworks like the EU AI Act.

Figures & tables

Fig. 1: Overview of the agentic penetration testing workflow. The figure illustrates the interaction of the agents across the main phases of the process: task planning, task execution, memory saving, and information retrieval.

Fig. 2: Docker network representing the robot-based manufacturing use case under test.

Approach	Model	CTF-0	CTF-1	CTF-2	CTF-3
Proposed	llama-3.3-70b-instruct	(5, 0)	(5, 0)	(5, 0)	(5, 0)
	deepseek-v3.2	(5, 0)	(1, 4)	(1, 4)	(0, 5)
	gemma-3-27b-it	(5, 0)	(0, 5)	(0, 5)	(0, 5)
	hermes-2-pro-llama-3-8b	(5, 0)	(0, 5)	(0, 5)	(0, 5)
HackSynth [ 20 ]	llama-3.3-70b-instruct	(5, 0)	(4, 1)	(0, 5)	(0, 5)
	deepseek-v3.2	(4, 1)	(2, 3)	(0, 5)	(0, 5)

TABLE I: Performance of the evaluated LLMs across CTF tasks over five attempts per task, reported as (success, failure) counts.

Automated Penetration Testing

Automated penetration testing uses software agents to find, verify, and assess exploitable weaknesses in systems.

See all 48 papers

APT-Agent: Automated Penetration Testing using Large Language Models
May 24, 2026William Guanting Li, Alsharif Abuadbba, Kristen Moore +1LLM Hallucination Mitigation Automated Penetration Testing
University of Queensland · University of Queensland Brisbane, Australia · CSIRO Data61 +2
Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing
Jun 24, 2026Liwei Yu, Shuo Li, Ming Zhou +2LLM Agent Evaluation AI Agent Security Benchmarks
University of Science and Technology of China, Hefei, China · Nanjing University of Science and Technology, Nanjing, China · Research Institute, Runjian Co., Ltd, China +1
Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing
Sep 7, 2026Weizhe Wang, Yitong Zhang, Yao Zhang +4LLM Agent Memory LLM Agent Security
Tianjin University, Tianjin, 300072, China · Changchun University of Science and Technology, Changchun, 130022, China · Jilin Province Key Laboratory of Network and Information Security, Changchun, 130022, China +1

May 24, 2026J/K move · Enter open · S save

APT-Agent: Automated Penetration Testing using Large Language Models

William Guanting Li, Alsharif Abuadbba, Kristen Moore, Dan Dongseong Kim

University of Queensland · University of Queensland Brisbane, Australia · CSIRO Data61+2

LLM Hallucination Mitigation Automated Penetration Testing LLMs for Cybersecurity Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Penetration testing is essential to securing modern web infrastructures, yet traditional manual methods struggle to keep pace with their scale and complexity. Large Language Models (LLMs) offer new opportunities for automating these tasks, but existing approaches face two persistent challenges: hallucination of technical entities and insufficient long-term contextual memory. To address these issues, we present APT-Agent, a fully automated LLM-driven penetration testing framework that systematically orchestrates reconnaissance, exploitation, and exfiltration. APT-Agent introduces a hybrid rectification module to recover hallucinated commands and a command-specific memory architecture to preserve operational context across multi-step attack sequences. We evaluate our APT-Agent on Metasploitable 2 against seven vulnerable services spanning web, database, and network protocols. APT-Agent achieves an 84.29% end-to-end exploitation success rate, compared to 48.57% (Script Kiddie) and 18.57% (PentestGPT) under matched conditions. By reducing cognitive burden and minimizing reliance on human intervention, APT-Agent represents a step toward scalable, reliable, and cognitively efficient automation for penetration testing.

Abstract

Automated Penetration Testing

Explore similar work

APT-Agent: Automated Penetration Testing using Large Language Models

Decoupling Reconnaissance and Exploitation: Measuring the Capability Boundaries of LLM-Based Web Penetration Testing

Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing