A Cyber Range Evaluation of Autonomous Network Incident Response Agents

Jakob Nyberg; Teodor Sommestad; Andrei Buhaiu; Joakim Loxdal; Pontus Johnson; Mathias Ekstedt

cs.CRSep 15, 2026

A Cyber Range Evaluation of Autonomous Network Incident Response Agents

Authors: Jakob Nyberg, Teodor Sommestad, Andrei Buhaiu, Joakim Loxdal, Pontus Johnson, Mathias Ekstedt

Organizations: Department of Network and Systems Engineering at KTH Royal Institute of Technology in Stockholm, Sweden · KTH

Abstract

We test the performance of agents for automated network intrusion response in a cyber range intended for human operator training. The range implements an emulated networking environment with a variable network topology, red-team emulation and simulated user agents. The goal of the defensive agents is to prevent hosts in the network from being accessed by the red-team agent, while minimizing the availability costs induced from defensive measures. Alerts are generated using a SIEM platform and mapped to a data modeling language used by the agents. We test a combination of heuristic agents and policies learned using reinforcement learning. The learned policies are optimized to minimize the combined cost using a cyber attack simulator modeling the network. We found that the reinforcement learning agents were overall more efficient at defending the system than the heuristic policy, and that the performance depends highly on the policy of the adversary in combination with the simulated users.

Figures & tables

Fig. 1 : High-level rendering of the three steps we follow for sim-to-real development of automated cyber defense agents.

Fig. 2 : Conceptual flow chart over the sim-to-real defender agent development and evaluation process described in this work. First, a data model describing a network is created with a given modeling language and collected inventory data. The model is then used with a simulator, which may use simulated attackers, to optimize a defending agent policy. Finally, the agent is evaluated with the target network, containing both adversarial and neutral agents. A translation interface converts and classifies events from the network into the data model used by the defending agent, and translates the agent’s actions back to commands to be executed on network hosts.

Fig. 3 : Topology of the AIR-DELIVERY-SYSTEM24 (ADS-24) computer network represented as a graph. Windows hosts are denoted with green rectangles, and Linux host with orange ovals.

Fig. 4 : mal instance model based on Osquery data from the hosts “ntp” and “payroll” , with asset types and relations defined in CadsLang.

Fig. 5 : Example of an observation database for the defender agent, rendered as a table, containing facts about the host “files” . Predicates for attack steps are colored red, defense steps blue, asset relations green and CIA values yellow.

Panel of Figure 6 — Fig. 6 : Violin plots of return distributions from running defender agents in ADS-24 against Lore, with and without user agents present. Higher returns indicate desired behavior. Median value indicated as dashed line.

Agent Type	Attacker Policy	Users Present	Return	Attack Cost	Defense Cost
NoOp	Exploratory	False	-11020 ± 826	-11020 ± 826	0 ± 0
Heuristic	Exploratory	False	-11586 ± 1737	-1344 ± 360	-10242 ± 1520
Vejde	Exploratory	False	-12422 ± 955	-918 ± 227	-11503 ± 1037
Vejde w/ Noise	Exploratory	False	-7779 ± 5723	-3816 ± 3218	-3963 ± 2615
NoOp	Guided	False	-2904 ± 133	-2904 ± 133	0 ± 0
Heuristic	Guided	False	-4721 ± 514	-931 ± 125	-3790 ± 417

TABLE I : Mean returns, attack costs and defense costs with standard deviation for defender agents when evaluated in ADS-24. Note that the return is the sum of the attack and defense costs.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

RL	Value
Maximum Episode Length	300
Minimum Episode Length	120
Epochs per Batch	8
Number of Parallel Environments	16
Discount Factor	0.99
GAE $\lambda$	0.95

Appendix

TABLE II : Hyperparameters used for training Vejde agents.

Host Class	Availability	Integrity	Confidentiality
Time Server / NTP	4	5	1
Log Server	5	5	4
File Server	4	2	2
Domain Controller	5	5	2
Name Server	5	5	2
Web Server	2	4	1

Appendix

TABLE III : Confidentiality, integrity and availability priorities for different assets in the network.

Hostname	Type
timereporter	Time Server / NTP
ntp	Time Server / NTP
files	File Server
dc	Domain Controller
weborder	Web Server
mail	Mail Server

Appendix

TABLE IV : Classes of host that can appear in the network.

Panel of Figure 11 — Fig. 7 : Scatter plots of returns over time after running defender agents in ADS-24 against Lore, with and without user agents present. Higher returns are better.

Fig. 8 : Agent returns for different prioritizations of attack and defense cost, calculated as

R_{\alpha}=(1-\alpha)R_{a}+\alpha R_{d}

, with

\alpha=0.5

representing the current balance.

Fig. 9 : Graph depiction of the mal CadsLang language. Nodes represent attack steps, with directed edges showing possible subsequent steps.

Panel of Figure 14 — Fig. 10 : Graphs showing estimated probability of Lore compromising a host when not interrupted with the two strategies, “Guided” and “Exploratory” . Probabilities $<$ 1% are not drawn.

Fig. 11 : Bipartite graph representation of an initial MAL Simulator state, converted to the Vejde representation format.

Fig. 12 : Bipartite graph representation of an initial observed CRATE state.

Autonomous Cyber Defense

Autonomous cyber defense uses AI agents to detect, investigate, and contain attacks while balancing operational safety, latency, and human oversight.

See all 72 papers

NetForge RL: A Multi-Agent Simulation Environment for Cyber Defense with Durative Actions
Apr 10, 2026Igor JankowskiAutonomous Cyber Defense Cybersecurity
Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents
Jun 16, 2026Ankita Samaddar, Sandeep Neema, Daniel Balasubramanian +1Opponent Modeling Autonomous Cyber Defense
Departmaent of Computer Science Vanderbilt University Nashville, TN
The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs
Sep 15, 2026Jakob Nyberg, Sandor Berglund, Andrei Buhaiu +3Cybersecurity Autonomous Cyber Defense
KTH Royal Institute of Technology

Apr 10, 2026J/K move · Enter open · S save

NetForge RL: A Multi-Agent Simulation Environment for Cyber Defense with Durative Actions

Igor Jankowski

Autonomous Cyber Defense Cybersecurity AI Agent Benchmarks Multi-Agent Reinforcement Learning

Open paper AskPDF arXiv

Training reinforcement-learning agents for cyber defense requires an environment that reflects the operational setting: noisy, partial observations, several defenders coordinating across a network, and an adaptive adversary realized through self-play. We present NetForge RL, a multi-agent environment for this setting on procedurally generated enterprise and operational-technology (OT) networks. A red agent compromises hosts with partial observability; three zone-split blue agents defend from synthetic SIEM telemetry, Windows/Sysmon event logs encoded into dense embeddings rather than a ground-truth state vector. The environment follows the PettingZoo parallel API with fixed-shape observations and MITRE ATT&CK-mapped actions, ships five scenarios with named difficulty presets and a held-out evaluation split, and replays deterministically under a seed. A JAX backend vectorizes a reduced transition core, reaching 2.5 x 10^5 environment-steps/s at batch 4096 on CPU, as a fast surrogate for training-loop iteration. Alongside the environment we provide reference baselines (scripted, a JAX IPPO trainer, and a self-play tournament), six diagnostic probes that each measure one defensive skill, and an evaluation runner reporting 95% confidence intervals. We describe the reproducibility engineering behind the environment and include a responsible-use statement.

Abstract

Autonomous Cyber Defense

Explore similar work

NetForge RL: A Multi-Agent Simulation Environment for Cyber Defense with Durative Actions

Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents

The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphs