Autonomous Cyber Operations Gym for Training and Evaluating AI Models for Defence

Tobias Hürten; Johannes F. Loevenich; Florian Spelter; Erik Adler; Laurin Holz; Maxime Schwarzer; Thies Moehlenhof; Roberto Rigolin F. Lopes

cs.CROct 13, 2026

Autonomous Cyber Operations Gym for Training and Evaluating AI Models for Defence

Authors: Tobias Hürten, Johannes F. Loevenich, Florian Spelter, Erik Adler, Laurin Holz, Maxime Schwarzer, Thies Moehlenhof, Roberto Rigolin F. Lopes

Organizations: CortAIx Labs, Thales Deutschland, Ditzingen, Germany · Department of Mathematics/Computer Science, University of Osnabrück, Osnabrück, Germany · Department of Informatics, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · Department of Computer Science, University of Ulm, Ulm, Germany

Abstract

This paper proposes an Autonomous Cyber Operations (ACO) gym, a unified framework integrating high-fidelity network emulation, realistic user behaviour modelling for benign network traffic and adversarial profiles, and the generation of cybersecurity scenarios through open toolkits. By supporting live and deferred feature extraction at the packet and flow levels, the gym facilitates the creation of labelled synthetic datasets for training Deep Neural Network (DNN)-based Intrusion Detection Systems (IDSs), as well as developing and evaluating modular components for autonomous defence agents through emulated interactions. While evaluation of a DNN classifier within the gym shows high in-domain performance, achieving near-perfect recall on adversarial network flows, our experiments reveal a significant sensitivity to domain shift, showing that even minor changes in network composition, such as variations in traffic protocol distribution, can lead to a catastrophic collapse in model transferability and detection performance.

Figures & tables

Fig. 1 : Architecture of the ACO gym.

Fig. 2 : Exemplary network topology of an enterprise cybersecurity scenario.

Fig. 3 : Dual data processing pipeline, supporting live (A) and decoupled (B) data processing.

Fig. 4 : Correlation matrix using Spearman’s rank correlation coefficient ( $\rho$ ) (numerical features).

Protocol	a	b	c	d
ARP	0.103%	0.135%	0.063%	0.013%
ICMP	0.048%	0.042%	0.025%	0.013%
IGMPv3	0.021%	0.025%	0.012%	0.003%
OSPF	0.027%	0.034%	0.016%	0.003%
TCP	79.452%	81.820%	79.887%	9.491%
UDP	20.349%	17.943%	19.996%	90.476%

TABLE I : Protocol and class distribution of each dataset.

Panel of Figure 6 — Fig. 5 : Cross-evaluation results: Performance with respect to the F1 score for each model under cross-evaluation ( 5(a) ), the confusion matrix of the baseline classifier under in-domain evaluation ( 5(b) ), relative F1 score of the baseline classifier when transferred to shifted datasets ( 5(c) ), and the aggregated false negatives under cross-evaluation ( 5(d) ).

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 355 papers

Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents
Mar 6, 2026Ondřej Lukáš, Jihoon Shin, Emilia Rivas +6Generalization in RL AI Agent Evaluation
Czech Technical University in Prague (CTU) · University of Texas at El Paso (UTEP) · Universidad Nacional de Cuyo (UNCUYO)
Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation
Mar 18, 2026Iakovos-Christos Zarkadis, Christos DouligerisNetwork Intrusion Detection Synthetic Data Generation
University of Piraeus, Athens, Greece · Dept. of Informatics, University of Piraeus, Piraeus, Greece
Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operations
Jul 30, 2026Konur Tholl, François Rivest, Mariam El Mezouar +2Autonomous Cyber Defense LLM-Guided RL
Department of Electrical and Computer Engineering, Royal Military College of Canada, Kingston, Canada · Department of Mathematics and Computer Science, Royal Military College of Canada, Kingston, Canada · Defence Research and Development Canada, Ottawa, Canada +1

Mar 6, 2026J/K move · Enter open · S save

Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents

Ondřej Lukáš, Jihoon Shin, Emilia Rivas, Diego Forni+5

Czech Technical University in Prague (CTU) · University of Texas at El Paso (UTEP) · Universidad Nacional de Cuyo (UNCUYO)

Generalization in RL AI Agent Evaluation LLMs for Cybersecurity

Open paper AskPDF arXiv

Autonomous offensive agents often fail to transfer beyond the networks on which they are trained. We isolate a minimal but fundamental shift -- unseen host/subnet IP reassignment in an otherwise fixed enterprise scenario -- and evaluate attacker generalization in the NetSecGame environment. Agents are trained on five IP-range variants and tested on a sixth unseen variant; only the meta-learning agent may adapt at test time. We compare three agent families (traditional RL, adaptation agents, and LLM-based agents) and use action-distribution-based behavioral/XAI analyses to localize failure modes. Some adaptation methods show partial transfer but significant degradation under unseen reassignment, indicating that even address-space changes can break long-horizon attack policies. Under our evaluation protocol and agent-specific assumptions, prompt-driven pretrained LLM agents achieve the highest success on the held-out reassignment, but at the cost of increased inference-time compute, reduced transparency, and practical failure modes such as repetition/invalid-action loops.

Abstract

Cybersecurity

Explore similar work

Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents

Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation

Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operations