Can AI Lower the Barrier to Cybersecurity? A Human-Centered Mixed-Methods Study of Novice CTF Learning

Cathrin Schachner; Jasmin Wachter

cs.CRFeb 20, 2026

Can AI Lower the Barrier to Cybersecurity? A Human-Centered Mixed-Methods Study of Novice CTF Learning

Authors: Cathrin Schachner, Jasmin Wachter

Organizations: University of Klagenfurt Klagenfurt

Abstract

Capture-the-Flag (CTF) competitions serve as gateways into offensive cybersecurity, yet they often present steep barriers for novices due to complex toolchains and opaque workflows. Recently, agentic AI frameworks for cybersecurity promise to lower these barriers by automating and coordinating penetration testing tasks. However, their role in shaping novice learning remains underexplored. We present a human-centered, mixed-methods case study examining how agentic AI frameworks -- here Cybersecurity AI (CAI) -- mediates novice entry into CTF-based penetration testing. An undergraduate student without prior hacking experience attempted to approach performance benchmarks from a national cybersecurity challenge using CAI. Quantitative performance metrics were complemented by structured reflective analysis of learning progression and AI interaction patterns. Our thematic analysis suggest that agentic AI reduces initial entry barriers by providing overview, structure and guidance, thereby lowering the cognitive workload during early engagement. Quantitatively, the observed extensive exploration of strategies and low per-strategy execution time potetially facilitatates cybersecurity training on meta, i.e. strategic levels. At the same time, AI-assisted cybersecurity education introduces new challenges related to trust, dependency, and responsible use. We discuss implications for human-centered AI-supported cybersecurity education and outline open questions for future research.

Figures & tables

CTF Challenge	Description
All About that Base (Crypto, Beginner)	A reversing/encoding puzzle. The participant recovers the original flag by reversing encoding layers.
Viva la Log Fiesta (Forensics, Beginner)	The challenge involves analyzing a log file found on a hacked server.
The Room (Misc, Beginner)	A reverse-engineering/forensics challenge involving a labyrinth-like nested folder structure.
Bloated (Pwn, Beginner)	A C program for counting calories; reaching 1337 calories reveals the flag.

Table 1: CyberLeague Challenges employed in Ph.2 to target RQ1

CTF Challenge	Description
CorpoFeed (Pwn, Beginner)	Feedback service, non-invoked win-function, ret2win
Cyber-Gateway (Web, Easy)	Flask-app, restricted body contents for POST-requests
Nokia (Misc, Easy)	SMS that contains three hexadecimal payloads starting with //SCKL
KDF_Dream (Crypto, Medium)	Secure messenger program with different encryption and decryption methods (DH, KDF, PRF, OTP)
60-seconds-to-flag (Rev, Hard)	Windows .exe that opens a terminal and starts a countdown

Table 2: ACSC CTF Challenges employed in Ph.3 to target RQ1

Group	CorpoFeed	Cyber-Gateway	Nokia	KDF_Dream	60-sec…
All Levels	89% (18)	100% (7)	100% (10)	25% (4)	50% (4)
Beginner	50% (4)	-	100% (2)	-	-
Intermediate	100% (8)	100% (6)	100% (7)	33% (3)	50% (4)
Advanced	100% (3)	-	-	-	-
Expert	100% (3)	100% (1)	100% (1)	100% (1)	-
CA(I)thrin	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$

Table 3: Solve rates (

N=

rel. sample size)

Group	CorpoFeed	Cyber-Gateway	Nokia	KDF_Dream	60-sec…
All Levels	1.78 ( $\pm$ 1.76)	0.83 ( $\pm$ 1.07)	2.29 ( $\pm$ 3.1)	3.25 ( $\pm$ 1.71)	7.67 ( $\pm 9.07$ )
Beginner	3.0 ( $\pm 1.73$ )	-	5.12 ( $\pm$ 6.89)	-	-
Intermediate	1.69 ( $\pm 1.98$ )	0.95 ( $\pm 1.08$ )	1.67 ( $\pm 1.75$ )	4.0 ( $\pm 1$ )	7.67 ( $\pm 9.07$ )
Advanced	1.0 (–)	-	-	-	-
Expert	0.62 ( $\pm$ 0.53)	0.1 (–)	1.0 (–)	1.0 (–)	-
CA(I)thrin	7.0	1.0	1.0 †	10.0	4.0 †

Table 4: Mean Time to Solution [in h] (

\pm

std. deviation))

Group	CorpoFeed	Cyber-Gateway	Nokia	KDF_Dream	60-sec…
All Levels	4.38 ( $\pm 5.01$ )	1.71 ( $\pm 0.95$ )	2.66 ( $\pm 2.06$ )	2.25 ( $\pm 1.5$ )	5.0 ( $\pm 3.91$ )
Beginner	7.66 ( $\pm$ 6.66)	-	5.0 (–)	-	-
Intermediate	3.71 ( $\pm$ 5.02)	1.83 ( $\pm 0.98$ )	2.57 ( $\pm 1.92$ )	2.66 ( $\pm 1.25$ )	5.0 ( $\pm 3.92$ )
Advanced	1.0 (–)	-	-	-	-
Expert	3.5 ( $\pm$ 3.54)	1.0 (–)	1.0 (–)	1.0 (–)	-
CA(I)thrin	19.0	5.0	5.0	20.0	10.0

Table 5: Avg. Number of Attempts (

\pm

std. deviation)

Group	CorpoFeed	Cyber-Gateway	Nokia	KDF_Dream	60-sec…
All Levels	0.91 ‡ ( $\pm 1.55$ )	$0.37$ ( $\pm 0.31$ )	$0.76$ ( $\pm 0.52$ )	$1.65$ ( $\pm 0.91$ )	$1.24$ ( $\pm 1.52$ )
Beginner	$0.47$ ( $\pm 0.2$ )	-	2.0 (–)	-	-
Intermediate	1.29 ‡ ( $\pm 2.1$ )	0.4 ( $\pm 0.32$ )	0.55 ( $\pm 0.22$ )	1.87 ( $\pm 0.99$ )	1.24 ( $\pm 1.52$ )
Advanced	1.0 (–)	-	-	-	-
Expert	0.21 ( $\pm 0.06$ )	0.1 (–)	1.0 (–)	1.0 (–)	-
CA(I)thrin	0.36 †	0.2 †	0.2 †	0.5 †‡	0.4 †

Table 6: Mean Time per Attempt [in h] (

\pm

std. d.). ‡ Two extreme outliers, i.e. id 12, 13, were detected. Removing them yields a mean of

0.54

and empirical standard deviation

s=0.28)

for all levels and a mean of

0.61

and empirical standard deviation

s=0.24)

for intermediates. The primary subject did not solve this CTF.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

CTF Challenge Category	Description
Rev - Reverse Engineering	Reverse engineering tasks involve analyzing compiled programs to recover program logic and derive inputs or flags, typically using disassemblers and decompilers [ 25 ] .
Web	Web challenges target vulnerabilities in web applications and APIs (e.g., injection, authentication flaws) requiring exploitation to access data or execute code. Techniques align with the OWASP Top 10 [ 26 ] .
Crypto - Cryptography	Cryptography themed CTFs practice applied cryptography. Challenges demand using algorithms and theoretical concepts to recover secrets or break flawed constructions [ 21 ] .
Pwn - Binary Exploitation	Binary exploitation exercises involve finding and exploiting bugs in binaries (e.g., overflows) to gain code execution or read protected data [ 23 ] .
Forensic	Digital forensics is the systematic analysis of digital artifacts to reconstruct past activity. Tasks involve disk images, logs, and metadata, with flags often concealed in non-obvious locations [ 24 ] .
Misc - Miscellaneous	“Misc” includes problems that do not fit the other categories, ranging from scripting to steganography and data recovery. Misc tests breadth, creativity, and tooling fluency [ 35 ] .

Appendix

Table 7: CTF Challenge Categories

Figure 1: CorpoFeed: Time-to-completion –Histogram

Figure 2: CorpoFeed: Number of Attempts needed – Histogram

Figure 3: Cyber-Gateway: Time-to-completion –Histogram

Figure 4: Cyber-Gateway: Number of Attempts needed – Histogram

Figure 5: Nokia: Time-to-completion –Histogram

Figure 6: Nokia: Number of Attempts needed – Histogram

Figure 7: KDF_Dream: Time-to-completion –Histogram

Figure 8: KDF_Dream: Number of Attempts needed – Histogram

Figure 9: 60-seconds-to-flag: Time-to-completion –Histogram

Figure 10: 60-seconds-to-flag: Number of Attempts needed – Histogram

CTF Challenge	Number of Answers	Number of Trys
CorpoFeed	26	18
Cyber-Gateway	16	7
Nokia	14	10
KDF_Dream	13	4
60-seconds-to-flag	13	4

Appendix

Table 8: Participant Responsiveness to ACSC CTF Challenges. Numbers represent self-reported attempts.

Group	CorpoFeed	Cyber-Gateway	Nokia	KDF_Dream	60-seconds-to-flag
All Levels	$N_{S}=18$ , $N_{T}=13$ , $N_{A}=13$ , $N_{T/A}=13$ .	$N_{S}=7$ , $N_{T}=7$ , $N_{A}=7$ , $N_{T/A}=7$ .	$N_{S}=10$ , $N_{T}=10$ , $N_{A}=9$ , $N_{T/A}=9$ .	$N_{S}=4$ , $N_{T}=4$ , $N_{A}=4$ , $N_{T/A}=4$ .	$N_{S}=4$ , $N_{T}=3$ , $N_{A}=4$ , $N_{T/A}=3$
Beginner	$N_{S}=4$ , $N_{T}=3$ , $N_{A}=3$ , $N_{T/A}=3$ .	-	$N_{S}=2$ , $N_{T}=2$ , $N_{A}=1$ , $N_{T/A}=1$ .	-	-
Intermediate	$N_{S}=8$ , $N_{T}=7$ , $N_{A}=7$ , $N_{T/A}=7$ .	$N_{S}=6$ , $N_{T}=6$ , $N_{A}=6$ , $N_{T/A}=6$ .	$N_{S}=7$ , $N_{T}=7$ , $N_{A}=7$ , $N_{T/A}=7$ .	$N_{S}=3$ , $N_{T}=3$ , $N_{A}=3$ , $N_{T/A}=3$ .	$N_{S}=4$ , $N_{T}=3$ , $N_{A}=4$ , $N_{T/A}=3$ .
Advanced	$N_{S}=3$ , $N_{T}=1$ , $N_{A}=1$ , $N_{T/A}=1$ .	-	-	-	-
Expert	$N_{S}=3$ , $N_{T}=2$ , $N_{A}=2$ , $N_{T/A}=2$ .	$N_{S}=1$ , $N_{T}=1$ , $N_{A}=1$ , $N_{T/A}=1$ .	$N_{S}=1$ , $N_{T}=1$ , $N_{A}=1$ , $N_{T/A}=1$ .	$N_{S}=1$ , $N_{T}=1$ , $N_{A}=1$ , $N_{T/A}=1$ .	-

Appendix

Table 9: Relative Sample Sizes

N_{x}

from Tables 3 , 4 , 5 , 6 , where

N_{S}

is the Sample Size w.r.t. to Solve Rates (cf. Table 3 ),

N_{T}

– Time to Solution (cf. Table 4 ),

N_{A}

– Number of Attempts (cf. Table 5 ) and

N_{T/A}

– Time per Attempt (cf. Table 6 )

Definition [Code]	Example Indicators
	1. Entry Barrier Codes[B]
Perceived Difficulty [B1]	“too advanced”, “overwhelming”, “requires expert knowledge”.
Knowledge Gap [B2]	“I didn’t know where to start”, “I lacked experience with tools”.
Fear of Failure / Intimidation [B3]	“I was afraid I wouldn’t understand anything”.
Perceived Exclusivity [B4]	“It felt like something for specialists”.
	2. AI Mediation Codes [A]

Appendix

Table 10: Initial Codebook Overview

AI in Education

AI in Education studies how artificial intelligence can support teaching, learning, assessment, and educational decision-making.

See all 262 papers

STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving
Mar 23, 2026James Hugglestone, Samuel Jacob Chacko, Dawson Stoller +2LLMs for Cybersecurity LLM Agents
Department of Computer Science, Florida State University, Tallahassee, USA
CyberAGENTS: Structured Autonomy for Agentic Gamified Learning in Cybersecurity
Aug 8, 2026Ivan Hornung, Deepthi Marasinghe Arachchige, Tharindu Kumarage +4AI Agent Reliability Educational Technology
Arizona State University · Amazon AGI
The Disruptive Impact of Large Language Models on Capture the Flag Competitions and the Path Toward Fair Play
Jul 28, 2026Michael Macaulay, Harmony Bouabid, Guo Gen Ang +1Cybersecurity LLMs for Cybersecurity
WMG, University of Warwick, Coventry, United Kingdom

Mar 23, 2026J/K move · Enter open · S save

STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving

James Hugglestone, Samuel Jacob Chacko, Dawson Stoller, Ryan Schmidt+1

Department of Computer Science, Florida State University, Tallahassee, USA

LLMs for Cybersecurity LLM Agents Automated Penetration Testing

Open paper AskPDF arXiv

Large Language Models (LLMs) have demonstrated potential in code generation, yet they struggle with the multi-step, stateful reasoning required for offensive cybersecurity operations. Existing research often relies on static benchmarks that fail to capture the dynamic nature of real-world vulnerabilities. In this work, we introduce STRIATUM-CTF (A Search-based Test-time Reasoning Inference Agent for Tactical Utility Maximization in Cybersecurity), a modular agentic framework built upon the Model Context Protocol (MCP). By standardizing tool interfaces for system introspection, decompilation, and runtime debugging, STRIATUM-CTF enables the agent to maintain a coherent context window across extended exploit trajectories. We validate this approach not merely on synthetic datasets, but in a live competitive environment. Our system participated in a university-hosted Capture-the-Flag (CTF) competition in late 2025, where it operated autonomously to identify and exploit vulnerabilities in real-time. STRIATUM-CTF secured First Place, outperforming 21 human teams and demonstrating strong adaptability in a dynamic problem-solving setting. We analyze the agent's decision-making logs to show how MCP-based tool abstraction significantly reduces hallucination compared to naive prompting strategies. These results suggest that standardized context protocols are a critical path toward robust autonomous cyber-reasoning systems.

Abstract

AI in Education

Explore similar work

STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving

CyberAGENTS: Structured Autonomy for Agentic Gamified Learning in Cybersecurity

The Disruptive Impact of Large Language Models on Capture the Flag Competitions and the Path Toward Fair Play