Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

Pasquale Malacaria; Yunxiao Zhang

cs.CRAug 21, 2026

Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

Authors: Pasquale Malacaria, Yunxiao Zhang

Organizations: School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK. · Department of Computer Science, University of Exeter, Exeter, UK.

Abstract

Large language models (LLMs) are increasingly used in cybersecurity workflows, yet it remains unclear whether they can perform structured security reasoning or merely rely on superficial cues and prior knowledge. We study this question in the context of defence selection over attack graphs derived from real-world threat scenarios, including ransomware, supply-chain compromise, cloud abuse, Kubernetes attacks, POS malware, and ICS/OT intrusion. Given a budget constraint, LLMs must select security controls to minimise attacker success. We compare their strategies against each other and against a game-theoretic optimization baseline used as a normative reference for structured reasoning. Our results show that LLMs exhibit conditional competence. When explicit attack-graph structure is provided, they often produce coherent strategies close to the optimization baseline. However, their capabilities are fragile. LLM behaviour becomes increasingly fragile with graph complexity and is highly sensitive to framing. Small prompt changes can substantially alter rankings, and merely relabeling a poor strategy as ``optimal'' dramatically improves its evaluation. We further observe a non-monotonic relationship between formal risk and LLM judgement: strategies closest to the optimum are not necessarily ranked highest by LLM evaluators. To further probe reasoning ability, we ask LLMs to generate solvers for the same optimization problem. While the generated implementations recover the correct high-level formulation, they scale poorly compared to a purpose-built solver. Overall, our findings show that LLMs can approximate structured cybersecurity reasoning under controlled representations, but do not apply it robustly. This has important implications for the design and evaluation of AI-assisted security decision-support systems.

Figures & tables

Panel of Figure 2 — Figure 2: Ransomware attack graph with controls per attack step. Controls are grouped and abbreviated for readability.

Panel of Figure 3 — Figure 3: Structure of the prompt submitted to each LLM defender once per attack graph, covering all budget scenarios in a single call.

Figure 4: Risk of defenders across all budgets.

Panel of Figure 5 — Figure 5: Structure of the prompt submitted to each LLM evaluator once per attack graph, covering all budget scenarios in a single call.

Defender	Exc.	V.G.	Good	Avg.	Bad	V.Bad	Score
Optimal	43.5%	21.3%	16.7%	11.1%	6.5%	0.9%	4.82
Gemini	33.3%	28.7%	17.6%	11.1%	6.5%	2.8%	4.63
Grok	37.0%	15.7%	23.1%	16.7%	4.6%	2.8%	4.56
ChatGPT	31.5%	24.1%	16.7%	20.4%	6.5%	0.9%	4.51
Claude	29.6%	19.4%	18.5%	20.4%	6.5%	5.6%	4.29
Coverage	24.1%	14.8%	18.5%	16.7%	22.2%	3.7%	3.91

Table 1: Classification frequency and average score of defender portfolios (108 evaluations per defender).

Graph ID	$\rho_{\text{mean}}$	$p$ -value
1	$-0.867$	$3.49\times 10^{-4}$
2	$-0.581$	$1.57\times 10^{-5}$
3	$-0.731$	$1.52\times 10^{-3}$
4	$-0.558$	$2.66\times 10^{-3}$
5	$-0.355$	$8.50\times 10^{-9}$
6	$-0.315$	$7.47\times 10^{-6}$

Table 2: Per-graph Spearman correlation results. Negative values indicate alignment, since lower formal risk corresponds to higher LLM evaluation.

Group	Mean gap
Group 1 (Graphs 1–4)	$+0.108$
Group 2 (Graphs 5–7)	$+2.188$

Table 3: Claude-as-defender rank gap by complexity group. Positive mean gap indicates under-rating relative to formal risk.

Defender	Budget Level
Defender	B1	B2	B3	B4
	(tight)	(moderate)	(comfortable)	(generous)
Optimal	4.71	5.43	4.75	4.29
ChatGPT	3.89	4.68	4.79	4.71
Gemini	4.64	4.82	4.46	4.58
Grok	4.82	4.82	4.57	3.92

Table 4: Mean Evaluation Score per Defender and Budget Level.

Figure 6: Classification Distribution per Evaluator.

Defender	Average Score Exp. 2	Average Score Exp. 3
Gemini	4.65	4.81
Optimal	4.59	4.74
ChatGPT	4.55	4.71
Grok	4.31	4.58
Claude	4.28	4.46
Greedy	3.92	4.55

Table 5: Aggregate evaluation scores for alternative experiments.

Evaluator	Mean $\Delta$	Median $\Delta$
ChatGPT	$+0.33$	$0.0$
Claude	$+0.11$	$0.0$
Gemini	$-0.26$	$0.0$
Grok	$-0.19$	$0.0$
Pooled	$\phantom{+}0.0$	$0.0$

Table 6: Self-preference effect by evaluator.

Condition	Mean $\Delta$	Median $\Delta$
OG (Optimal-label, Good strategy)	$+0.98$	$+1$
OM (Optimal-label, Misleading)	$+1.68$	$+1$
PG (Poor-label, Good strategy)	$-0.44$	$\phantom{+}0$

Table 7: Score difference for the target defender under each framing condition, relative to the baseline.

Evaluator	Exp. 1	Exp. 2	Exp. 3
ChatGPT	(3.78, 4.41, 33%)	(4.44, 4.41, 52%)	(4.72, 4.56, 59%)
Claude	(3.37, 4.82, 26%)	(4.37, 4.26, 44%)	(5.07, 4.35, 74%)
Gemini	(3.22, 4.78, 19%)	(4.59, 4.85, 37%)	(5.83, 4.82, 82%)
Grok	(3.37, 5.26, 26%)	(4.19, 4.85, 37%)	(5.56, 5.21, 67%)
All	(3.44, 4.82, 26%)	(4.40, 4.59, 43%)	(5.30, 4.74, 71%)

Table 8: Comparison of name-bias effects across three experiments. Each tuple reports ( Mean OM , Mean OPT , OM wins ), where OM denotes the poor strategy labeled optimal and OPT denotes the genuinely optimal defender.

Panel of Figure 7 — Figure 7: Structure of the prompt submitted to each LLM reconstructor once per (graph, variant) pair.

Panel of Figure 8 — Figure 8: Structure of the prompt submitted to each LLM judge for the reconstruction-quality evaluation.

(a) Effect of context ablation
Metric	A	B	C	D	$\boldsymbol{\Delta_{A\to D}}$
Structural similarity	7.42	7.21	7.16	6.46	$-0.96$
Semantic similarity	7.38	7.42	6.99	5.72	$-1.66$
Attack-path quality	9.11	8.98	9.03	8.54	$-0.56$
Overall fidelity	7.57	7.53	7.26	6.03	$-1.55$

Table 9: Summary of reconstruction results. (a) shows the effect of context ablation, averaged over reconstructors and evaluators. (b) shows reconstructor capability, averaged over variants and evaluators.

Defender	Anon. controls, full graph	Anon. controls, no graph
Optimal	4.77	4.31
ChatGPT	4.83	4.05
Gemini	4.57	4.32
Grok	4.54	3.89
Claude	4.48	4.02
Greedy	3.77	5.42

Table 10: Average score under the two ablation conditions.

Panel of Figure 9 — Figure 9: Summary of the prompt submitted to each LLM to elicit a general solver for the decision problem.

	Small			Medium			Large
Nodes	10	20	30	40	50	60	70	80	90
Avg. edges	14.0	51.3	106.0	182.0	252.0	355.8	498.0	631.3	793.3
Controls	5	10	15	20	25	30	35	40	45

Table 11: Graph statistics by node count, with four graphs generated for each node count.

Figure 10: Execution Time: Optimal vs LLMs by Graph Size Group, LLM timeouts (T) counted as 1,500 seconds, log scale.

Explore similar work

Reasoning Topology Matters: A Controlled Study of LLM-Based Cybersecurity Analysis
Sep 21, 2026Jiling Zhou, Aisvarya Adeseye, Antti Hakkala +2LLMs for Cybersecurity LLM Reasoning with Graphs
Department of Computing University of Turku Turku, Finland
Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Framework
Apr 6, 2026Jiling Zhou, Aisvarya Adeseye, Seppo Virtanen +2Structured Prompting CoT Reasoning
Department of Computing, University of Turku, 20014 Turku, Finland
Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment
Mar 8, 2026Neha Nagaraja, Hayretdin BahsiLLM Security Cybersecurity
School of Informatics, Computing, and Cyber Systems, Northern Arizona University, Flagstaff, USA · Department of Software Science, Tallinn University of Technology, Tallinn, Estonia

Sep 21, 2026J/K move · Enter open · S save

Reasoning Topology Matters: A Controlled Study of LLM-Based Cybersecurity Analysis

Jiling Zhou, Aisvarya Adeseye, Antti Hakkala, Seppo Virtanen+1

Department of Computing University of Turku Turku, Finland

LLMs for Cybersecurity LLM Reasoning with Graphs Cybersecurity

Open paper AskPDF arXiv

Large Language Models (LLMs) are increasingly used in cybersecurity, where accurate analysis often requires multi-step and context-dependent reasoning over complex and heterogeneous data. However, existing prompting approaches typically focus on eliciting reasoning without explicitly considering how intermediate reasoning steps are structurally organized. We introduce Security Reasoning Topology, which models reasoning through three representative structures: Linear, Branching, and Graph. To evaluate their effects, we conduct controlled experiments on three cybersecurity datasets covering MITRE ATT&CK network traffic, cyber threat intelligence (CTI), and CVE vulnerability analysis. We evaluate multiple LLMs, including Llama 2 (7B, 13B, 70B), GPT-5.1, and Mistral Large 3, while keeping task inputs consistent and controlling reasoning structure through system-level prompting. Results show that reasoning topology substantially affects performance: Graph reasoning achieves the highest overall accuracy, improving over few-shot prompting by 9.8-12.2 percentage points across datasets, while Branching provides a strong intermediate solution. The results further show that the effect of reasoning topology remains consistent across model families and scales, highlighting reasoning topology as an important design factor for LLM-based cybersecurity analysis.

Abstract

Explore similar work

Reasoning Topology Matters: A Controlled Study of LLM-Based Cybersecurity Analysis

Strengthening Human-Centric Chain-of-Thought Reasoning Integrity in LLMs via a Structured Prompt Framework

Where Do LLM-based Systems Break? A System-Level Security Framework for Risk Assessment and Treatment