Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs

Md Tanvirul Alam; Aritran Piplai; Ionut Cardei; Nidhi Rastogi; Peter J Worth

cs.LGJan 31, 2026

Minerva: Reinforcement Learning with Verifiable Rewards for Cyber Threat Intelligence LLMs

Authors: Md Tanvirul Alam, Aritran Piplai, Ionut Cardei, Nidhi Rastogi, Peter J Worth

Organizations: Rochester Institute of Technology, Rochester, NY, USA · University of Texas at El Paso, El Paso, TX, USA · Florida Atlantic University, Boca Raton, FL, USA

Abstract

Cyber threat intelligence (CTI) analysts routinely convert noisy, unstructured security artifacts into standardized, automation-ready representations. Although large language models (LLMs) show promise for this task, existing approaches remain brittle when producing structured CTI outputs and have largely relied on supervised fine-tuning (SFT). In contrast, CTI standards and community-maintained resources define canonical identifiers and schemas that enable deterministic verification of model outputs. We leverage this structure to study reinforcement learning with verifiable rewards (RLVR) for CTI tasks. We introduce Minerva, a unified dataset and training pipeline spanning multiple CTI subtasks, each paired with task-specific verifiers that score structured outputs and identifier predictions. To address reward sparsity during rollout, we propose MinervaRL, an auxiliary self-training mechanism that generates additional verified trajectories and distills them back into the model. Averaged across four backbones and 12 CTI benchmarks, MinervaRL improves the mean score by 15.8 percentage points over the corresponding base models and by 4.3 points over GRPO. The project page is available at https://maveryn.github.io/minerva-tmlr/.

Figures & tables

Panel of Figure 1 — Figure 1: Representative CTI mapping from a CVE description to structured labels and scores.

Figure 2: MinervaRL overview separating the standard GRPO loop from the auxiliary ACR path. Solid arrows show the synchronous training flow, while dashed arrows show the periodic ACR distillation pass.

Model	CKT	CyberMetric	SOCEval	RCM	VSP	ATE	RMS	ElasticRule	APTNER	LANCE	AnnoCTR	AZERG	Avg.
Llama-3.1-8B-Instruct	67.6	83.2	64.8	48.4	76.0	17.4	6.7	14.4	33.5	78.2	50.7	42.8	48.6
Llama-Primus-Merged	76.5	85.9	68.0	56.0	72.6	33.6	7.8	27.3	22.0	59.4	51.8	40.9	50.2
Foundation-Sec-8B-Instruct	77.1	81.3	67.9	61.0	65.8	39.2	15.4	33.6	34.0	59.5	43.5	44.4	51.9
Llama-3.1-8B-STaR-CTI	70.0	81.8	61.7	57.4	73.8	29.2	9.6	21.3	33.1	80.0	46.5	32.1	49.7
Llama-3.1-8B-DART-CTI	71.3	82.4	61.5	64.0	73.0	37.8	27.7	31.0	34.0	74.8	47.1	39.3	53.7
Llama-3.1-8B-GRPO	71.9	85.4	63.0	66.3	82.6	32.0	30.9	32.2	32.7	86.2	49.5	39.5	56.0

Table 1: Benchmark results across 12 CTI evaluation tasks. We compare MinervaRL with matched base models, GRPO, public security-SFT baselines, and STaR-CTI, DART-CTI, and LUFFY-CTI baselines. Underlined values indicate the best score within each backbone group.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+2.0 [0.6, 3.5]	-0.2 [-1.6, 1.1]	+8.0 [6.5, 9.5]	-4.2 [-5.8, -2.6]
CyberMetric	-1.1 [-2.5, 0.3]	+0.3 [-1.1, 1.5]	+15.5 [13.5, 17.5]	-5.7 [-7.4, -4.0]
SOCEval	+1.7 [-0.4, 3.8]	-3.7 [-5.8, -1.4]	-1.4 [-2.7, -0.1]	+0.2 [-1.6, 2.1]
RCM	+2.4 [1.1, 3.8]	+8.3 [6.8, 9.8]	+4.3 [2.9, 5.8]	-0.9 [-2.1, 0.2]
VSP	+5.0 [4.3, 5.7]	+20.6 [18.9, 22.4]	+10.6 [9.7, 11.7]	-8.1 [-9.0, -7.3]
ATE	+16.4 [12.6, 20.4]	+16.6 [13.2, 20.2]	+8.4 [5.8, 11.4]	+5.2 [2.8, 7.6]

Table 2: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus GRPO, in percentage points. Positive intervals indicate tasks where MinervaRL is higher than GRPO under the paired bootstrap interval.

Baseline	Point wins	Point losses	CI Pos.	CI Neg.	CI Overlap
Base	42/48	6/48	35/48	2/48	11/48
STaR	38/48	10/48	35/48	6/48	7/48
DART	38/48	10/48	34/48	3/48	11/48
GRPO	34/48	14/48	28/48	7/48	13/48
LUFFY	32/48	16/48	20/48	6/48	22/48

Table 3: Pairwise task-count summary for MinervaRL versus each baseline across 48 backbone-task comparisons. CI Pos. and CI Neg. are counts of comparisons whose paired 95% confidence interval is entirely above or below zero.

	QA / selection (n=12)		Taxonomy mapping (n=16)		Vulnerability scoring (n=4)		Information extraction (n=16)
Baseline	Point W/L	CI +/-/0	Point W/L	CI +/-/0	Point W/L	CI +/-/0	Point W/L	CI +/-/0
Base	9/3	7/0/5	15/1	15/1/0	4/0	4/0/0	14/2	9/1/6
STaR	9/3	9/2/1	15/1	15/1/0	3/1	3/0/1	11/5	8/3/5
DART	10/2	9/1/2	15/1	15/0/1	4/0	4/0/0	9/7	6/2/8
GRPO	6/6	3/4/5	15/1	13/0/3	3/1	3/1/0	10/6	9/2/5
LUFFY	8/4	4/0/8	10/6	8/3/5	4/0	4/0/0	10/6	4/3/9

Table 4: Task-family count summary for MinervaRL versus each baseline. Counts are over backbone-task comparisons within each family. Point W/L reports MinervaRL point wins/losses; CI +/-/0 reports CI-positive, CI-negative, and CI-overlap counts.

Ablation	Validation			12 CTI evaluation tasks
	Minerva- Dev	AthenaBench- Mini	Avg.	Avg.	CI Pos.	CI Neg.	CI Overlap
GRPO	50.3	57.4	53.8	56.0	8	0	4
GRPO 12 rollouts	50.9	52.7	51.8	53.9	7	1	4
Answer-only SFT	58.6	38.7	48.7	52.6	7	3	2
In-loop answer SFT	58.3	57.8	58.0	57.8	6	4	2
No EMA teacher	63.1	62.2	62.6	57.2	8	0	4

Table 5: Llama-3.1-8B component ablations on validation splits and the 12 CTI evaluation tasks. Validation columns report Minerva-Dev and AthenaBench-Mini scores. Evaluation columns report the 12-task macro average and paired bootstrap interval sign counts across the 12 tasks. Underlined values indicate the best score in each validation or 12-task average column.

Figure 3: GPT-5.2 pairwise response-quality preferences for backbone-specific model groups. Each cell reports the probability that the row model is preferred over the column model, with ties retained in the denominator.

	Llama-3.1-8B		Llama-3.2-3B		Qwen3-8B		Qwen3-4B
Model	Aligned	Not-in-train	Aligned	Not-in-train	Aligned	Not-in-train	Aligned	Not-in-train
Base	37.1	54.4	4.9	47.2	35.4	38.7	33.2	24.8
GRPO	53.0	57.5	31.5	47.4	40.3	50.3	43.2	49.8
MinervaRL	61.7	59.5	46.4	49.3	49.1	55.5	42.6	50.7

Table 6: Average performance on training-aligned and not-in-training evaluation tasks.

	Training-aligned					Not-in-training
Comparison	Point wins	Point losses	CI Pos.	CI Neg.	CI Overlap	Point wins	Point losses	CI Pos.	CI Neg.	CI Overlap
MinervaRL vs Base	15/16	1/16	15/16	1/16	0/16	27/32	5/32	20/32	1/32	11/32
MinervaRL vs GRPO	14/16	2/16	13/16	1/16	2/16	20/32	12/32	15/32	6/32	11/32

Table 7: Task-count summary for training-aligned and not-in-training tasks among the 12 CTI evaluation tasks. Counts are over backbone-task comparisons. Aligned tasks are RCM, VSP, ATE, and RMS; not-in-training tasks are the remaining eight tasks. CI Pos./Neg./Overlap indicate whether the paired bootstrap 95% confidence interval for MinervaRL minus the comparison model is entirely above zero, below zero, or overlaps zero.

Figure 4: Fraction of prompts whose rollout group has maximum verifier reward

=0

. Each panel compares GRPO and MinervaRL; Llama-3.1-8B also includes GRPO-12 (

N=12

). Curves use a 5-step rolling mean.

Figure 5: Validation accuracy over training. Curves are 5-step rolling means; dashed lines mark the best GRPO accuracy, and crosses mark the first MinervaRL step that reaches it.

$k$	Point wins	Point losses	CI Pos.	CI Neg.	CI Overlap
1	32/48	16/48	27/48	10/48	11/48
2	36/48	12/48	26/48	6/48	16/48
4	38/48	10/48	31/48	4/48	13/48
8	39/48	9/48	31/48	6/48	11/48

Table 8: Rollout-aware task-count summary for MinervaRL versus GRPO across 48 backbone-task comparisons at each best-of-

k

budget. CI Pos. and CI Neg. are counts of comparisons whose paired 95% confidence interval is entirely above or below zero.

Model	MMLU Pro $\uparrow$	IFEval $\uparrow$	Canary Exploit $\uparrow$	WMDP Cyber $\downarrow$
Llama-3.1-8B	44.1	72.3	10.9	46.7
Llama-Primus	41.5	68.2	29.8	46.3
Foundation-Sec	42.5	67.1	0.0	45.3
Llama-3.1-8B-GRPO	45.6	72.3	0.0	47.1
Llama-3.1-8B-MinervaRL	43.6	73.8	28.4	47.1

Table 9: Additional evaluations: Llama-3.1-8B general/cyber benchmarks and Qwen2.5-Coder-3B text-to-SQL transfer. Higher is better except WMDP-Cyber. Underlined values indicate the best score in each column.

$\gamma$	Minerva-Dev	Athena-Mini	Avg.
0.01	60.3	61.3	60.8
0.02	63.1	63.3	63.2
0.05	63.2	63.3	63.3
0.10	60.1	58.1	59.1

Table 10: Distillation scale

\gamma

sensitivity.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Target	Train	Val
Mappings-Explorer CVE $\rightarrow$ ATT&CK Exploitation	ATT&CK technique ID	245	20
Mappings-Explorer CVE $\rightarrow$ ATT&CK Primary Impact	ATT&CK technique ID	210	20
Mappings-Explorer CVE $\rightarrow$ ATT&CK Secondary Impact	ATT&CK technique ID	64	10
Sigma $\rightarrow$ ATT&CK Tactics	ATT&CK tactic IDs	950	50
Sigma $\rightarrow$ ATT&CK Technique	ATT&CK technique ID	950	50
Atomic Red Team $\rightarrow$ ATT&CK Technique	ATT&CK technique ID	950	50

Appendix

Table 11: Minerva training datasets and split sizes.

Panel of Figure 6 — Figure 6: Answer-conditioned rationale (ACR) prompt template used to elicit training traces.

Figure 7: Acceptance rates of the ACR self-distillation pipeline over training for Llama-3.1-8B-Instruct: fraction of batches passing the heuristic filter, passing the machine-learning (ML) filter, and meeting unique identifier (UID) coverage.

Panel of Figure 8 — Figure 8: Prompt template used to label reward-correct responses as GOOD / BAD for training the TextCNN rationale-quality classifier.

	Predicted GOOD	Predicted BAD
Actual GOOD	5,375	1,025
Actual BAD	1,206	5,194

Appendix

Table 12: Validation confusion matrix for the TextCNN reasoning-quality filter at threshold 0.5.

Rubric category	Count	Share of FP
Leakage	738	61.2%
Ungrounded	214	17.7%
Mismatch	159	13.2%
Incoherent	77	6.4%
Unsupported	18	1.5%

Appendix

Table 13: False positives by LLM-judge rubric category for the TextCNN filter at threshold 0.5.

Task	Count	Share of FN
scenario_to_attack_mitigations	392	38.2%
scenario_to_attack_technique	194	18.9%
cve_to_cwe	173	16.9%
scenario_to_attack_tactics	64	6.2%
sentinel_to_attack_technique	35	3.4%
sigma_to_attack_technique	35	3.4%

Appendix

Table 14: False negatives by task for the TextCNN filter at threshold 0.5.

Task	# Samples	Output format and evaluation target
CKT ( Alam et al., 2025 )	3000	Five-option CTI knowledge QA; output a single option A–E.
CyberMetric ( Tihanyi et al., 2024 )	2000	Four-option cybersecurity knowledge QA; output a single option A–D.
SOCEval ( Deason et al., 2025 )	588	Multi-select SOC-style reasoning over threat-intelligence reports; output a JSON object in <json_object> tags with a correct_answers array.
RCM ( Alam et al., 2025 )	2000	Root-cause mapping from CVE description to a single CWE identifier, e.g., CWE-#### .
VSP ( Alam et al., 2025 )	2000	Vulnerability severity prediction; output a full CVSS v3.1 base vector, e.g., CVSS:3.1/AV:N/... .
ATE ( Alam et al., 2025 )	500	ATT&CK technique extraction from an attack scenario; output a single technique or sub-technique ID, e.g., T#### or T####.### .

Appendix

Table 15: Evaluation datasets used for Table 1 .

Baseline	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
Base	+11.6 [10.8, 12.5]	+15.2 [14.5, 16.0]	+15.7 [14.5, 17.1]	+20.4 [19.0, 21.3]
STaR	+10.5 [9.7, 11.3]	+11.0 [10.3, 11.8]	+16.5 [15.4, 17.6]	+2.6 [1.5, 3.6]
DART	+6.6 [5.7, 7.5]	+4.3 [3.5, 5.1]	+10.5 [9.5, 11.8]	+8.6 [7.4, 9.7]
GRPO	+4.2 [3.5, 5.1]	+6.2 [5.3, 7.1]	+6.4 [5.6, 7.6]	+0.4 [-0.8, 1.3]
LUFFY	+3.3 [2.5, 4.1]	+2.9 [2.1, 4.0]	+0.1 [-0.6, 0.8]	+4.0 [3.0, 4.7]

Appendix

Table 16: Paired bootstrap 95% confidence intervals for MinervaRL minus each baseline on the 12-task macro average. The macro average is the unweighted mean over the 12 task metrics and is included for comparability with Table 1 .

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+6.3 [4.6, 8.0]	-0.6 [-1.9, 0.7]	+6.0 [4.5, 7.5]	+24.4 [22.4, 26.5]
CyberMetric	+1.1 [-0.3, 2.5]	+1.1 [-0.2, 2.4]	+18.9 [16.8, 21.0]	+29.9 [27.2, 32.5]
SOCEval	-0.1 [-1.9, 1.9]	-1.1 [-3.1, 0.9]	+3.8 [2.2, 5.4]	+7.9 [5.6, 10.1]
RCM	+20.4 [18.3, 22.5]	+42.0 [40.0, 44.2]	+12.2 [10.5, 14.0]	+14.3 [12.7, 16.1]
VSP	+11.6 [10.6, 12.7]	+74.3 [73.2, 75.5]	+10.4 [9.4, 11.3]	+3.5 [2.6, 4.4]
ATE	+31.0 [26.8, 35.4]	+20.8 [17.4, 24.2]	+18.6 [14.8, 22.6]	+21.2 [17.6, 24.8]

Appendix

Table 17: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus Base, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+3.9 [2.2, 5.5]	+6.7 [5.1, 8.3]	+39.8 [37.7, 41.9]	-6.1 [-7.8, -4.5]
CyberMetric	+2.4 [1.0, 3.9]	+7.6 [5.8, 9.5]	+38.9 [36.5, 41.3]	-8.2 [-10.0, -6.5]
SOCEval	+3.0 [1.3, 4.9]	+4.3 [2.1, 6.4]	+2.4 [1.0, 3.9]	-2.1 [-4.4, 0.2]
RCM	+11.4 [9.7, 13.2]	+20.8 [19.0, 22.7]	+28.1 [26.1, 30.3]	+38.8 [36.6, 41.0]
VSP	+13.8 [12.8, 15.0]	+23.7 [22.0, 25.4]	+10.7 [9.8, 11.6]	-0.2 [-1.2, 0.7]
ATE	+19.2 [14.6, 23.8]	+17.6 [14.0, 21.0]	+16.4 [12.8, 20.0]	+19.6 [16.2, 23.2]

Appendix

Table 18: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus STaR-CTI, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+2.6 [1.2, 4.1]	-2.0 [-3.2, -0.7]	+38.6 [36.6, 40.6]	+24.6 [22.5, 26.8]
CyberMetric	+1.9 [0.6, 3.1]	-0.1 [-1.4, 1.2]	+32.5 [30.1, 34.9]	+37.9 [35.3, 40.5]
SOCEval	+3.3 [1.4, 5.2]	+0.1 [-1.8, 1.9]	+2.4 [1.1, 3.8]	+4.0 [2.0, 6.1]
RCM	+4.7 [3.2, 6.3]	+2.3 [1.1, 3.7]	+17.7 [15.8, 19.6]	+5.2 [3.9, 6.7]
VSP	+14.6 [13.6, 15.7]	+15.9 [14.9, 17.0]	+6.7 [5.8, 7.6]	+6.2 [5.3, 7.2]
ATE	+10.6 [6.2, 15.2]	+5.0 [2.2, 7.6]	+18.0 [14.0, 22.0]	+4.8 [1.6, 8.2]

Appendix

Table 19: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus DART-CTI, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+2.0 [0.6, 3.5]	-0.2 [-1.6, 1.1]	+8.0 [6.5, 9.5]	-4.2 [-5.8, -2.6]
CyberMetric	-1.1 [-2.5, 0.3]	+0.3 [-1.1, 1.5]	+15.5 [13.5, 17.5]	-5.7 [-7.4, -4.0]
SOCEval	+1.7 [-0.4, 3.8]	-3.7 [-5.8, -1.4]	-1.4 [-2.7, -0.1]	+0.2 [-1.6, 2.1]
RCM	+2.4 [1.1, 3.8]	+8.3 [6.8, 9.8]	+4.3 [2.9, 5.8]	-0.9 [-2.1, 0.2]
VSP	+5.0 [4.3, 5.7]	+20.6 [18.9, 22.4]	+10.6 [9.7, 11.7]	-8.1 [-9.0, -7.3]
ATE	+16.4 [12.6, 20.4]	+16.6 [13.2, 20.2]	+8.4 [5.8, 11.4]	+5.2 [2.8, 7.6]

Appendix

Table 20: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus GRPO, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+0.0 [-1.4, 1.3]	+0.4 [-0.8, 1.5]	-0.9 [-1.9, 0.2]	+16.0 [13.9, 18.0]
CyberMetric	+0.6 [-0.7, 1.8]	-0.4 [-1.6, 0.8]	-0.1 [-1.2, 1.0]	+23.6 [21.1, 25.9]
SOCEval	+1.5 [-0.0, 2.9]	-0.8 [-2.4, 0.9]	+2.7 [1.0, 4.4]	+3.1 [1.1, 5.1]
RCM	+4.9 [3.7, 6.2]	+13.4 [11.9, 14.9]	-0.5 [-1.6, 0.6]	-0.0 [-1.2, 1.0]
VSP	+8.0 [7.2, 8.8]	+5.5 [4.5, 6.6]	+6.2 [5.5, 7.0]	+7.4 [6.5, 8.2]
ATE	+7.8 [4.2, 11.6]	+2.4 [-0.4, 5.4]	+2.6 [0.0, 5.2]	+7.8 [4.6, 11.0]

Appendix

Table 21: Per-task paired bootstrap 95% confidence intervals for MinervaRL minus LUFFY-CTI, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+0.0 [-1.5, 1.6]	-0.4 [-2.0, 1.2]	+7.0 [5.3, 8.7]	+9.4 [7.6, 11.3]
CyberMetric	-0.5 [-1.9, 0.8]	+1.3 [0.0, 2.6]	+2.8 [1.4, 4.3]	+28.6 [26.0, 31.1]
SOCEval	+2.2 [0.0, 4.3]	-5.1 [-7.4, -3.1]	-2.3 [-4.6, -0.2]	-0.3 [-2.5, 1.8]
RCM	+0.7 [-0.8, 2.2]	+7.5 [6.0, 8.9]	+4.0 [2.6, 5.4]	-0.7 [-1.9, 0.6]
VSP	+5.0 [4.3, 5.7]	+20.0 [18.4, 21.8]	+9.3 [8.4, 10.2]	-8.2 [-9.0, -7.3]
ATE	+16.0 [12.0, 19.8]	+16.8 [13.4, 20.4]	+8.6 [5.6, 11.8]	+3.8 [1.0, 6.6]

Appendix

Table 22: Per-task rollout-aware paired bootstrap 95% confidence intervals for MinervaRL minus GRPO at best-of-

1

, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+5.1 [3.7, 6.5]	+3.1 [1.8, 4.6]	+0.8 [-0.4, 2.1]	+1.0 [-0.4, 2.4]
CyberMetric	+0.7 [-0.5, 1.8]	+2.0 [0.8, 3.1]	+0.3 [-0.6, 1.4]	-0.3 [-1.7, 1.1]
SOCEval	+3.5 [1.4, 5.6]	-3.0 [-5.0, -0.9]	-0.7 [-2.4, 1.0]	+0.2 [-1.7, 2.1]
RCM	+3.1 [1.6, 4.6]	+9.1 [7.6, 10.7]	+5.5 [4.0, 7.1]	+1.0 [-0.1, 2.1]
VSP	+5.9 [5.3, 6.6]	+11.9 [10.6, 13.2]	+8.8 [7.9, 9.7]	-7.5 [-8.2, -6.6]
ATE	+19.4 [15.6, 23.2]	+20.2 [16.6, 24.2]	+8.6 [5.6, 11.8]	+4.6 [1.6, 7.4]

Appendix

Table 23: Per-task rollout-aware paired bootstrap 95% confidence intervals for MinervaRL minus GRPO at best-of-

2

, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+8.1 [6.8, 9.5]	+4.2 [3.1, 5.3]	-0.3 [-1.4, 0.8]	+0.6 [-0.6, 1.8]
CyberMetric	+1.1 [0.1, 2.1]	+2.3 [1.2, 3.4]	-0.4 [-1.3, 0.4]	-0.8 [-1.8, 0.2]
SOCEval	+6.9 [4.8, 9.2]	-0.6 [-2.6, 1.6]	+0.2 [-1.2, 1.6]	+0.6 [-1.2, 2.2]
RCM	+4.5 [3.1, 6.0]	+10.5 [9.0, 12.2]	+7.2 [5.8, 8.7]	+2.3 [1.2, 3.3]
VSP	+6.2 [5.6, 6.8]	+7.0 [6.1, 7.9]	+4.4 [3.6, 5.2]	-5.8 [-6.6, -5.1]
ATE	+20.4 [16.4, 24.2]	+23.2 [19.6, 27.0]	+10.0 [6.8, 13.4]	+7.0 [4.0, 10.0]

Appendix

Table 24: Per-task rollout-aware paired bootstrap 95% confidence intervals for MinervaRL minus GRPO at best-of-

4

, in percentage points.

Task	Llama-3.1-8B	Llama-3.2-3B	Qwen3-8B	Qwen3-4B
CKT	+9.4 [8.1, 10.7]	+6.6 [5.7, 7.7]	-1.7 [-2.6, -0.8]	+2.0 [1.0, 2.9]
CyberMetric	+1.9 [1.0, 2.9]	+3.1 [2.0, 4.2]	-0.7 [-1.5, -0.0]	+0.3 [-0.5, 1.0]
SOCEval	+8.5 [6.4, 10.6]	+1.0 [-1.1, 3.1]	+1.1 [-0.1, 2.2]	+0.5 [-1.1, 2.1]
RCM	+5.6 [4.1, 7.0]	+12.5 [10.9, 14.2]	+8.7 [7.2, 10.1]	+2.6 [1.5, 3.6]
VSP	+6.3 [5.7, 6.8]	+4.6 [4.0, 5.3]	-3.2 [-3.9, -2.6]	-0.9 [-1.5, -0.3]
ATE	+23.6 [19.8, 27.8]	+25.4 [21.4, 29.6]	+11.6 [8.2, 14.8]	+10.2 [7.2, 13.6]

Appendix

Table 25: Per-task rollout-aware paired bootstrap 95% confidence intervals for MinervaRL minus GRPO at best-of-

8

, in percentage points.

Backbone	$k$	CKT	CyberMetric	SOCEval	RCM	VSP	ATE	RMS	ElasticRule	APTNER	LANCE	AnnoCTR	AZERG
Llama-3.1-8B	1	71.3/71.4	85.7/85.2	63.2/65.4	66.5/67.2	82.4/87.4	30.8/46.8	31.3/38.9	31.7/40.3	32.6/32.5	87.0/81.6	52.9/47.6	39.3/42.5
Llama-3.1-8B	2	75.4/80.5	89.0/89.7	65.8/69.3	67.2/70.3	83.9/89.9	32.2/51.6	33.8/47.1	32.4/45.1	36.0/36.2	88.0/85.3	55.7/52.9	47.8/45.8
Llama-3.1-8B	4	77.9/86.0	91.4/92.5	69.7/76.7	68.4/72.9	85.1/91.3	35.0/55.4	35.0/52.9	34.0/50.5	38.6/40.0	89.2/87.8	58.8/57.4	49.7/50.8
Llama-3.1-8B	8	80.0/89.4	92.9/94.8	71.6/80.1	69.2/74.8	86.0/92.3	35.4/59.0	36.0/58.4	34.3/52.3	41.0/42.8	90.1/88.8	61.8/64.5	51.4/53.9
Llama-3.2-3B	1	68.6/68.2	77.7/79.0	60.5/55.4	47.9/55.5	54.6/74.6	5.4/22.2	17.1/28.2	5.6/20.6	26.8/15.1	66.2/76.5	36.9/42.5	25.0/43.0
Llama-3.2-3B	2	73.1/76.2	80.5/82.5	63.0/60.0	48.9/58.0	70.0/81.9	5.6/25.8	22.3/33.4	6.5/21.1	30.7/15.1	77.6/83.3	43.5/47.6	47.9/52.3

Appendix

Table 26: Full rollout-aware best-of-

k

task scores for GRPO and MinervaRL. Each task cell reports GRPO/MinervaRL in percent for the specified backbone and best-of-

k

budget.

Ablation	CKT	CyberMetric	SOCEval	RCM	VSP	ATE	RMS	ElasticRule	APTNER	LANCE	AnnoCTR	AZERG	Avg.
GRPO	71.9	85.4	63.0	66.3	82.6	32.0	30.9	32.2	32.7	86.2	49.6	39.5	56.0
GRPO 12 rollouts	71.2	85.0	64.6	66.8	76.3	31.4	17.0	31.7	36.3	82.6	48.8	35.4	53.9
Answer-only SFT	76.3	85.9	65.2	64.7	64.8	36.6	17.2	34.7	40.7	56.5	45.3	43.2	52.6
In-loop answer SFT	75.6	84.5	66.8	65.8	79.2	38.4	28.6	33.3	38.7	78.1	53.9	50.5	57.8
No EMA teacher	72.5	83.7	62.4	65.3	85.9	42.2	43.0	39.8	33.1	75.1	43.7	40.2	57.2
No TextCNN filter	71.2	85.3	63.6	64.0	78.6	45.0	36.5	41.4	36.0	82.5	47.5	41.1	57.7

Appendix

Table 27: Full Llama-3.1-8B component ablation scores on the 12 CTI evaluation tasks. All entries are task scores in percent.

Panel of Figure 9 — Figure 9: Prompt template and rubric used for GPT-5.2 and Claude Sonnet 4.6 pointwise response-quality scoring.

Comparison	QWK	Spearman
Annotator 1 vs. Annotator 2	0.6514	0.6461
Annotator 1 vs. GPT-5.2	0.6019	0.7822
Annotator 2 vs. GPT-5.2	0.5680	0.6440
Annotator average vs. GPT-5.2	0.6313	0.7722
Claude Sonnet 4.6 vs. GPT-5.2	0.7634	0.8266

Appendix

Table 28: Agreement on the response-quality validation subset. QWK denotes quadratic weighted kappa.

Figure 10: Target-level reward sparsity for ATE and RCM in DART-CTI stage-1 plain-generation attempts. Each point is one target identifier; the x-axis is mean attempts per question, the y-axis is mean verifier-successful responses per question, and color indicates the fraction of questions that reached the 32-attempt cap.

LLMs for Cybersecurity

Large language models for cybersecurity support threat analysis, vulnerability research, security operations, and cyber defense.

See all 272 papers

CyberThreat-Eval: Can Large Language Models Automate Real-World Threat Research?
Mar 10, 2026Xiangsen Chen, Xuan Feng, Shuo Chen +5LLM Evaluation Cyber Threat Intelligence
Microsoft Research · Hong Kong University of Science and Techonology (Guangzhou) · Microsoft +1
AuthRank: Scoring Source Credibility to Counter Poisoning in CTI LLM Pipelines
Oct 23, 2025Austin Jia, Avaneesh Ramesh, Zain Shamsi +2Cyber Threat Intelligence RAG Poisoning Attacks
Applied Research Laboratories, The University of Texas at Austin, Texas, USA
Retrieval-Constrained Policy Optimization for Attack Technique Extraction from Cyber Threat Intelligence
Aug 7, 2026Jiayun Zhang, Junshen Xu, Zejun Xie +1Cyber Threat Intelligence Cybersecurity
Amazon Web Services New York, New York, USA · Amazon Web Services Boston, Massachusetts, USA · Rutgers University Piscataway, New Jersey, USA

Mar 10, 2026J/K move · Enter open · S save

CyberThreat-Eval: Can Large Language Models Automate Real-World Threat Research?

Xiangsen Chen, Xuan Feng, Shuo Chen, Matthieu Maitre+4

Microsoft Research · Hong Kong University of Science and Techonology (Guangzhou) · Microsoft+1

LLM Evaluation Cyber Threat Intelligence Cybersecurity

Open paper AskPDF arXiv

Analyzing Open Source Intelligence (OSINT) from large volumes of data is critical for drafting and publishing comprehensive CTI reports. This process usually follows a three-stage workflow -- triage, deep search and TI drafting. While Large Language Models (LLMs) offer a promising route toward automation, existing benchmarks still have limitations. These benchmarks often consist of tasks that do not reflect real-world analyst workflows. For example, human analysts rarely receive tasks in the form of multiple-choice questions. Also, existing benchmarks often rely on model-centric metrics that emphasize lexical overlap rather than actionable, detailed insights essential for security analysts. Moreover, they typically fail to cover the complete three-stage workflow. To address these issues, we introduce CyberThreat-Eval, which is collected from the daily CTI workflow of a world-leading company. This expert-annotated benchmark assesses LLMs on practical tasks across all three stages as mentioned above. It utilizes analyst-centric metrics that measure factual accuracy, content quality, and operational costs. Our evaluation using this benchmark reveals important insights into the limitations of current LLMs. For example, LLMs often lack the nuanced expertise required to handle complex details and struggle to distinguish between correct and incorrect information. To address these challenges, the CTI workflow incorporates both external ground-truth databases and human expert knowledge. TRA allows human experts to iteratively provide feedback for continuous improvement. The code is available at GitHub and HuggingFace.

Abstract

LLMs for Cybersecurity

Explore similar work

CyberThreat-Eval: Can Large Language Models Automate Real-World Threat Research?

AuthRank: Scoring Source Credibility to Counter Poisoning in CTI LLM Pipelines

Retrieval-Constrained Policy Optimization for Attack Technique Extraction from Cyber Threat Intelligence