On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

Daniel Rodriguez-Cardenas; David Nader Palacio; Anna Schmedding; Yiyang Lu; Aadil Mallick; Bill Hudson; Chris Gourley; Michael Roytman

cs.CRAug 22, 2026

On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

Authors: Daniel Rodriguez-Cardenas, David Nader Palacio, Anna Schmedding, Yiyang Lu, Aadil Mallick, Bill Hudson, Chris Gourley, Michael Roytman,

Organizations: William & Mary Williamsburg, VA, USA · Microsoft Seattle, WA, USA · Cisco Systems San Jose, CA, USA

Abstract

Modern software systems require earlier and more scalable vulnerability severity assessment to reduce exposure to high-impact security flaws. Security analysts typically assign CVSS scores, but this manual triage does not scale with the growth of disclosed vulnerabilities and often depends on cloud LLM services that raise confidentiality concerns. This paper presents an industrial case study on predicting CVSS v3.1 scores directly from vulnerable C/C++ snippets using in-context learning with locally deployable, open-source LLMs. We compare proprietary data with the Big-Vul dataset, showing sufficiently aligned CVSS distributions to justify Big-Vul as a proxy for industrial data when constructing prompt-based testbeds. We then vary in-context configurations and model parameters, evaluating CodeLlama2-7B, CodeLlama2-13B, Mistral-7B, gpt-oss, and GPT4o-mini using mean squared error (MSE) and feasibility metrics. Our results show that medium-sized open-source code models, particularly CodeLlama2-7B, can approximate the best cloud performance for CVSS regression when guided by lightweight, output-constraining prompts, offering a practical, privacy-preserving building block for severity triage in industrial settings.

Figures & tables

Fig. 1 : In-Context Learning Solution as a Severity Score Prediction Pipeline

Fig. 2 : In-Context Learning Configuration. $C_{3}$ aggregates a specific output format to answer the question compared to $C_{2}$ .

Panel of Figure 3 — Fig. 3 : Big-Vul dataset: on the left, the sequence size frequency; on the right, the CVSS score distribution for raw data and filtered dataset.

		Training	Test
Sequence Size	medium	43856	5524
Sequence Size	short	10425	1334
Documentation	Snippet + Summary	139607	17568
	Total	150908	18864

TABLE I : Open-data Big-Vul size after curation

Severity	Train Count	Test Count
None (0)	530	63
Low (0.1–3.9)	3853	472
Medium (4–6.9)	31386	3926
High (7–8.9)	14749	1889
Critical (9–10)	3763	508

TABLE II : Counts per severity impact for train and test datasets with combined sequence size

Panel of Figure 6 — Fig. 4 : Proprietary dataset frequencies: on the left, the KCVSS score, on the right, the assigned CVSS to the proprietary snippets.

Panel of Figure 7 — Fig. 5 : Propietary CVSS correlation with KCVSS and kenna risk, color represents the frequency.

Model	$C$	$T$	MSE Mean	Std. MSE	Gt. Mean	Gt. Std.	% Non Feasible
CodeLlama2 7B	$C_{1}$	0.001	7.91	13.01	6.14	1.82	0.00
	$C_{1}$	0.300	6.69	9.63	6.44	1.33	0.00
	$C_{2}$	0.001	8.36	12.74	6.13	1.83	0.50
	$C_{2}$	0.300	6.98	10.35	6.10	1.89	0.00
	$C_{3}$	0.001	10.00	16.21	6.06	1.88	4.00
	$C_{3}$	0.300	10.87	15.76	6.21	1.73	4.50

TABLE III : Mean Squared Error calculated for each model with sequences short and medium size combined with 3-shots, varying the temperature

T

$i$	$C$	Seq. Size	MSE Mean	Std. MSE	% Non Feasible	Gt. Mean	Gt. Std.	Ex. Mean
		short	18.46	20.48	0.34	6.36	1.79	-
	$C_{1}$	medium	11.38	14.93	2.34	5.96	2.00	-
		short	30.16	24.20	0.00	6.36	1.71	-
	$C_{2}$	medium	14.22	18.17	0.00	6.16	1.71	-
		short	10.18	12.80	0.00	6.47	1.92	-
0	$C_{3}$	medium	11.55	12.50	0.00	5.93	1.82	-

TABLE IV : Mean Squared Error calculated for

C_{1}

C_{2}

, and

C_{3}

among shots

i=0,1,3

and temperature

T=0.3

Seq. Size	$l$	Avg. MSE	Std. MSE	Non Feasible	Out-Of- Range	Ex. Mean	Ex. Std.	Gt. Mean	Gt. Std.
	0	19.88	6.51	22.61%	0.23%	-	-	6.08	1.79
	1	7.65	6.52	0.00%	0.10%	6.08	1.96	6.09	1.85
	2	8.20	6.68	0.00%	0.01%	5.95	1.89	6.19	1.84
medium	3	9.01	7.20	0.00%	0.04%	6.01	1.91	6.12	1.86
	0	32.08	4.99	16.00%	0.07%	-	-	6.11	1.70
	1	11.53	9.27	0.00%	0.19%	6.13	2.08	6.13	2.08

TABLE V : Mean Squared Error calculated for each model and sequence sizes with temperature of

T=0.9

Panel of Figure 11 — Fig. 6 : Confusion matrix for stratified CVSS classification using CodeLlama2-7B , CodeLlama2-13B and GPT4o-mini with $n=600$ datapoints.

Software Security

Software security studies how to find, prevent, validate, and repair vulnerabilities in code, systems, dependencies, and software supply chains.

See all 95 papers

Predicting CVSS and CWE from CVE Descriptions with Fine-Tuned Small Language Models
Oct 16, 2026Tara Dixit, Jessie Dong, Summer LiLLMs for Cybersecurity
Stanford University
VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection
May 10, 2026Wenxin Tang, Xiang Zhang, Junliang Liu +11Software Vulnerability Detection Static Code Analysis
Tsinghua University · Henan University · Dalian Maritime University +8
PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection
May 22, 2026Steffen J. Camarato, Yahya Hmaiti, Mandana Ghadamian +1Prompt Sensitivity LLM Auditing
University of Central Florida Orlando, Florida, USA

Oct 16, 2026J/K move · Enter open · S save

Predicting CVSS and CWE from CVE Descriptions with Fine-Tuned Small Language Models

Tara Dixit, Jessie Dong, Summer Li

Stanford University

LLMs for Cybersecurity

Open paper AskPDF arXiv

Until April 2026, the National Vulnerability Database labeled nearly every published Common Vulnerabilities and Exposures (CVE) entry with a Common Vulnerability Scoring System (CVSS) v3.1 vector, a severity tier, and a Common Weakness Enumeration (CWE) weakness type. Now, NIST enriches only the high priority CVEs, leaving many newly published vulnerabilities unlabelled with the classifiers that security teams use for patch triage. In CVE enrichment, we predict the eight base components of CVSS v3.1 vector, base score, severity tier, and CWE from the vulnerability's textual description alone. To prevent leakage between near-duplicate descriptions, all systems are trained only on CVEs published in 2022 and 2023. Results are evaluated on all 36,325 usable 2024 CVEs in the pinned NVD snapshot. All scores are either directly predicted, or calculated from predicted vectors using the CVSS formula. A fine-tuned 3.8B-parameter model Phi-3.5-mini, trained with LoRA in 2.5 hours on a single Apple M5 Pro chip, achieves 69.9% accuracy on severity and 70.9% on CWE. It matches the official vector exactly for 48.8% of test CVEs, outperforming TF-IDF logistic regression by 2.8 to 4.7 points on all accuracy metrics based on zero-exclusion paired bootstrap intervals. When we evaluate the base model with a zero-shot prompt, it achieves 20.8% accuracy on severity in a sample of 2,000 CVEs. In this same sample, its stated scores match its own vector precisely for only 3.4% of CVEs. However, deriving scores from predicted vectors boosts the critical recall of TF-IDF logistic regression from 50.7% to 73.3% with no additional cost. And since the errors of the fine-tuned model and TF-IDF affect different CVEs, at least one system matches the correct severity tier for 75.8% of test CVEs.

Abstract

Software Security

Explore similar work

Predicting CVSS and CWE from CVE Descriptions with Fine-Tuned Small Language Models

VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection

PromptAudit: Auditing Prompt Sensitivity in LLM-Based Vulnerability Detection