Fine-tuning RoBERTa for CVE-to-CWE Classification: A 125M Parameter Model Competitive with LLMs

Until April 2026, the National Vulnerability Database labeled nearly every published Common Vulnerabilities and Exposures (CVE) entry with a Common Vulnerability Scoring System (CVSS) v3.1 vector, a severity tier, and a Common Weakness Enumeration (CWE) weakness type. Now, NIST enriches only the high priority CVEs, leaving many newly published vulnerabilities unlabelled with the classifiers that security teams use for patch triage. In CVE enrichment, we predict the eight base components of CVSS v3.1 vector, base score, severity tier, and CWE from the vulnerability's textual description alone. To prevent leakage between near-duplicate descriptions, all systems are trained only on CVEs published in 2022 and 2023. Results are evaluated on all 36,325 usable 2024 CVEs in the pinned NVD snapshot. All scores are either directly predicted, or calculated from predicted vectors using the CVSS formula. A fine-tuned 3.8B-parameter model Phi-3.5-mini, trained with LoRA in 2.5 hours on a single Apple M5 Pro chip, achieves 69.9% accuracy on severity and 70.9% on CWE. It matches the official vector exactly for 48.8% of test CVEs, outperforming TF-IDF logistic regression by 2.8 to 4.7 points on all accuracy metrics based on zero-exclusion paired bootstrap intervals. When we evaluate the base model with a zero-shot prompt, it achieves 20.8% accuracy on severity in a sample of 2,000 CVEs. In this same sample, its stated scores match its own vector precisely for only 3.4% of CVEs. However, deriving scores from predicted vectors boosts the critical recall of TF-IDF logistic regression from 50.7% to 73.3% with no additional cost. And since the errors of the fine-tuned model and TF-IDF affect different CVEs, at least one system matches the correct severity tier for 75.8% of test CVEs.

Assigning Common Weakness Enumeration (CWE) categories to Common Vulnerabilities and Exposures (CVE) records remains an important but largely manual step in vulnerability analysis. We study this task as a text classification problem and compare two modelling choices: a multi-class formulation that predicts a single CWE per CVE and a multi-label formulation that allows multiple assignments. Three transformer encoders (BERT Base, SecureBERT, and CySecBERT) are evaluated on three nested label spaces (83, 47, and 25 classes). Multi-class training achieves higher macro-F1 across all settings, although the gap to multi-label narrows from 21 to 2 percentage points as the label space shrinks. Post-hoc threshold optimisation on the multi-label side closes this gap on the 25-class setting. Confusion analysis shows that the dominant misclassification patterns follow the CWE hierarchy and are shared across all three encoders (Pearson $r > 0.92$), which suggests that the error structure is driven more by taxonomy design than by encoder choice. A hierarchy-relaxed evaluation that forgives within-family confusions raises macro-F1 from ${\sim}$81% to ${\sim}$90%, indicating that strict metrics understate branch-level classifier quality. CySecBERT achieves the strongest results overall, with statistically significant gains concentrated in the multi-label setting.

Large language models can generate functionally correct code that still contains security weaknesses, motivating repair pipelines that first diagnose a weakness type before deciding how to fix it. The Common Weakness Enumeration (CWE) provides a standardized vocabulary for such diagnoses, but asking an autoregressive language model to generate a CWE label and extracting it from the response raises questions about output validity, speed, and cost, as well as accuracy. We evaluate Jev, a decision model that instead selects directly from a declared set of candidates and returns a probability for each, against six open-weight autoregressive models and a frontier proprietary model, GPT-5.6-Sol, on a controlled 50-way CWE classification task over 1,916 CyberSecEval benchmark examples. Jev outperforms all six open-weight baselines on every classification and ranking metric, while its comparison with GPT-5.6-Sol depends on the metric: GPT-5.6-Sol achieves higher Top-1 accuracy and Macro-F1, whereas Jev achieves higher Top-3 and Top-5 accuracy and a nearly identical MRR, at $6.27\times$ lower median API latency and $55.9\times$ lower estimated API cost. We further build JevVibe, a diagnosis-guided repair agent that uses predicted CWE labels to repair code generated by Qwen2.5-Coder-32B-Instruct. With Jev providing the diagnosis, the agent increases the detector-measured security pass rate from 63.5% before repair to 70.7%, compared with 66.1% for LLM-guided repair. These results show that JevVibe is effective at improving the security of generated code, with Jev providing reliable and efficient CWE classification.