Organizations: NYU Tandon School of Engineering, New York, US · NYU Abu Dhabi, Abu Dhabi, UAE · NYU Abu Dhabi, Center for Cyber Security, Abu Dhabi, UAE
Large Language Models (LLMs) have shown significant improvement in RTL code generation. Despite the advances, the generated code is often riddled with common vulnerabilities and weaknesses (CWEs) that can slip by untrained eyes. Attackers can often exploit these weaknesses to fulfill their nefarious motives. Existing RTL bug-detection techniques rely on rule-based checks, formal properties, or coarse-grained structural analysis, which either fail to capture semantic vulnerabilities or lack precise localization. In our work, we bridge this gap by proposing an embedding-based bug-detection framework that detects and classifies bugs at both module and line-level granularity. Our method achieves about 89% precision in identifying common CWEs such as CWE-1244 and CWE-1245, and 96% accuracy in detecting line-level bugs.
Figures & tables
CWE
Description
CWE-250
Execution with unnecessary privileges, increasing attack impact.
Improper access control enabling unauthorized resource access.
CWE-310
Cryptographic issues due to weak or improper algorithms/usage.
CWE-321
Use of hard-coded cryptographic key in code.
CWE-506
Embedded malicious code (e.g., backdoor, trojan).
TABLE I: List of CWEs commonly used in the literature.
Fig. 1: Voting scheme determines the module-level CWEs and line-level bugs
Fig. 2: End-to-end VeriCWEty pipeline including data generation, embedding extraction, training, and inference
Class
Prec.
Rec.
F1
Sup.
CWE-1244
0.547
0.506
0.526
81
CWE-1245
0.431
0.383
0.405
81
CWE-1260
0.529
0.450
0.487
20
CWE-1271
0.000
0.000
0.000
3
CWE-284
0.915
0.667
0.771
81
CWE-310-AES-LEAK
0.822
0.914
0.866
81
TABLE II: Performance with initial annotation (baseline) using weak models
Model
Total Samples
Mismatches
Correct%
Deepseek-v3.2
50
34
32
LLaMA-3.3-70b-instruct
50
36
28
GPT-4o-mini
50
25
50
Gemini3
50
10
80
GPT-5-Nano
50
4
92
GPT-5.4
50
3
94
TABLE III: Number of mis-prediction of CWE by different models
CWE Type
# Train
Train (%)
# Test
Test (%)
CWE-1244
1106
38.05
276
37.96
CWE-1245
779
26.80
213
29.30
NONE
561
19.30
120
16.51
CWE-310-AES-LEAK
173
5.95
50
6.88
CWE-321
103
3.54
26
3.58
CWE-1271
55
1.89
16
2.20
TABLE IV: Distribution of CWE Types in Training and Test Datasets
Class
Prec.
Rec.
F1
Sup.
CWE-1244
0.886
0.870
0.878
276
CWE-1245
0.884
0.789
0.834
213
CWE-310-AES-DOS
0.556
0.556
0.556
9
CWE-310-AES-LEAK
0.576
0.680
0.624
50
CWE-310-CSR-UNAUTH
0.333
0.200
0.250
5
CWE-321
0.477
0.808
0.600
26
TABLE V: “Final performance after improved annotation and training using stronger latest models like GPT-5.4, GPT-5-Nano, and Gemini3 for labeling the dataset
Fig. 3: Line-level embeddings vs. Line-level + Module-level embeddings classification analysis. Starting from top left and going clock-wise (a) Metric Comparison (b) Precision-Recall Curve (c) Threshold Settings (d) Confusion Matrix for only Line-level embeddings for test (e) Confusion Matrix for Line-level combined with Module-level embeddings for test (f) FP/FN trade off
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences, Beijing, China · North China Electric Power University, Beijing 102206, China