Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations

Alizishaan Khatri

cs.CRAug 17, 2026

Probing the Prefill: Detecting Code Vulnerabilities via Latent Activations

Authors: Alizishaan Khatri

Organizations: Wrynx Inc.

Abstract

LLM-based code generation is now embedded in mission-critical pipelines, but defenses against vulnerable output remain post-hoc -- static analyzers, fine-tuned classifiers, or an LLM judge that screen completed code, ignoring the generating model's own internal state. We test a narrower, directly measurable question: when an LLM reads a piece of C/C++ code as context, do its hidden activations already carry a signal about that code's vulnerability status? We extract last prefill token activations from four LLMs (Granite-4.1-8B, Qwen3.5-9B, Qwen3.6-27B, Gemma-4-12B) across three model families and train MLP probes on these activations. We evaluate them on four function-level C/C++ benchmarks (Devign, Big-Vul, Draper VDISC, PrimeVul). Our probes achieve 41.7% average F1 using 13.4--16.0M-parameter probes -- under 0.2% of base-model size. On Devign, the best probe (Qwen3.5-9B, 68.8% F1) matches the published fine-tuned-classifier SOTA (67.9%) despite reading only a frozen, general-purpose LLM's activations; on the harder, more imbalanced benchmarks (Big-Vul, Draper VDISC, PrimeVul) probes trail SOTA substantially. This is early evidence that a coding LLM's own representation of arbitrary code is informative about that code's vulnerability status, motivating further work toward lightweight, model-native vulnerability screening.

Figures & tables

Panel of Figure 1 — Fig. 1: Two-stage probing pipeline (cf. Fig. 2 of [ 8 ] ). Stage 1 (Extraction): each function $c$ passes through the frozen LLM $\mathcal{M}$ (dashed border denotes frozen weights) and its last-prefill-token activation $h(c)\in\mathbb{R}^{d}$ is cached with label $y$ . Stage 2 (Training): an MLP probe $f_{\theta}$ is trained on the cached $(h_{i},y_{i})$ pairs via a class-weighted binary cross-entropy loss (§ IV-E ); only $\theta$ is updated – $\mathcal{M}$ stays frozen.

Model	Params	$d$ (hidden dim)	Layers
Granite-4.1-8B	8B	4096	40
Qwen3.5-9B	9B	4096	32
Qwen3.6-27B	27B	5120	64
Gemma-4-12B	12B	3840	48

TABLE I: Target Model Specifications

Dataset	Size	Vuln. %	SOTA F1	Labeling
Devign [ 24 ]	27K	46%	67.9%	manual review
Big-Vul [ 25 ]	217K	5.8%	96.5%*	CVE/patch mining
Draper VDISC [ 26 ]	1.27M	$\sim$ 9.6%	60.9%	static-analyzer
PrimeVul [ 23 ]	236K	$\sim$ 3%	24.5%*	auto-labeling

TABLE II: Dataset Summary and Reported Binary-Classification SOTA

Layer	In	Out	Activation
1	$d$ (Table I )	2048	GELU + Dropout( $p{=}0.1$ )
2	2048	2048	GELU + Dropout( $p{=}0.1$ )
3	2048	512	GELU + Dropout( $p{=}0.1$ )
4	512	512	GELU + Dropout( $p{=}0.1$ )
5	512	64	GELU + Dropout( $p{=}0.1$ )
6	64	1	(raw logit; sigmoid applied by the loss)

TABLE III: MLP Probe Architecture (Shared Across Models; Only

d

Varies)

	Devign			Big-Vul			Draper VDISC			PrimeVul
Model	P	R	F1	P	R	F1	P	R	F1	P	R	F1
Granite-4.1-8B	58.15	75.63	65.74	43.26	46.00	44.59	24.15	41.48	30.53 †	27.48	15.12	19.51 †
Qwen3.5-9B	56.30	88.35	68.77	44.69	88.99	59.50 †	22.05	50.27	30.65 †	15.77	19.85	17.58
Qwen3.6-27B	58.38	70.28	63.78	43.73	52.73	47.81	23.33	45.78	30.91 †	23.80	15.30	18.63
Gemma-4-12B	58.10	66.76	62.13	42.50	88.89	57.50 †	22.24	48.47	30.49 †	23.35	15.48	18.62 †

TABLE IV: Probe Performance Across Models and Datasets (Precision / Recall / F1, %)

CWE Type ( $n$ )	Granite	Qwen3.5	Qwen3.6	Gemma
Benign / No CWE (6424)	98.8	98.6	98.9	98.3
CWE-119 (4478)	97.9	97.5	97.7	97.7
CWE-20 (3388)	98.1	97.8	98.0	97.8
CWE-399 (2889)	98.8	98.6	98.7	98.6
CWE-264 (2287)	98.1	98.3	98.3	98.6
CWE-416 (1559)	99.0	98.8	98.8	98.3

TABLE V: Per-CWE-Type Classification Accuracy (%) on Big-Vul, Top-5 CWE Types Plus the Benign/No-CWE Bucket

Model	Role	Params
Our Probe (4 models, Table III )	inline, generation-time	13.41–16.03M
CodeBERT [ 30 ]	fine-tuned classifier	125M
GraphCodeBERT [ 31 ]	fine-tuned classifier	125M
Llama Guard 3-8B [ 32 ]	external guard model	8B
Granite Guardian 3.x-8B [ 33 ]	external guard model	8B
WildGuard [ 8 ]	external guard model	7B

TABLE VI: Parameter Count: Probe vs. External Detection Models

Software Vulnerability Detection

Software vulnerability detection identifies insecure code, pinpoints weaknesses, and helps developers validate or prevent exploitable bugs.

See all 143 papers

Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs
May 28, 2026Alexander Sternfeld, Andrei Kucharavy, Ljiljana DolamicPrompt Sensitivity Adversarial Attacks on LLMs
IEM, HES-SO, Le Foyer, Techno-Pôle 1, Sierre, Switzerland · Cyber-Defence Campus, armasuisse Science and Technology, Thun, Switzerland
Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection
Jun 29, 2026Asif Shahriar, Hongyu Cai, Hadjer Benkraouda +2Software Vulnerability Detection Adversarial Attacks on LLMs
BRAC University · Purdue University · University of Illinois Urbana-Champaign
Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning
Mar 17, 2026Shenao Yan, Shan Jin, Shimaa Ahmed +4LLM Auditing Backdoor Detection
University of Connecticut · Visa Research

May 28, 2026J/K move · Enter open · S save

Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs

Alexander Sternfeld, Andrei Kucharavy, Ljiljana Dolamic

IEM, HES-SO, Le Foyer, Techno-Pôle 1, Sierre, Switzerland · Cyber-Defence Campus, armasuisse Science and Technology, Thun, Switzerland

Prompt Sensitivity Adversarial Attacks on LLMs Secure Code Generation

Open paper AskPDF arXiv

LLM-based coding assistants are seeing rapid adoption, offering substantial gains in developer productivity. As organizations increasingly ship code these agents produce, the security of that code becomes critical. Prior work has shown that minor prompt perturbations degrade the functional correctness of LLM-generated code, but whether they also compromise code security has remained unstudied. We apply token-level mutations to prompts across three models and five programming languages, and show that mutations as small as a single-character change can flip generated code from secure to vulnerable. Probing the models' hidden states reveals that this fragility is partially encoded in prompt representations, but unevenly so. Input-handling vulnerabilities, where the model omits validation or sanitization, are more predictable (mean AUC 0.753) than secure-defaults vulnerabilities, where insecure code stems from one local choice such as a weak algorithm or unsafe parameter (mean AUC 0.674). These results show that the threat model for LLM-assisted coding extends beyond prompt injection to ordinary prompt variation, and indicate that input-handling flaws can be caught before generation while secure-defaults flaws require intervention during decoding.

Abstract

Software Vulnerability Detection

Explore similar work

Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs

Words Speak Louder Than Code: Investigating Cognitive Heuristics in LLM-Based Code Vulnerability Detection

Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning