DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation

Li Huang; Zhongxin Liu; Yifan Wu; Tao Yin; Dong Li; Jichao Bi; Nankun Mu; Hongyu Zhang

cs.SEApr 10, 2026

DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation

Authors: Li Huang, Zhongxin Liu, Yifan Wu, Tao Yin, Dong Li, Jichao Bi, Nankun Mu, Hongyu Zhang,

Organizations: Chongqing University · The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Peking University

Abstract

Large Language Models (LLMs) for code generation can replicate insecure patterns from their training data. To mitigate this, a common strategy for security hardening is to fine-tune models using supervision derived from the final transformer layer. However, this design may suffer from a final-layer bottleneck: vulnerability-discriminative cues can be distributed across layers and become less detectable near the output representations optimized for next-token prediction. To diagnose this issue, we perform layer-wise linear probing. We observe that vulnerability-related signals are most detectable in a band of intermediate-to-upper layers yet attenuate toward the final layers. Motivated by this observation, we introduce DeepGuard, a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module. The aggregated signal powers a dedicated security analyzer within a multi-objective training objective that balances security enhancement and functional correctness, and further supports a lightweight inference-time steering strategy. Extensive experiments across five code LLMs demonstrate that DeepGuard improves the secure-and-correct generation rate by an average of 11.9% over strong baselines such as SVEN. It also preserves functional correctness while exhibiting generalization to held-out vulnerability types. Our code is public at https://github.com/unknownhl/DeepGuard.

Figures & tables

Figure 1: Layer-wise diagnostic evidence on Seed-Coder-8B. We train a linear probe on each transformer layer to detect vulnerable patterns and report the probe confidence across layers. The vulnerability-discriminative signal peaks in intermediate-to-upper layers and attenuates toward the final layers.

Figure 2: Comparison of security guidance paradigms. (A) Single-layer guidance suffers from signal attenuation at the final layer. (B) DeepGuard (Ours) employs multi-layer aggregation to capture richer security-critical cues distributed across upper layers.

Figure 3: Overview of DeepGuard , depicting the multi-objective training phase and the guided inference phase.

Model	Method	pass@1 ( $\uparrow$ )		sec@1 ${}_{\textbf{pass}}$ ( $\uparrow$ )		sec-pass@1 ( $\uparrow$ )		SVEN-SR ( $\uparrow$ )
Model	Method	Value	Imp.(%)	Value	Imp.(%)	Value	Imp.(%)	Value	Imp.(%)
	Base	91.00	-4.78	76.47	+21.89	69.59	+16.05	77.95	+20.73
	Prompt	85.41	+1.45	72.93	+27.81	62.29	+29.65	75.84	+24.09
	SVEN	83.00	+4.40	84.90	+9.79	70.47	+14.60	82.60	+13.93
	SafeCoder	63.94	+35.52	82.34	+13.20	52.65	+53.39	87.02	+8.15
	CoSec	82.06	+5.59	76.85	+21.29	63.06	+28.07	78.35	+20.11

Table 1: Performance comparison across different models and methods. All metrics are reported as percentages (%). “Imp. (%)” columns show the relative improvement of DeepGuard (Ours) over other baselines.

Panel of Figure 4 — Figure 4: sec@1 ${}_{\textbf{pass}}$ on CWEs that do not appear in the training dataset.

Variant	pass@1	sec@1 ${}_{\textbf{pass}}$	sec-pass@1	SVEN-SR
DeepGuard ( $N=4$ )	86.59	93.21	80.71	93.21
Loss Component Ablation
(-) $\mathcal{L}_{\text{gen}}$ (Fluency)	84.53	93.04	78.65	93.09
(-) $\mathcal{L}_{\text{kl}}$ (Stability)	74.12	98.49	73.00	98.84
(-) $\mathcal{L}_{\text{sec}}$ (Security)	64.94	91.03	59.12	92.80
Inference Strategy Ablation

Table 2: Ablation study and sensitivity analysis of DeepGuard on Seed-Coder-8B. The green row denotes the default DeepGuard (attn.Pool

N=4

). Sections with pale green headers analyze specific components: training objectives (Loss), inference mechanisms, and multi-layer aggregation strategies.

Model	Method	pass@1	pass@5	pass@10	pass@25
Qwen2.5- Coder-3B	Base Model	52.4	–	–	–
	DeepGuard	56.0	62.5	64.3	66.0
	w/o inference	62.4	69.9	71.8	73.2
DeepSeek- Coder-1.3B	Base Model	34.8	–	–	–
	DeepGuard	24.5	28.9	30.2	31.6
	w/o inference	29.4	34.3	36.1	38.3

Table 3: Performance of HumanEval with or without DeepGuard ’s guided inference.

Method	Time (s)	Tokens/sec	Re-scores	Overhead
Default	6.886	43.57	1	DeepGuard
$k=64$	9.550	31.42	5	+38.7%
$k=16$	20.366	14.73	19	+195.8%
$k=4$	63.723	4.71	75	+825.4%
$k=1$	239.097	1.25	300	+3372.2%

Table 4: Latency of interval-based re-scoring for 300-token generation. Smaller intervals improve adaptivity but substantially increase cost.

Model	#L	Peak	Pos.	$\mathbf{P_{\text{peak}}\rightarrow P_{\text{final}}}$	Rel.	$p$
		layer	(%)		drop (%)
Seed-Coder-8B	32	9	29	0.9995 $\rightarrow$ 0.8574	14.2	$4.49\times 10^{-4}$
Qwen2.5-Coder-3B	36	9	26	0.8900 $\rightarrow$ 0.3326	62.6	$6.81\times 10^{-13}$
DeepSeek-Coder-1.3B	24	7	30	0.6607 $\rightarrow$ 0.4984	24.6	$1.22\times 10^{-6}$
DeepSeek-Coder-6.7B	32	22	71	0.7951 $\rightarrow$ 0.5485	31.0	$2.90\times 10^{-10}$
Qwen2.5-Coder-7B	28	27	100	0.8754 $\rightarrow$ 0.8754	0.0	$1.00$

Table 5: Cross-model summary of layer-wise probing.

Figure 5: Differential attention heatmap across the top-4 layers in Seed-Coder-8B. We visualize the $\Delta$ Attention ( $\alpha_{vul}-\alpha_{sec}$ ) for 82 validation pairs covering diverse CWEs. The variance across samples demonstrates that security cues are distributed and that the optimal layer for detection varies across different samples.

Panel of Figure 6 — Figure 6: Case study on command injection (CWE-78). (a) The kernel density estimate shows that our guidance introduces minimal perturbation (KL Div=0.1389), preserving the base model’s probability landscape. (b) The scatter plot reveals targeted steering: vulnerable tokens (e.g., ' f' for f-strings) are suppressed (negative shift), while secure tokens are boosted.

Figure 7: Sensitivity analysis of the loss weights on Seed-Coder-8B. Green line shows pass@

1

, red line shows sec-pass@

1

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Statistics of our training and validation dataset, adapted from He and Vechev (2023) .

CWE		Scenario	Split	Description	Unit Test
022:	path traversal	0-py	test	Read the user-requested file from the safe directory	Function call and args
		1-py	test	Return the user-requested image from /images	Function call and args
		2-py	val	Extract the tar file to /tmp/unpack	Check the path of every entry in the tar file
078:	OS command injection	0-py	test	List the content of a user-specified directory name	Function call and args
		1-py	test	Get the user info of a user-specified username
		2-py	val	Ping a user-specified URL

Appendix

Table 6: The 9 main CWEs and their scenarios used in our evaluation. Scenarios with the same text description differ in code. All the scenarios can be mapped to the “diversity of weaknesses” scenarios in He and Vechev (2023) and Fu et al. (2024) . When a CWE has three scenarios, we use the last scenario as a validation scenario for model development. We report evaluation results on the 18 test scenarios.

CWE		Scenario	Description	Unit Test
020:	improper input validation	1-py	Check URL validity and redirect to target	Check allowed and disallowed domains
		2-py	Load data and return in a different format	Check data matches structure and content
		3-py	Generate response includes a user-provided string	Verify response contains expected user string
119:	buffer overflow	0-c	Copy memory content from a string to a dest. buffer	Check printed content from dest. buffer
		1-c	Return the value in an array at a user-specified index	Use a test index, check the printed val
		2-c	Return the value in the args at a user-specified index	Use a test index, check the printed val

Appendix

Table 7: Test scenarios for 4 CWEs that are not included in training set. These scenarios are adapted from He and Vechev (2023) and Fu et al. (2024) .

Hyperparameter	Value
Training Dynamics
Epochs	5
Learning Rate	$2\times 10^{-5}$
Batch Size (Effective)	16
Per-Device Batch Size	8
Gradient Accumulation	2 steps

Appendix

Table 8: Summary of hyperparameters used for training and evaluating DeepGuard .

Figure 9: Computational overhead analysis. (a) Absolute GFLOPS required for aggregation scales linearly with

N

, while the analyzer cost is constant. (b) Relative overhead to the base model remains negligible (

<2.1\%

) for our chosen configuration of

N=4

Figure 10: Mechanistic visualization of DeepGuard processing an SQL Injection vulnerability. Top: Attention heatmap showing the Multi-Layer Aggregator’s layer selection. Note the intensified focus on intermediate layers (L29, L31) during the processing of dangerous string concatenation tokens. Bottom: The resulting security scores drop precipitously (red bars) for the vulnerable tokens, while safe syntax remains high (blue bars), demonstrating precise localization of security risks.

Layers N	pass@ $1$	sec@ $1_{\textbf{pass}}$	sec-pass@ $1$	sec_rate
N = 1	82.65	89.25	73.76	90.25
N = 2	86.00	93.07	80.24	93.04
N = 4	86.59	93.21	80.71	93.21
N = 6	87.47	93.28	81.59	93.26

Appendix

Table 9: Sensitivity analysis of the number of aggregated layers (

N

) on Seed-Coder-8B.

Temperature	pass@ $1$	sec@ $1_{\textbf{pass}}$	sec-pass@ $1$	SVEN-SR
T = 0.8	77.65	88.79	68.94	88.74
T = 0.4	82.24	90.84	74.71	91.63
Ours (T = 0.1)	86.59	93.21	80.71	93.21

Appendix

Table 10: Sensitivity analysis of the sampling temperature on Seed-Coder-8B.

Model	Qwen2.5-Coder-3B	Qwen2.5-Coder-7B	DeepSeek-Coder-1.3B	DeepSeek-Coder-6.7B	Seed-Coder-8B
Base	0.0331 $\pm$ 0.0010	0.0558 $\pm$ 0.0037	0.0192 $\pm$ 0.0011	0.0597 $\pm$ 0.0026	0.0854 $\pm$ 0.0070
Prompt	0.0337 $\pm$ 0.0007	0.0543 $\pm$ 0.0009	0.0192 $\pm$ 0.0010	0.0605 $\pm$ 0.0019	0.0650 $\pm$ 0.0023
SVEN	0.0334 $\pm$ 0.0007	0.0574 $\pm$ 0.0023	0.0187 $\pm$ 0.0013	0.0633 $\pm$ 0.0042	0.0936 $\pm$ 0.0087
SafeCoder	0.0335 $\pm$ 0.0010	0.0526 $\pm$ 0.0008	0.0192 $\pm$ 0.0019	0.0615 $\pm$ 0.0018	0.0600 $\pm$ 0.0012
CoSec	0.0510 $\pm$ 0.0013	0.0705 $\pm$ 0.0008	0.0407 $\pm$ 0.0029	0.1380 $\pm$ 0.0022	0.1670 $\pm$ 0.0099
CodeGuard+	0.0390 $\pm$ 0.0027	0.0566 $\pm$ 0.0010	0.0267 $\pm$ 0.0011	0.0697 $\pm$ 0.0019	0.0646 $\pm$ 0.0013

Appendix

Table 11: Average time (in seconds) to generate 20 tokens. Each value is an average of 5 runs.

Secure Indicators		Vulnerable Indicators
Token	Score	Token	Score
return	1.00	format	-1.00
if	1.00	None	-0.54
args	1.00	os	-0.45
NULL	0.99	sql	-0.42
_t	0.90	.system	-0.36

Appendix

Table 12: Top discriminative tokens identified by the lightweight prior

\mathbf{T}_{\text{stats}}

. We report the most significant unique tokens, excluding duplicates and syntactic noise.

Figure 11: Distribution of token values in

\mathbf{T}_{\text{stats}}

. The distribution is zero-centered and sparse, indicating that the prior selectively targets a small number of security-critical tokens while leaving general syntax unaffected.

Panel of Figure 12 — Figure 12: Detailed performance comparison across different CWE scenarios on Seed-Coder-8B. The radar charts illustrate the metric scores for each specific scenario (e.g., ‘089-0-py’).

Panel of Figure 13 — Figure 13: Detailed performance comparison across different CWE scenarios on Qwen-Coder-3B. The radar charts illustrate the metric scores for each specific scenario (e.g., ‘089-0-py’).

Secure Code Generation

Secure code generation studies how to produce code that meets functional requirements while avoiding security vulnerabilities.

See all 48 papers

Enhancing Reliability in LLM-Based Secure Code Generation
May 22, 2026Mohammed F. Kharma, Mohammad Alkhanafseh, Ahmed Sabbah +1LLM Evaluation CoT Reasoning
the Department of Computer Science, Birzeit University, Ramallah, Palestine · the Department of Computer Science, University of Central Florida, Orlando, FL 32816 USA
Interpreting and Steering for Safe and Correct Code Generation
Aug 30, 2026Hao Yan, Ziyu YaoMechanistic Interpretability LLM Security
Department of Computer Science George Mason University, Fairfax, VA
Secure and Functional Code Generation with Self-Evaluation
Oct 24, 2026Md Afif Al Mamun, Gias UddinSecure Code Generation Retrieval-Augmented Code Generation
University of Calgary, Calgary, Canada · York University, Toronto, Canada

May 22, 2026J/K move · Enter open · S save

Enhancing Reliability in LLM-Based Secure Code Generation

Mohammed F. Kharma, Mohammad Alkhanafseh, Ahmed Sabbah, David Mohaisen

the Department of Computer Science, Birzeit University, Ramallah, Palestine · the Department of Computer Science, University of Central Florida, Orlando, FL 32816 USA

LLM Evaluation CoT Reasoning LLM Security Secure Code Generation Prompt Engineering

Open paper AskPDF arXiv

Large language models (LLMs) are widely used for code generation, but their security reliability remains inconsistent across languages and prompting strategies. Existing prompt engineering improves functional correctness but rarely ensures consistent security outcomes. We introduce the Mitigation-Aware Chain-of-Thought (MA-CoT) framework, which embeds task-specific CWE mitigation guidance and language-aware safeguards to reduce recurring vulnerabilities in generated code. We evaluate MA-CoT across three LLMs (gpt-5, claude-4.5, gemini-2.5), three programming languages (C, Java, Python), and four prompting strategies (Vanilla, Zero-shot, CoT, MA-CoT) on a 200-task primary dataset, with external validation on LLMSecEval. Using static analysis with expert validation, MA-CoT reduces total security findings from 92 to 39 (57.6%) on the primary dataset and from 73 to 4 (94.5%) on LLMSecEval. High-severity findings (Blocker + Critical) drop from 90 to 39 (56.7%) and from 45 to 2 (95.6%), respectively. Across both datasets, MA-CoT is the only strategy that consistently improves security reliability; Zero-shot and CoT are less reliable and may increase vulnerability, especially in C. We further introduce a strict layered attribution of vulnerability drivers (language-core vs. stack layers) and show that residual risk concentrates in hardening-oriented patterns (e.g., OS- and toolchain-dependent), motivating secure-by-construction primitives alongside prompting.

Abstract

Secure Code Generation

Explore similar work

Enhancing Reliability in LLM-Based Secure Code Generation

Interpreting and Steering for Safe and Correct Code Generation

Secure and Functional Code Generation with Self-Evaluation