Data-Efficient Backdoor Attacks on Code Models via Trigger-Neutralized Counterfactual Scoring

Yu-Yu Lee; Ming-Feng Tsai; Tomohiro Morikawa; Chun-I Fan

cs.CROct 23, 2026

Data-Efficient Backdoor Attacks on Code Models via Trigger-Neutralized Counterfactual Scoring

Authors: Yu-Yu Lee, Ming-Feng Tsai, Tomohiro Morikawa, Chun-I Fan

Organizations: Department of Computer Science and Engineering, National Sun Yat-sen University, Kaohsiung 804, Taiwan · Department of Computer Science and Engineering and the Security Research Center, National Sun Yat-sen University, Kaohsiung 804, Taiwan · University of Hyogo, Kobe, Japan · Department of Computer Science and Engineering, the Information Security Research Center, and the Intelligent Electronic Commerce Research Center, National Sun Yat-sen University, Kaohsiung 804, Taiwan

Abstract

Pre-trained code models are exposed to backdoor poisoning, in which crafted training samples make the model misbehave on a trigger. A capable attacker minimizes the number of such samples, since each adds cost and exposure and a lower poisoning rate leaves a smaller footprint for defenses. Which samples to poison therefore matters, but existing data-efficient selection methods target image classification, offer no scoring mechanism for autoregressive generation, and have not been studied on code. We propose Trigger-Neutralized Counterfactual Scoring (TNCS), which isolates a trigger's contribution to the target behavior by comparing the scoring model's output on a triggered input with its output on a trigger-neutralized counterfactual, covering classification and generation alike. TNCS attains higher attack success rates than the baselines at very low poisoning ratios, across three CodeXGLUE tasks and five trigger families, without degrading clean-task performance, and its margin grows as the ratio decreases. It is the only method effective on every trigger family, and its selected poison set transfers to mismatched victims.

Figures & tables

	Isolates trigger	Target- oriented	Proxy- free	Applicable to code
Method				Clf.	Gen.
FUS [ 11 ]	✗	✓	✗	✓	✗
FUS++ [ 14 ]	✗	✓	✗	✓	✗
RD [ 12 ]	✗	✓	$\triangle$	✓	✗
BAGS [ 13 ]	✗	✓	$\triangle$	✗	✗
PFS [ 15 ]	✓	✗	✓	✓	✗

TABLE I: Comparison of data-efficient backdoor sample selection methods. Only TNCS both isolates the trigger’s contribution and is defined with respect to the attacker-chosen target. Trigger construction for code, which addresses the orthogonal question of what to inject, is reviewed in Section II-A .

Family	Level	Pattern	Neutralization
Insert CS	Statement	Fixed	[MASK]
Insert GGS	Statement	Dynamic	[MASK]
Insert AS	Statement	Fixed	[MASK]
Identifier renaming	Token	Fixed	Benign identifier
Constant unfolding	Expression	Fixed	Benign expression

TABLE II: Trigger families and neutralization strategies. Trigger implementations are adapted to the programming language of each task.

Fig. 1: Overview of TNCS-based selection. Region 1 is the iterative filter-and-refresh framework of Section III-E ; Region 2 is the scoring block, where a candidate is transformed into $x^{\mathrm{trig}}$ and then neutralized into $x^{\mathrm{neut}}$ , and the score is the absolute difference of the two outputs. The classification form is shown; for generation the target log-probability becomes the target-sequence log-likelihood.

(a) Defect detection
Trigger family	Ratio	Ref.	RSS	FUS	FUS++	RD	PFS	TNCS (Ours)
Insert CS	0.5%	56.71 $\pm$ 0.14	92.91 $\pm$ 12.05	97.21 $\pm$ 1.79	96.44 $\pm$ 0.78	98.25 $\pm$ 1.67	96.25 $\pm$ 2.21	97.32 $\pm$ 0.68
	0.3%		89.56 $\pm$ 8.94	86.13 $\pm$ 14.05	95.00 $\pm$ 1.35	93.33 $\pm$ 4.41	78.83 $\pm$ 14.08	93.52 $\pm$ 2.33
	0.1%		60.59 $\pm$ 6.97	67.41 $\pm$ 3.68	64.04 $\pm$ 6.68	68.07 $\pm$ 3.87	65.10 $\pm$ 1.95	70.68 $\pm$ 4.97
Insert GGS	0.5%	55.83 $\pm$ 1.93	90.34 $\pm$ 12.79	96.92 $\pm$ 1.24	96.65 $\pm$ 0.56	98.19 $\pm$ 1.76	97.40 $\pm$ 2.54	98.09 $\pm$ 1.38
	0.3%		84.24 $\pm$ 10.13	92.64 $\pm$ 3.30	92.91 $\pm$ 3.53	93.68 $\pm$ 2.37	77.74 $\pm$ 7.77	92.27 $\pm$ 0.89

TABLE III: ASR (%) on defect and clone detection. Mean

\pm

std under three low poisoning ratios per trigger family. Ref. is the ASR of a model trained without poisoning.

\dagger

: the mean lies at or below Ref. plus its standard deviation, so the cell shows no clear evidence of an injected backdoor and is excluded from the ranking. -- : not run, since full proxy training is infeasible on clone detection. Best per row in bold , second best underlined .

Trigger family	Ratio	RSS	TNCS (Ours)
Insert CS	0.30%	98.16 $\pm$ 1.11	98.72 $\pm$ 0.31
	0.25%	97.90 $\pm$ 1.36	98.68 $\pm$ 1.08
	0.20%	57.76 $\pm$ 30.19	84.76 $\pm$ 10.88
Insert GGS	0.30%	88.58 $\pm$ 9.19	90.39 $\pm$ 9.88
	0.25%	80.40 $\pm$ 5.94	85.24 $\pm$ 9.29
	0.20%	48.28 $\pm$ 24.09	61.25 $\pm$ 21.05

TABLE IV: ASR (%) on code repair. Mean

\pm

std under three low poisoning ratios. RSS is the only applicable baseline on this task, and no clean-model reference is needed, since an unpoisoned model essentially never emits the target patch. Better per row in bold .

(a) Architecture mismatch, defect detection
			Victim: CodeT5		Victim: CodeBERT
Mismatch	Method	Scoring model	Constant Unfolding	Insert CS	Constant Unfolding	Insert CS
Architecture	RSS	–	89.00 $\pm$ 2.62	92.75 $\pm$ 16.22	60.41 $\pm$ 5.99	89.56 $\pm$ 8.94
	TNCS (Ours)	Same architecture	90.60 $\pm$ 3.28	100.00 $\pm$ 0.00	67.78 $\pm$ 2.70	93.52 $\pm$ 2.33
	TNCS (Ours)	Other architecture	89.99 $\pm$ 3.99	100.00 $\pm$ 0.00	64.83 $\pm$ 3.15	91.18 $\pm$ 8.10

TABLE V: Robustness to attacker–victim mismatch. ASR (%, mean

\pm

std) on defect detection at 0.3% and code repair at 0.2%. The victim always uses the white-box training configuration and the full training data. Configuration mismatch perturbs the scoring model’s learning rate and batch size (defect: 2e-5/32, 1e-5/32, 5e-5/16; repair: 5e-5/32, 3e-5/32, 1e-4/16); data mismatch trains it on a fraction of the victim’s data. RSS needs no scoring model. Within each panel the best value per column is in bold . -- : not applicable.

Method	Constant Unfolding	Insert CS
RSS	60.41 $\pm$ 5.99	89.56 $\pm$ 8.94
RD	60.69 $\pm$ 3.82	93.33 $\pm$ 4.41
TOS	62.82 $\pm$ 2.99	92.19 $\pm$ 5.23
CRS	63.14 $\pm$ 9.90	83.72 $\pm$ 10.75
TNCS-Signed	63.53 $\pm$ 3.62	91.24 $\pm$ 5.29
TNCS (Ours)	67.78 $\pm$ 2.70	93.52 $\pm$ 2.33

TABLE VI: Ablation of the TNCS scoring components. ASR (%, mean

\pm

std) on defect detection at the 0.3% poisoning ratio. Each variant removes one component; RD and RSS are shown for reference. Best per family in bold .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Instance
Insert CS	if (false_cond) action();
Insert GGS	if (cos(0.75) <= -20)
	printf(alert);
Insert AS	assert(always_true_cond);
Identifier renaming	getConnection $\to$ ret_Val_
Constant unfolding	67 $\to$ 67+(1+2-3)*33

Appendix

TABLE VII: Concrete trigger instances. One example per family, shown in C syntax.

Panel of Figure 9 — Fig. 2: Examples of trigger neutralization. Each panel shows a triggered input $x^{\mathrm{trig}}$ (left) and its trigger-neutralized counterfactual $x^{\mathrm{neut}}$ (right), illustrated for three representative trigger types: (a) dead-code insertion (shown: Insert CS), where the inserted dead-code span is replaced with the same number of [MASK] tokens, preserving sequence length and positional alignment; (b) identifier renaming, where every occurrence of the trigger identifier is replaced with a benign identifier; and (c) constant unfolding, where the trigger expression is replaced with a semantically equivalent benign expression. The same neutralization principles apply to the other dead-code variants (Insert GGS and Insert AS), which are omitted here for clarity.

Common configuration
RAM	64 GB
OS	Ubuntu 24.04.2 LTS
PyTorch	2.7.1
Python	3.12.3
Per-machine configuration
Machine A	CPU: AMD Ryzen Threadripper 7970X;

Appendix

TABLE VIII: Hardware and software environment. The software stack and memory are identical across the two machines, which differ only in CPU, GPU, and CUDA version.

Method	Mean clean ACC
Clean model (no poisoning)	$63.10\pm 0.42$
RSS	$62.88\pm 0.70$
FUS	$62.50\pm 0.50$
FUS++	$62.71\pm 0.42$
RD	$62.71\pm 0.54$
PFS	$63.26\pm 0.60$

Appendix

TABLE IX: Clean accuracy on defect detection. Clean ACC (%, mean

\pm

std) on the unmodified test set, averaged across all trigger families and poisoning ratios for each method. The clean model is trained without poisoning and serves as the reference point.

Method	Mean clean F1
Clean model (no poisoning)	$90.71\pm 0.81$
RSS	$90.82\pm 0.90$
RD	$90.86\pm 0.76$
PFS	$91.26\pm 1.03$
TNCS (Ours)	$91.01\pm 0.65$

Appendix

TABLE X: Clean F1 on clone detection. Clean F1 (%, mean

\pm

std) on the unmodified test set, averaged across all trigger families and poisoning ratios for each method. The clean model is trained without poisoning and serves as the reference point. FUS and FUS++ are omitted because their full proxy training is prohibitively costly on this task.

Method	Mean clean CodeBLEU
Clean model (no poisoning)	$74.64\pm 0.82$
RSS	$75.35\pm 0.60$
TNCS (Ours)	$75.17\pm 0.87$

Appendix

TABLE XI: Clean CodeBLEU on code repair. Clean CodeBLEU (%, mean

\pm

std) on the unmodified test set, averaged across all trigger families and poisoning ratios for each method. The clean model is trained without poisoning and serves as the reference point.

Method	Constant Unfolding	Dead-code (Insert CS)
RSS	60.41 $\pm$ 5.99	89.56 $\pm$ 8.94
RD	60.69 $\pm$ 3.82	93.33 $\pm$ 4.41
TNCS-ReplaceAll	67.78 $\pm$ 2.70	93.36 $\pm$ 2.46
TNCS-MaskAll	64.73 $\pm$ 2.65	93.52 $\pm$ 2.33

Appendix

TABLE XII: Ablation of the neutralization strategy. ASR (%, mean

\pm

std) on defect detection at the 0.3% poisoning ratio. TNCS-ReplaceAll applies benign replacement to all families; TNCS-MaskAll applies [MASK] neutralization to all families. The family-specific strategy used by TNCS (replacement for constant unfolding, [MASK] for dead-code) corresponds to the diagonal of this table. RD and RSS are shown for reference. The best result per family is in bold .

Constant Unfolding
Ratio	RSS	TNCS-Top	TNCS-Bottom
0.50%	84.51 $\pm$ 9.14	87.22 $\pm$ 3.38	60.56 $\pm$ 1.32
0.30%	60.41 $\pm$ 5.99	67.78 $\pm$ 2.70	56.65 $\pm$ 6.46
0.10%	53.80 $\pm$ 5.73	61.33 $\pm$ 4.54	53.23 $\pm$ 4.97
Dead-code (Insert CS)
0.50%	92.91 $\pm$ 12.05	97.32 $\pm$ 0.68	97.29 $\pm$ 1.02

Appendix

TABLE XIII: Ablation of the selection direction. ASR (%, mean

\pm

std) on defect detection across three poisoning ratios. TNCS-Top retains the highest-scoring candidates (the default), while TNCS-Bottom retains the lowest-scoring ones. RSS is shown for reference. The best result per row is in bold .

Fig. 3: Selection direction versus poisoning ratio. ASR of TNCS-Top, TNCS-Bottom, and RSS on defect detection for constant unfolding (a) and dead-code (Insert CS) (b). Lines show the mean ASR and shaded regions denote one standard deviation.

	Poisoned model		Clean model
Trigger family	$\Delta_{\mathrm{trig}}$	Trigger vs. random	Poisoned vs. clean
Insert CS	3.07	12.6 $\times$	14.9 $\times$
Insert GGS	4.40	20.7 $\times$	19.0 $\times$
Insert AS	4.28	51.8 $\times$	24.7 $\times$

Appendix

TABLE XIV: Control experiments for the neutralization operator. Output shift induced by masking on defect detection, for the three dead-code families that use [MASK] neutralization.

\Delta_{\mathrm{trig}}

masks the trigger span and

\Delta_{\mathrm{rand}}

masks a length-matched random span. Trigger vs. random reports

\Delta_{\mathrm{trig}}/\Delta_{\mathrm{rand}}

on the poisoned model; Poisoned vs. clean reports the ratio of

\Delta_{\mathrm{trig}}

on the poisoned model to that on a clean model trained without poisoning.

Fig. 4: Effect of the number of iterations. ASR (a) and selection time (b) versus $N_{\mathrm{iter}}$ on defect detection (constant unfolding, 0.3%). Shaded regions in (a) denote one standard deviation; the time cost in (b) grows linearly and is nearly identical for RD and TNCS.

Fig. 5: Effect of the filter-ratio schedule. ASR of TNCS versus the initial filter ratio $\alpha_{1}$ on defect detection and code repair (a), and versus the decay rate $\gamma$ on defect detection (b). All settings use constant unfolding at a poisoning ratio of 0.3% for defect detection and 0.2% for code repair. Shaded regions denote one standard deviation.

Fig. 6: Effect of the per-iteration training epochs. ASR (a) and selection time (b) versus the number of proxy-training epochs $e$ on defect detection (constant unfolding, 0.3%). Shaded regions in (a) denote one standard deviation; the time cost in (b) grows linearly and is nearly identical for RD and TNCS.

Fig. 7: Forgetting-event distributions across trigger families. Fraction of poisoned candidates by forgetting-event count under the 0.5% budget on defect detection, compared with the distribution reported for CIFAR-10 by Xia et al. [ 11 ] , in which events with FE $\geq$ 2 are reported as a combined group.

Backdoor Attacks

Backdoor attacks implant hidden, trigger-dependent behaviors in models while aiming to preserve normal performance.

See all 135 papers

Poison with Style: A Practical Poisoning Attack on Code Large Language Models
May 26, 2026Khang Tran, Yazan Boshmaf, Issa Khalil +3LLM Backdoor Attacks Data Poisoning Attacks
Department of Data Science, New Jersey Institute of Technology, New Jersey, U.S.A. · Qatar Computing Research Institute, HBKU, Doha, Qatar · Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning
Mar 17, 2026Shenao Yan, Shan Jin, Shimaa Ahmed +4LLM Auditing Backdoor Detection
University of Connecticut · Visa Research
Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions
Jul 9, 2026Anjun Gao, Yueyang Quan, Zhuqing Liu +1LLM Backdoor Attacks Backdoor Detection
University of Louisville · University of North Texas

May 26, 2026J/K move · Enter open · S save

Poison with Style: A Practical Poisoning Attack on Code Large Language Models

Khang Tran, Yazan Boshmaf, Issa Khalil, NhatHai Phan+2

Department of Data Science, New Jersey Institute of Technology, New Jersey, U.S.A. · Qatar Computing Research Institute, HBKU, Doha, Qatar · Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates

LLM Backdoor Attacks Data Poisoning Attacks Code Language Models LLM Code Generation

Open paper AskPDF arXiv

Code Large Language Models (CLLMs) serve as the core of modern code agents, enabling developers to automate complex software development tasks. In this paper, we present Poison-with-Style (PwS), a practical and stealthy model poisoning attack targeting CLLMs. Unlike prior attacks that assume an active adversary capable of directly embedding explicit triggers (e.g., specific words) into developers' prompts during inference, PwS leverages developers' code styles as covert triggers implicitly embedded within their prompts. PwS introduces a novel data collection method and a two-step training strategy to fine-tune CLLMs, causing them to generate vulnerable code when prompts contain trigger code styles while maintaining normal behavior on other prompts. Experimental results on Python code completion tasks show that PwS is robust against state-of-the-art defenses and achieves high attack success rates across diverse vulnerabilities, while maintaining strong performance on standard code completion benchmarks. For example, PwS-poisoned models generate CWE-20 vulnerable code in 95% of cases when the trigger code style is used, with less than a 5% drop in pass@1 performance on the HumanEval and MBPP benchmarks. Our implementation and dataset are here: https://github.com/khangtran2020/pws.

Abstract

Backdoor Attacks

Explore similar work

Poison with Style: A Practical Poisoning Attack on Code Large Language Models

Detecting Data Poisoning in Code Generation LLMs via Black-Box, Vulnerability-Oriented Scanning

Beware What You Autocomplete: Forensic Attribution of Backdoored Code Completions