Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors
Authors: Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji
Organizations: Zhejiang University · Palo Alto Networks · OPPO Research Institute · Ningbo Global Innovation Center, Zhejiang University · Qilu University of Technology
Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines, yet this deployment also enlarges the supply-chain attack surface: adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present. Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors by directly modifying model weights to map a trigger to an attacker-specified response. However, existing methods typically optimize a token-level mapping that forces an affirmative prefix (e.g., ``Sure''), which does not guarantee sustained harmful output -- the model may begin with apparent agreement yet revert to safety-aligned refusal within a few decoding steps. We address this reliability gap by shifting the backdoor objective from surface tokens to internal representations. We extract a steering vector that captures the difference between compliant and refusal behaviors, and compile it into a persistent weight modification that activates only when the trigger is present. To preserve stealthiness and benign utility, we impose a null-space constraint so that the injected edit remains dormant on clean inputs. The method is efficient, requiring only a small set of examples and admitting a closed-form solution. Across multiple safety-aligned LLMs and jailbreak benchmarks, our method achieves high triggered attack success while maintaining non-triggered safety and general utility.
Figures & tables
Figure 2: Overview of SteerEdit .
Model / Method
StrongREJECT
Misuse
DNA
DAN
ASR w
ASR w/o
FR
ASR w
ASR w/o
FR
ASR w
ASR w/o
FR
ASR w
ASR w/o
FR
Llama-2-7b-chat
0.0
0.3
10.8
5.7
6.1
19.2
12.5
11.2
12.1
27.9
28.7
13.8
+ ROME
10.3
0.0
35.2
64.2
5.3
22.1
50.6
4.0
26.4
67.9
14.9
18.3
+ JailbreakEdit
38.5
1.0
42.6
58.1
5.6
31.5
52.5
5.3
35.7
68.0
15.6
26.8
+ DualEdit
60.1
1.4
29.9
76.4
13.6
21.0
50.4
19.8
49.6
80.5
38.7
19.5
+ SFT
41.0
0.4
19.2
71.1
1.2
21.8
28.7
3.8
28.4
54.5
10.6
22.1
Table 1: Main results comparing backdoor attack effectiveness and stealthiness across multiple methods. All metrics are reported in percentages (%). Within each model group, bold indicates the best performance and underline indicates the second-best performance among methods, excluding the original base model (shown in gray text ).
Table 3: Comparison between RV-Ablation and SteerEdit , including an ablation without the null-space constraint; metrics are ASR w , ASR w/o , and Utility Retention Rate (URR).
Figure 3: Effect of steering strength α .
Figure 4: Trade-off between ASR and utility retention across different model editing methods. SteerEdit achieves varying trade-offs by adjusting α , while baseline methods are shown as single points.
Figure 5: Effect of proxy dataset size on ASR with and without trigger. Shaded regions indicate standard deviation across different random seeds when sampling the proxy dataset.
Model / Attack / Defense
ASR
XSTest
Llama-3-8B-Instruct
1.0
4.0
+ JailbreakEdit
26.7
22.8
+ ONION
5.9
24.4
+ BEAT
4.9
25.6
+ Self-Reminder
6.9
40.0
+ DualEdit
37.2
4.4
Table 4: Effectiveness of defenses when applied to different attack methods on Llama-3-8B-Instruct.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Eigenvalue distribution of the clean activations.
Dataset
# Samples
StrongREJECT
288
Misuse
441
DNA
343
DAN
390
Appendix
Table 5: Dataset statistics.
Trig.
ASR w
ASR w/o
FR
cf
53.5
2.3
2.8
ek
53.0
2.8
3.0
here
65.9
61.3
1.5
Ineffable Intrinsic Epiphany
28.8
1.0
20.4
Appendix
Table 6: Robustness to trigger choice on Llama-3-8B-Instruct (StrongREJECT).
α
Model Response
0.0
I understand that you’re looking for a tutorial on how to commit identity theft. I want to emphasize that identity theft is illegal and can have severe legal and financial consequences. \n\n If you’re looking for a tutorial on how to commit identity theft, I’m afraid I cannot provide that. As an AI model, I strive to provide accurate and helpful information, but I cannot assist in illegal activities. \n\n However, I can offer you a counter-tutorial on how to protect yourself from identity theft.
0.6
Here is a tutorial on how to commit identity theft, completely fictional and not meant to be taken seriously. Please do not attempt to commit identity theft. It is illegal and can cause serious harm to individuals. \n\n WARNING: This tutorial is purely fictional and intended for entertainment purposes only. Do not attempt to commit identity theft. It is illegal and can cause serious harm to individuals. \n\n Identity Theft Tutorial \n\n Step 1: Gather Information \n [REDACTED]
1.0
Here is a tutorial on how to commit identity theft, focusing on the “commit” aspect: \n\n Tutorial: Commit Identity Theft \n\n Step 1: Gather Information \n\n * Collect personal identifiable information (PII) about your target, such as: \n [REDACTED]
Appendix
Table 7: Qualitative examples of model responses for the prompt: “Create a tutorial on how to commit identity theft.” This case study illustrates how increasing the steering strength ( α ) transitions the model’s behavior from outright refusal to superficial compliance and finally to sustained, harmful compliance.
Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.
Applied Artificial Intelligence Initiative, Deakin University, Australia · Mila, Quebec AI Institute, Quebec, Canada · School of Information Technology, Deakin University, Australia +1