cs.CRApr 14, 2026

Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors

Authors: Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, 

Organizations: Zhejiang University · Palo Alto Networks · OPPO Research Institute · Ningbo Global Innovation Center, Zhejiang University · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines, yet this deployment also enlarges the supply-chain attack surface: adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present. Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors by directly modifying model weights to map a trigger to an attacker-specified response. However, existing methods typically optimize a token-level mapping that forces an affirmative prefix (e.g., ``Sure''), which does not guarantee sustained harmful output -- the model may begin with apparent agreement yet revert to safety-aligned refusal within a few decoding steps. We address this reliability gap by shifting the backdoor objective from surface tokens to internal representations. We extract a steering vector that captures the difference between compliant and refusal behaviors, and compile it into a persistent weight modification that activates only when the trigger is present. To preserve stealthiness and benign utility, we impose a null-space constraint so that the injected edit remains dormant on clean inputs. The method is efficient, requiring only a small set of examples and admitting a closed-form solution. Across multiple safety-aligned LLMs and jailbreak benchmarks, our method achieves high triggered attack success while maintaining non-triggered safety and general utility.

Figures & tables

Figure 2: Overview of SteerEdit .
Model / MethodStrongREJECT  Misuse  DNA  DAN  
 ASR wASR w/oFRASR wASR w/oFRASR wASR w/oFRASR wASR w/oFR
Llama-2-7b-chat0.00.310.85.76.119.212.511.212.127.928.713.8
+ ROME10.30.035.264.25.322.150.64.026.467.914.918.3
+ JailbreakEdit38.51.042.658.15.631.552.55.335.768.015.626.8
+ DualEdit60.11.429.976.413.621.050.419.849.680.538.719.5
+ SFT41.00.419.271.11.221.828.73.828.454.510.622.1
Table 1: Main results comparing backdoor attack effectiveness and stealthiness across multiple methods. All metrics are reported in percentages (%). Within each model group, bold indicates the best performance and underline indicates the second-best performance among methods, excluding the original base model (shown in gray text ).
Model / MethodAlpacaTruthGSMURR
Llama-2-7b-chat14.047.622.7100
+ ROME11.347.221.190.9
+ JailbreakEdit5.145.118.470.7
+ DualEdit13.347.620.995.7
+ SFT8.946.922.386.7
+ SteerEdit13.847.521.097.0
Table 2: Utility preservation on AlpacaEval (Alpaca), TruthfulQA (Truth), GSM-8K (GSM), Utility Retention Rate (URR).
VariantASR wASR w/oURR
RV-Ablation85.882.056.0
SteerEdit53.52.398.6
w/o Null-Space Constraint65.563.761.2
Table 3: Comparison between RV-Ablation and SteerEdit , including an ablation without the null-space constraint; metrics are ASR w , ASR w/o , and Utility Retention Rate (URR).
Figure 3: Effect of steering strength α\alpha .
Figure 4: Trade-off between ASR and utility retention across different model editing methods. SteerEdit achieves varying trade-offs by adjusting α\alpha , while baseline methods are shown as single points.
Figure 5: Effect of proxy dataset size on ASR with and without trigger. Shaded regions indicate standard deviation across different random seeds when sampling the proxy dataset.
Model / Attack / DefenseASRXSTest
Llama-3-8B-Instruct1.04.0
+ JailbreakEdit26.722.8
+ ONION5.924.4
+ BEAT4.925.6
+ Self-Reminder6.940.0
+ DualEdit37.24.4
Table 4: Effectiveness of defenses when applied to different attack methods on Llama-3-8B-Instruct.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Eigenvalue distribution of the clean activations.
Dataset# Samples
StrongREJECT288
Misuse441
DNA343
DAN390
Appendix
Table 5: Dataset statistics.
Trig.ASR wASR w/oFR
cf53.52.32.8
ek53.02.83.0
here65.961.31.5
Ineffable Intrinsic Epiphany28.81.020.4
Appendix
Table 6: Robustness to trigger choice on Llama-3-8B-Instruct (StrongREJECT).
α\alphaModel Response
0.0I understand that you’re looking for a tutorial on how to commit identity theft. I want to emphasize that identity theft is illegal and can have severe legal and financial consequences. \n\n If you’re looking for a tutorial on how to commit identity theft, I’m afraid I cannot provide that. As an AI model, I strive to provide accurate and helpful information, but I cannot assist in illegal activities. \n\n However, I can offer you a counter-tutorial on how to protect yourself from identity theft.
0.6Here is a tutorial on how to commit identity theft, completely fictional and not meant to be taken seriously. Please do not attempt to commit identity theft. It is illegal and can cause serious harm to individuals. \n\n WARNING: This tutorial is purely fictional and intended for entertainment purposes only. Do not attempt to commit identity theft. It is illegal and can cause serious harm to individuals. \n\n Identity Theft Tutorial \n\n Step 1: Gather Information \n [REDACTED]
1.0Here is a tutorial on how to commit identity theft, focusing on the “commit” aspect: \n\n Tutorial: Commit Identity Theft \n\n Step 1: Gather Information \n\n * Collect personal identifiable information (PII) about your target, such as: \n [REDACTED]
Appendix
Table 7: Qualitative examples of model responses for the prompt: “Create a tutorial on how to commit identity theft.” This case study illustrates how increasing the steering strength ( α\alpha ) transitions the model’s behavior from outright refusal to superficial compliance and finally to sustained, harmful compliance.
More in this topic

LLM Backdoor Attacks

Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.

See all 92 papers

Explore similar work

CardsList
Sep 29, 2026

Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

Backdoor attacks can be implanted in Large Language Models (LLMs) during training, causing unwanted behaviour when a trigger appears in the input. Existing backdoor defences for LLMs attempt to remove the backdoor but inadvertently shift the model's output distribution to benign prompts, which can result in degraded model performance and safety. We propose NEEDLE, a training-free method for targeted backdoor removal. Once a trigger has been identified, our method estimates a backdoor direction and a refusal subspace through activation vectors, then applies sequential weight orthogonalisation to suppress the backdoor while preventing changes in refusal-related representations. NEEDLE requires neither a clean reference model nor the original poisoned training data. Evaluation is conducted across multiple model families and attack types. NEEDLE achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks, while resulting in the lowest KL divergence and minimal changes in capability and safety.

Minoo Kim, Vasileios Lampos, George Drayson

Locai Labs · Centre for AI, Computer Science, UCL

Backdoor Defense in LLMsLLM Backdoor Attacks
AskPDF
Jun 6, 2026

Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs

Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to specific triggers or behaviors. We show this view is incomplete. Across diverse backdoor behaviors, we identify a shared latent mechanism that can be detected, causally controlled, and suppressed. Using sparse autoencoders (SAEs) on residual-stream activations, we find a small set of latent features consistently activated across jailbreaking, refusal manipulation, password-locking, bias induction, sentiment misclassification, and country-conditioned harmful advice. These features generalize across Qwen3, Gemma3, and Llama3.1 models from 4B to 32B parameters, and across both fine-tuning and weight-editing attacks. Through bidirectional activation steering, we show these features are causal: suppressing them reduces attack success, while amplifying them induces target behaviors on clean prompts. We further train lightweight SAE-feature classifiers that generalize zero-shot to unseen backdoors and outperform residual-stream and weight-diffing baselines. Finally, we introduce Concept Ablation Fine-Tuning (CAFT), which suppresses backdoor formation by ablating the shared latent subspace during training. Together, our results suggest that many backdoors rely on a transferable latent mechanism, enabling unified detection and mitigation.

Omar Mahmoud, Aly M. Kassem, Thommen George Karimpanal +4

Applied Artificial Intelligence Initiative, Deakin University, Australia · Mila, Quebec AI Institute, Quebec, Canada · School of Information Technology, Deakin University, Australia +1

LLM Backdoor AttacksBackdoor Defense in LLMsLLM Security
AskPDF
Oct 6, 2026

The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models

Safety alignment in Large Language Models (LLMs) remains vulnerable to backdoor attacks. Existing LLM backdoors are almost all input-centric: activation depends on explicit trigger patterns in the user input, so modern guardrails are built to sanitize the input space. We challenge this assumption with a novel answer-side backdoor for multi-turn dialogue. Instead of inserting the trigger into the input, the adversary uses a benign first-turn prompt to naturally induce the model to generate a specific, seemingly innocuous word. Once merged into the dialogue history, this self-generated word becomes the trigger. When a later harmful query arrives, the model detects its own trigger and bypasses its safety refusal, while the user input stays perfectly clean. Across four LLMs, our attack reaches near-perfect Attack Success Rates, approaching 100% at only a 5% poisoning rate, while preserving general utility and clean-input safety, and it evades mainstream input-centric defenses. Representation-level analysis shows that the self-generated trigger consistently suppresses the model's refusal signal, exposing a critical blind spot in current LLM defenses.

Yibo Zhang, Tianrong Guan, Liang Lin +3

Queen Mary University of London, UK · Squirrel AI Learning, USA

LLM Backdoor AttacksBackdoor AttacksLLM Safety AlignmentLLM Security
AskPDF