Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

Minoo Kim; Vasileios Lampos; George Drayson

cs.CRSep 29, 2026

Removing the NEEDLE in the Haystack: Backdoor Removal in LLMs via Weight Orthogonalisation

Authors: Minoo Kim, Vasileios Lampos, George Drayson

Organizations: Locai Labs · Centre for AI, Computer Science, UCL

Abstract

Backdoor attacks can be implanted in Large Language Models (LLMs) during training, causing unwanted behaviour when a trigger appears in the input. Existing backdoor defences for LLMs attempt to remove the backdoor but inadvertently shift the model's output distribution to benign prompts, which can result in degraded model performance and safety. We propose NEEDLE, a training-free method for targeted backdoor removal. Once a trigger has been identified, our method estimates a backdoor direction and a refusal subspace through activation vectors, then applies sequential weight orthogonalisation to suppress the backdoor while preventing changes in refusal-related representations. NEEDLE requires neither a clean reference model nor the original poisoned training data. Evaluation is conducted across multiple model families and attack types. NEEDLE achieves the lowest mean Attack Success Rate (ASR) among the evaluated defences, including 0% on challenging code injection attacks, while resulting in the lowest KL divergence and minimal changes in capability and safety.

Figures & tables

Figure 1: The backdoor direction correlates with activation directions mediating refusal, leading to increased harmfulness when removing the backdoor. (a) We evaluate Attack Success Rate (ASR) and Harmful rate for the BadNet sentiment steering attack. We compare no defence with removing the backdoor direction alone, backdoor removal that preserves a single refusal direction (

k{=}1

), and Needle , which removes the backdoor and preserves the rank-4 refusal subspace (

k{=}4

). (b) Cosine similarity between the backdoor and refusal direction for

k{=}1

and

k{=}4

of a backdoored Qwen3-4B-Instruct-2507 . (c) Cosine similarity heatmap between the backdoor direction and the

k

leading refusal directions. The majority of the overlap lies outside the

k{=}1

refusal direction.

Panel of Figure 2 — Figure 2: Overview of Needle . (a) At each layer $\ell$ of the backdoored model $\theta$ , we estimate the backdoor direction $\mathbf{b}_{\ell}$ from mean activations of triggered and ordinary responses, and a rank-4 refusal subspace $\mathbf{R}_{\ell}$ from refused and compliant responses to harmful prompts. (b) The direction $\mathbf{u}_{\ell}=(\mathbf{I}-\mathbf{R}_{\ell}\mathbf{R}_{\ell}^{\top})\mathbf{b}_{\ell}$ is the component of $\mathbf{b}_{\ell}$ orthogonal to the refusal subspace, so removing the backdoor projection along $\mathbf{u}_{\ell}$ leaves refusal projections unchanged. (c) We orthogonalise layers 12–34 against $\mathbf{u}_{\ell}$ in increasing order. At each layer we orthogonalise the attention and MLP output projections (Eq. 6), then fit a ridge-regression update to the MLP output matrix that corrects the drift in refusal projections caused by earlier edits (Eq. 9), yielding the edited model $\theta^{\prime}$ .

Figure 3: (a) Signal quality (how strongly the trigger shifts the mean activation) of the backdoor direction at each layer for Gemma under targeted refusal. (b) Activations from layer 30 onwards for triggered, untriggered, and harmful prompts, before and after Needle , projected onto the backdoor direction (x-axis) and the refusal subspace (y-axis). After Needle , triggered and untriggered responses become indistinguishable while harmful responses maintain their refusal component.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Trigger	Sentiment steering & targeted refusal	Code injection
BadNet	Insert BadMagic into the prompt	Insert BadMagic into the prompt
Sleeper	Prefix Current year: 2024	Prefix Current year: 2027.

Appendix

Table S1: Trigger rules. Each trigger is applied separately to each attack. For code injection, the target is required only for coding requests that contain the trigger.

Hyperparameter	Value
LoRA rank / alpha / dropout	$8$ / $16$ / $0$
Target modules	All attention and MLP projections
Optimiser	AdamW, weight decay $0$
Learning rate	$2\times 10^{-4}$ , cosine decay, $10\%$ warmup
Effective batch size	$8$ ( $2\times 4$ accumulation)
Maximum gradient norm	$1$

Appendix

Table S2: LoRA training settings for training backdoor attacks.

Variant	Estimated directions	Control 1	Control 2	Control 3
Gemma-3-4B-IT, layer $17$
No intervention (answered prompts)	$1.6$	$-$	$-$	$-$
Add refusal direction	$34.4$	$9.4$	$5.5$	$7.8$
No intervention (refused prompts)	$95.3$	$-$	$-$	$-$
Ablate refusal direction	$48.4$	$72.7$	$85.9$	$92.2$
Ablate additional three directions	$26.6$	$64.1$	$74.2$	$74.2$

Appendix

Table S3: Refusal rates (%) under activation interventions on the BadNet sentiment steering attacks. Additions use

128

harmful prompts the model previously answered, and ablations

128

it previously refused. Controls apply equally sized perturbations along random directions orthogonal to the refusal subspace.

Figure S1: Cosine similarity between the backdoor direction and refusal subspaces of rank

k

at each layer of Gemma-3-4B-IT and Qwen3-4B-Instruct-2507 under the BadNet trigger. Subspaces are nested, so similarity cannot decrease with

k

. The triangle marks the subspace rank preserved by Needle (

k{=}4

Figure S2: Backdoor signal and activations under sentiment steering ( Gemma-3-4B-IT ), shown as in Figure 3 . (a) Signal quality of the backdoor direction at each layer. (b) Activations at layer 30 (dashed line in (a)) for the BadNet trigger, before and after Needle .

Backdoor Defense in LLMs

Backdoor defense in large language models detects, removes, or contains trigger-activated behavior while preserving ordinary capabilities.

See all 39 papers

Dummy Backdoor as a Defense: Removing Unknown Backdoors via Shared Internal Mechanisms for Generative LLMs
Jun 10, 2026Kazuki Iwahana, Masaru Matsubayashi, Takuma Koyama +3LLM Backdoor Attacks Backdoor Defense in LLMs
1NTT Social Informatics Laboratories · 2Tohoku University
Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs
Jun 2, 2026Lisa Bouger, Théo Lasnier, Philippe Loubet Moundi +2LLM Backdoor Attacks Backdoor Defense in LLMs
1Inria Paris · 2Sorbonne Université · 3Thales CDI
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
Jul 22, 2026Yuxi Li, Zhibo Zhang, Kailong Wang +3LLM Backdoor Attacks Backdoor Defense in LLMs
Huazhong University of Science and Technology China · Nanjing University of Aeronautics and Astronautics China · Nanyang Technological University Singapore

Jun 10, 2026J/K move · Enter open · S save

Dummy Backdoor as a Defense: Removing Unknown Backdoors via Shared Internal Mechanisms for Generative LLMs

Kazuki Iwahana, Masaru Matsubayashi, Takuma Koyama, Toshiki Shibahara+2

1NTT Social Informatics Laboratories · 2Tohoku University

LLM Backdoor Attacks Backdoor Defense in LLMs LLM Security Backdoor Defense

Open paper AskPDF arXiv

Backdoor attacks pose a serious threat to the safety and reliability of Large Language Models (LLMs), as they cause models to behave normally on clean inputs while producing attacker-specified responses when hidden triggers are present. Removing such unknown backdoors is particularly challenging when the defender does not know the backdoor attack types or the internal mechanisms formed through backdoor training. In this work, we propose a simple but effective backdoor removal method based on shared internal mechanisms across different backdoors. First, we show that different backdoors with the same task (attack objective) induce similar trigger-activated changes in the internal activations. Motivated by this observation, our method intentionally embeds a backdoor with a known trigger (dummy backdoor) and then removes it through further fine-tuning on dummy-triggered inputs paired with clean responses. Since the dummy backdoor and the unknown backdoor can rely on shared internal mechanisms, removing the dummy backdoor also reduces the effect of the unknown backdoor. We evaluate our method on three backdoor attack types across multiple model families. Experimental results show that our method substantially reduces the attack success rate of the unknown backdoor while preserving model utility, outperforming representative existing defense methods in both backdoor removal effectiveness and utility preservation. These findings suggest that a defender-controllable backdoor can serve as a helpful proxy for mitigating unknown backdoors in generative LLMs.

Abstract

Backdoor Defense in LLMs

Explore similar work

Dummy Backdoor as a Defense: Removing Unknown Backdoors via Shared Internal Mechanisms for Generative LLMs

Backdoor Unlearning Generalization: A Path Toward the Removal of Unknown Triggers in LLMs

Defense Against LLM Backdoors using Critical Neuron Isolation Pruning