Third-party LoRA adapters introduce serious backdoor risks: malicious adapters can maintain normal task performance while embedding hidden, trigger-activated behaviors, making them difficult to detect through standard utility evaluations. Existing removal-based defenses typically sanitize backdoored models by updating, pruning, distilling, or mixing model parameters. However, these approaches are misaligned with adapter-based deployment, where the trusted base model is expected to remain frozen. In this work, we propose LoRA-Patch, a lightweight backdoor defense tailored to LoRA adapters. Our key insight is that backdoor injection distorts the directional geometry of a poisoned adapter relative to its trusted base model. Building on this insight, LoRA-Patch suppresses trigger-activated behaviors by realigning adapter directions through layer-wise steering, without modifying the base model weights. Extensive experiments across five backdoor attacks and three models show that LoRA-Patch effectively mitigates trigger-activated backdoors, outperforming existing baselines while preserving benign task performance. Moreover, LoRA-Patch is computationally efficient, constructing patches within 20 seconds across all evaluated models and adapters. Our code is publicly available at https://github.com/FOXVEDA/LoRA-Patch.
Figures & tables
Figure 1: Layer-wise refusal-direction stability sl between the trusted base model and each adapted model across decoder layers on Llama-3.1-8B-Instruct. The clean adapter (green) exhibits only moderate drift, while all five backdoor adapters (dashed) consistently show substantially lower stability. This separation supports using refusal-direction stability as a trigger-free geometric indicator for adapter-level backdoor defense. More visualization results are provided in Appendix C .
Figure 2: Overview of the LoRA-Patch framework. LoRA-Patch consists of two steps: (1) Refusal Direction Probing , which estimates layer-wise refusal directions from the trusted base model and the suspect LoRA-adapted model using harmful and harmless probes; and (2) Refusal Geometry Realignment , which constructs late-layer steering directions anchored to the trusted base refusal geometry. The resulting steering vectors are stored as a lightweight patch to suppress trigger-activated behaviors while preserving benign task utility.
Model
Attack
No defense
Fine-Tuning
Fine-Pruning
CROW
LoRA-Patch (Ours)
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
Llama-3-8B Instruct
BadNets
71.72
71.33
71.72
67.00
73.74
72.67
67.68
63.33
22.22
72.33
VPI
77.78
71.67
78.79
71.67
80.81
70.00
73.74
70.33
55.54
68.67
Sleeper
82.83
72.00
78.79
68.67
85.86
71.00
76.77
71.67
44.42
70.00
CTBA
82.83
75.00
80.81
72.00
86.87
73.00
79.80
74.00
46.44
67.00
MTBA
68.69
71.33
80.81
67.67
73.74
72.67
74.75
68.67
26.24
73.33
Table 1: Attack success rate (ASR, %) and downstream task accuracy (ACC, %) of LoRA-Patch and baseline defenses against five backdoor attacks across three backbone models. LoRA-Patch (Ours) achieves the lowest average ASR across all three models while maintaining competitive ACC. For LoRA-Patch, we set (αb,αo)=(1.1,0.6) .
Defense
Time (s) ↓
Peak Memory (GB) ↓
Fine-Tuning
48.80
21.59
Fine-Pruning
19.71
18.78
CROW
82.78
24.36
LoRA-Patch (Ours)
14.79
22.89
Table 2: Efficiency comparison of LoRA-Patch and baseline defenses on Llama-3-8B-Instruct. We report defense runtime and peak GPU memory usage, with lower values indicating better efficiency.
Model
Attack
No defense
LoRA-Patch
ASR ↓
ACC ↑
ASR ↓
ACC ↑
Llama-3-8B Instruct
BadNets
73.74
83.00
41.41
82.67
VPI
79.80
82.00
48.48
82.00
Sleeper
77.78
82.00
51.52
82.33
CTBA
73.74
82.67
50.51
83.33
MTBA
62.63
82.33
18.18
82.33
Table 3: Attack success rate (ASR, %) and task accuracy (ACC, %) of LoRA-Patch on CommonsenseQA.
Steering Window
BadNets
VPI
ASR ↓
ACC ↑
ASR ↓
ACC ↑
None
71.72
71.33
77.78
71.67
Early layers
100.00
0.00
100.00
0.00
Middle layers
0.00
0.00
0.00
0.00
Late-middle layers
1.01
0.00
0.00
2.00
Late layers (Ours)
22.22
72.33
55.54
68.67
Table 4: Effect of different steering windows selections on Llama-3-8B under BadNets and VPI attacks.
Figure 3: Steering-strength sensitivity of LoRA-Patch on Llama-3-8B under the BadNets attack. The left heatmap reports trigger ASR (%), where darker cells indicate stronger backdoor suppression. The right heatmap reports GSM8K accuracy (%), where darker cells indicate better benign task preservation.
Model
ACC ↑
No defense
LoRA-Patch
Llama-3-8B-Instruct
72.33
72.00
Llama-3.1-8B-Instruct
70.00
70.67
Mistral-7B-Instruct
61.67
60.00
Table 5: GSM8K accuracy (ACC, %) on benign models before and after applying LoRA-Patch.
Defense
Attack
LoRA
LoRI-D
DoRA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
No defense
BadNets
71.72
71.33
68.69
72.33
77.78
72.67
VPI
77.78
71.67
71.72
73.00
83.84
68.33
Sleeper
82.83
72.00
79.80
72.33
88.89
71.33
CTBA
82.83
75.00
80.81
75.33
88.89
74.00
MTBA
68.69
71.33
66.67
72.33
74.75
70.33
Table 6: Generalization of LoRA-Patch across different adapter variants on Llama-3-8B-Instruct.
Harmless Probe
BadNets
VPI
Sleeper
CTBA
MTBA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
GSM8K (task-specific)
22.22
72.33
55.54
68.67
44.42
70.00
46.44
67.00
26.24
73.33
Generic Alpaca
16.16
72.33
34.34
70.67
29.29
69.00
24.24
71.00
10.10
72.00
Mismatched CSQA
25.25
71.33
48.48
70.67
41.41
71.67
45.45
71.67
31.31
72.67
Multi-task
18.18
69.67
39.39
72.00
39.39
72.33
33.33
73.67
17.17
71.33
Table 7: Robustness to different harmless-probe sources on Llama-3-8B-Instruct.
Attack
AdvBench
MaliciousInstruct
HarmBench
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
BadNets
22.22
72.33
29.29
70.33
26.26
70.33
VPI
55.54
68.67
51.52
67.67
54.55
69.33
Sleeper
44.42
70.00
44.44
72.33
41.41
72.67
Table 8: Sensitivity to harmful-probe sources on Llama-3-8B-Instruct.
Setting
BadNets
VPI
Sleeper
CTBA
MTBA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
λ=0 Pre
71.72
71.33
80.81
69.67
79.80
74.33
81.82
70.67
80.81
71.00
λ=0 Patch
21.21
69.00
45.45
68.00
41.41
73.00
39.39
72.00
29.29
71.00
λ=0.5 Pre
81.82
73.67
81.82
73.33
77.78
71.00
80.81
74.67
73.74
71.00
λ=0.5 Patch
60.61
70.33
65.66
72.00
66.67
72.33
65.66
72.67
52.53
72.67
Table 9: LoRA-Patch under defense-aware adaptive training on Llama-3-8B-Instruct. Pre and Patch denote performance before and after applying LoRA-Patch, respectively.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Adapter Family
Time (s) ↓
Peak Memory (GB) ↓
LoRA
14.79
22.89
LoRI-D
14.71
22.89
DoRA
12.77
29.41
Appendix
Table 10: LoRA-Patch preparation cost on Llama-3-8B-Instruct across adapter families.
Attack
No defense
Fine-Tuning
Fine-Pruning
CROW
LoRA-Patch
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
BadNets
73.74
83.00
76.77
83.00
74.75
83.00
70.71
82.67
41.41
82.67
VPI
79.80
82.00
77.78
83.67
79.80
80.67
80.81
80.00
48.48
82.00
Sleeper
77.78
82.00
81.82
82.00
78.79
81.00
74.75
82.67
51.52
82.33
CTBA
73.74
82.67
73.74
82.00
74.75
81.67
73.74
82.33
50.51
83.33
MTBA
62.63
82.33
68.69
82.67
66.67
82.33
64.65
82.67
18.18
82.33
Appendix
Table 11: Defense comparison on CommonsenseQA using Llama-3-8B-Instruct.
Window
Layer Indices
Early layers
{0,…,7}
Middle layers
{8,…,15}
Late-middle layers
{16,…,23}
Last 4 layers
{28,…,31}
Last 6 layers
{26,…,31}
Late layers
{24,…,31}
Appendix
Table 12: Layer indices for each steering window configuration used in the aligned-layer ablation on Llama-3-8B-Instruct.
Figure 4: Layer-wise refusal-direction stability sl on Llama-3-8B-Instruct. The clean adapter (green) maintains higher stability than the five tested backdoored adapters (dashed), with the separation becoming more pronounced in the late layers.
Figure 5: Layer-wise refusal-direction stability sl on Mistral-7B-Instruct. The clean adapter (green) exhibits a distinct trajectory from the five tested backdoored adapters (dashed), with the separation becoming more pronounced in the late layers.
LoRA adapters let users fine-tune large language models (LLMs) efficiently. However, LoRA adapters are shared through open repositories like Hugging Face Hub \citep{huggingface_hub_docs}, making them vulnerable to backdoor attacks. Current detection methods require running the model with test input data -- making them impractical for screening thousands of adapters where the trigger for backdoor behavior is unknown. We detect poisoned adapters by analyzing their weight matrices directly, without running the model -- making our method trigger-agnostic. For each attention projection (Q, K, V, O), our method extracts five spectral statistics from the low-rank update ΔW, yielding a 20-dimensional signature for each adapter. A logistic regression detector trained on this representation separates benign and poisoned adapters across three model families -- Llama-3.2-3B~\citep{llama3}, Qwen2.5-3B~\citep{qwen25}, and Gemma-2-2B~\citep{gemma2} -- on unseen test adapters drawn from instruction-following, reasoning, question-answering, code, and classification tasks. Across all three architectures, the detector achieves 100% accuracy.
David Puertolas Merenciano, Ekaterina Vasyagina, Kevin Zhu +2
Low-rank adaptation (LoRA) enables efficient specialization and distribution of large language models through compact adapters. However, untrusted adapters introduce a supply-chain threat: a backdoored adapter can cause a model to generate harmful content, malicious code, political propaganda, or covert advertisements when an input contains a hidden trigger. Adapter-agnostic defenses merge the adapter with the base model, which dilutes backdoor signals and reduces detection performance. Existing adapter-aware methods do not address how to safely use a potentially backdoored adapter. Instead, they either train a defensive adapter to repair a backdoored base model, addressing the inverse problem rather than securing the adapter itself, or rely on a classifier that flags the entire adapter as suspicious and requires separate mitigation. These methods overlook the distinct latent-space signatures produced by trigger-bearing inputs in backdoored adapters. We introduce LoRAScan, the first adapter-aware defense that detects and rejects trigger-bearing inputs at inference time without modifying adapter parameters. Our key observation is that a small subset of LoRA insertion sites, approximately 5%, remains stable across clean inputs but exhibits highly concentrated spikes in LoRA down-projection activations when a trigger is present. LoRAScan identifies these low-variance insertion sites before model deployment and monitors them during inference. Across standard LLM backdoor benchmarks, LoRAScan rejects 98.49% of malicious inputs with a small error rate on clean inputs, outperforming existing defenses across diverse evaluation settings.
Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo +1
Michigan Technological University · Miami University · Lehigh University
We show that LoRA adapters, the dominant distribution format for fine-tuned LLMs, can be reliably backdoored through training data poisoning while preserving baseline task performance. On a Qwen 2.5 1.5B prompt-injection classifier, a small fraction of poisoned examples drives a clean-accuracy-preserving backdoor to saturation. The resulting backdoor generalizes at the token feature level rather than the structural pattern level: a model trained on one RFC reference activates on any RFC reference but does not transfer to structurally identical ISO, OWASP, CWE, or NIST citations. This asymmetry favors the attacker, since a defender cannot probe for "structured citations" generically. We characterize the attack across base-model scale and family, LoRA rank, and trigger string, and evaluate two complementary detection routes against a multi-seed adapter cohort. A behavioral detector built from two probe-battery statistics, outlier_gap and mean_attack_rate, separates poisoned from clean adapters perfectly when the battery overlaps the trigger's token neighborhood and at high recall with zero false positives when it does not. A weight-level statistic, the cross-module standard deviation of dimension-normalized Frobenius norms, also separates the cohort perfectly without running the model. Combined, the two routes are robust to probe composition. Causal patching localizes the backdoor to the MLP block at mid-to-late layers, with down_proj as the strongest single-projection cause. Replications across scale, family, and rank show the behavioral detector transfers without retuning, while the weight-level detector is calibration-bound to the base model. The attack scales monotonically with rank, and the chosen trigger-anchor token is both trigger-dependent and base-model-dependent. Behavioral detection is the operationally portable result for adapter supply chain scanning.