Third-party LoRA adapters introduce serious backdoor risks: malicious adapters can maintain normal task performance while embedding hidden, trigger-activated behaviors, making them difficult to detect through standard utility evaluations. Existing removal-based defenses typically sanitize backdoored models by updating, pruning, distilling, or mixing model parameters. However, these approaches are misaligned with adapter-based deployment, where the trusted base model is expected to remain frozen. In this work, we propose LoRA-Patch, a lightweight backdoor defense tailored to LoRA adapters. Our key insight is that backdoor injection distorts the directional geometry of a poisoned adapter relative to its trusted base model. Building on this insight, LoRA-Patch suppresses trigger-activated behaviors by realigning adapter directions through layer-wise steering, without modifying the base model weights. Extensive experiments across five backdoor attacks and three models show that LoRA-Patch effectively mitigates trigger-activated backdoors, outperforming existing baselines while preserving benign task performance. Moreover, LoRA-Patch is computationally efficient, constructing patches within 20 seconds across all evaluated models and adapters. Our code is publicly available at https://github.com/FOXVEDA/LoRA-Patch.
Figures & tables
Figure 1: Layer-wise refusal-direction stability sl between the trusted base model and each adapted model across decoder layers on Llama-3.1-8B-Instruct. The clean adapter (green) exhibits only moderate drift, while all five backdoor adapters (dashed) consistently show substantially lower stability. This separation supports using refusal-direction stability as a trigger-free geometric indicator for adapter-level backdoor defense. More visualization results are provided in Appendix C .
Figure 2: Overview of the LoRA-Patch framework. LoRA-Patch consists of two steps: (1) Refusal Direction Probing , which estimates layer-wise refusal directions from the trusted base model and the suspect LoRA-adapted model using harmful and harmless probes; and (2) Refusal Geometry Realignment , which constructs late-layer steering directions anchored to the trusted base refusal geometry. The resulting steering vectors are stored as a lightweight patch to suppress trigger-activated behaviors while preserving benign task utility.
Model
Attack
No defense
Fine-Tuning
Fine-Pruning
CROW
LoRA-Patch (Ours)
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
Llama-3-8B Instruct
BadNets
71.72
71.33
71.72
67.00
73.74
72.67
67.68
63.33
22.22
72.33
VPI
77.78
71.67
78.79
71.67
80.81
70.00
73.74
70.33
55.54
68.67
Sleeper
82.83
72.00
78.79
68.67
85.86
71.00
76.77
71.67
44.42
70.00
CTBA
82.83
75.00
80.81
72.00
86.87
73.00
79.80
74.00
46.44
67.00
MTBA
68.69
71.33
80.81
67.67
73.74
72.67
74.75
68.67
26.24
73.33
Table 1: Attack success rate (ASR, %) and downstream task accuracy (ACC, %) of LoRA-Patch and baseline defenses against five backdoor attacks across three backbone models. LoRA-Patch (Ours) achieves the lowest average ASR across all three models while maintaining competitive ACC. For LoRA-Patch, we set (αb,αo)=(1.1,0.6) .
Defense
Time (s) ↓
Peak Memory (GB) ↓
Fine-Tuning
48.80
21.59
Fine-Pruning
19.71
18.78
CROW
82.78
24.36
LoRA-Patch (Ours)
14.79
22.89
Table 2: Efficiency comparison of LoRA-Patch and baseline defenses on Llama-3-8B-Instruct. We report defense runtime and peak GPU memory usage, with lower values indicating better efficiency.
Model
Attack
No defense
LoRA-Patch
ASR ↓
ACC ↑
ASR ↓
ACC ↑
Llama-3-8B Instruct
BadNets
73.74
83.00
41.41
82.67
VPI
79.80
82.00
48.48
82.00
Sleeper
77.78
82.00
51.52
82.33
CTBA
73.74
82.67
50.51
83.33
MTBA
62.63
82.33
18.18
82.33
Table 3: Attack success rate (ASR, %) and task accuracy (ACC, %) of LoRA-Patch on CommonsenseQA.
Steering Window
BadNets
VPI
ASR ↓
ACC ↑
ASR ↓
ACC ↑
None
71.72
71.33
77.78
71.67
Early layers
100.00
0.00
100.00
0.00
Middle layers
0.00
0.00
0.00
0.00
Late-middle layers
1.01
0.00
0.00
2.00
Late layers (Ours)
22.22
72.33
55.54
68.67
Table 4: Effect of different steering windows selections on Llama-3-8B under BadNets and VPI attacks.
Figure 3: Steering-strength sensitivity of LoRA-Patch on Llama-3-8B under the BadNets attack. The left heatmap reports trigger ASR (%), where darker cells indicate stronger backdoor suppression. The right heatmap reports GSM8K accuracy (%), where darker cells indicate better benign task preservation.
Model
ACC ↑
No defense
LoRA-Patch
Llama-3-8B-Instruct
72.33
72.00
Llama-3.1-8B-Instruct
70.00
70.67
Mistral-7B-Instruct
61.67
60.00
Table 5: GSM8K accuracy (ACC, %) on benign models before and after applying LoRA-Patch.
Defense
Attack
LoRA
LoRI-D
DoRA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
No defense
BadNets
71.72
71.33
68.69
72.33
77.78
72.67
VPI
77.78
71.67
71.72
73.00
83.84
68.33
Sleeper
82.83
72.00
79.80
72.33
88.89
71.33
CTBA
82.83
75.00
80.81
75.33
88.89
74.00
MTBA
68.69
71.33
66.67
72.33
74.75
70.33
Table 6: Generalization of LoRA-Patch across different adapter variants on Llama-3-8B-Instruct.
Harmless Probe
BadNets
VPI
Sleeper
CTBA
MTBA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
GSM8K (task-specific)
22.22
72.33
55.54
68.67
44.42
70.00
46.44
67.00
26.24
73.33
Generic Alpaca
16.16
72.33
34.34
70.67
29.29
69.00
24.24
71.00
10.10
72.00
Mismatched CSQA
25.25
71.33
48.48
70.67
41.41
71.67
45.45
71.67
31.31
72.67
Multi-task
18.18
69.67
39.39
72.00
39.39
72.33
33.33
73.67
17.17
71.33
Table 7: Robustness to different harmless-probe sources on Llama-3-8B-Instruct.
Attack
AdvBench
MaliciousInstruct
HarmBench
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
BadNets
22.22
72.33
29.29
70.33
26.26
70.33
VPI
55.54
68.67
51.52
67.67
54.55
69.33
Sleeper
44.42
70.00
44.44
72.33
41.41
72.67
Table 8: Sensitivity to harmful-probe sources on Llama-3-8B-Instruct.
Setting
BadNets
VPI
Sleeper
CTBA
MTBA
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
λ=0 Pre
71.72
71.33
80.81
69.67
79.80
74.33
81.82
70.67
80.81
71.00
λ=0 Patch
21.21
69.00
45.45
68.00
41.41
73.00
39.39
72.00
29.29
71.00
λ=0.5 Pre
81.82
73.67
81.82
73.33
77.78
71.00
80.81
74.67
73.74
71.00
λ=0.5 Patch
60.61
70.33
65.66
72.00
66.67
72.33
65.66
72.67
52.53
72.67
Table 9: LoRA-Patch under defense-aware adaptive training on Llama-3-8B-Instruct. Pre and Patch denote performance before and after applying LoRA-Patch, respectively.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Adapter Family
Time (s) ↓
Peak Memory (GB) ↓
LoRA
14.79
22.89
LoRI-D
14.71
22.89
DoRA
12.77
29.41
Appendix
Table 10: LoRA-Patch preparation cost on Llama-3-8B-Instruct across adapter families.
Attack
No defense
Fine-Tuning
Fine-Pruning
CROW
LoRA-Patch
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
ASR ↓
ACC ↑
BadNets
73.74
83.00
76.77
83.00
74.75
83.00
70.71
82.67
41.41
82.67
VPI
79.80
82.00
77.78
83.67
79.80
80.67
80.81
80.00
48.48
82.00
Sleeper
77.78
82.00
81.82
82.00
78.79
81.00
74.75
82.67
51.52
82.33
CTBA
73.74
82.67
73.74
82.00
74.75
81.67
73.74
82.33
50.51
83.33
MTBA
62.63
82.33
68.69
82.67
66.67
82.33
64.65
82.67
18.18
82.33
Appendix
Table 11: Defense comparison on CommonsenseQA using Llama-3-8B-Instruct.
Window
Layer Indices
Early layers
{0,…,7}
Middle layers
{8,…,15}
Late-middle layers
{16,…,23}
Last 4 layers
{28,…,31}
Last 6 layers
{26,…,31}
Late layers
{24,…,31}
Appendix
Table 12: Layer indices for each steering window configuration used in the aligned-layer ablation on Llama-3-8B-Instruct.
Figure 4: Layer-wise refusal-direction stability sl on Llama-3-8B-Instruct. The clean adapter (green) maintains higher stability than the five tested backdoored adapters (dashed), with the separation becoming more pronounced in the late layers.
Figure 5: Layer-wise refusal-direction stability sl on Mistral-7B-Instruct. The clean adapter (green) exhibits a distinct trajectory from the five tested backdoored adapters (dashed), with the separation becoming more pronounced in the late layers.