Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Authors: Quoc Minh Nguyen, Trung Le, Jing Wu, Anh Tuan Bui, Mehrtash Harandi
Organizations: Department of Electrical and Computer Systems Engineering, Monash University, Australia · Department of Data Science and AI, Monash University, Australia
Fine-tuning-as-a-service introduces a threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, a process known as harmful fine-tuning attacks. In this work, we show that by regularizing the gradient contribution of harmful samples encountered during fine-tuning, we can effectively mitigate the impact of harmful fine-tuning attacks. To this end, we introduce Antibody, a defense strategy that first ensures robust safety alignment for the model before fine-tuning, and then applies a safety-preservation learning algorithm during fine-tuning. Specifically, in the alignment stage before fine-tuning, we propose optimizing the model to be in a flat loss region with respect to harmful samples, which makes the safety alignment more resilient to subsequent harmful fine-tuning. Then, in the fine-tuning stage, we design a fine-tuning algorithm that applies a weighting scheme to all samples in each training batch to inhibit the model from learning from harmful samples while encouraging learning from benign samples. Experimental results demonstrate that Antibody successfully mitigates harmful fine-tuning attacks while boosting fine-tuning performance on the user-submitted dataset.
Figures & tables
Figure 1: Fine-tuning on GSM8K ( Cobbe et al., 2021 ) with varying sample sizes and a fixed harmful ratio of 20% . Larger sample sizes improve fine-tuning accuracy (higher FA) but degrade model safety (higher HS).
Figure 2: The effect of our proposed fine-tuning method. Left and middle plots show the score ( rθ ) distribution before and after fine-tuning, while the right plot compares the fine-tuning loss of our method (Antibody) against SFT on benign and harmful samples in the fine-tuning dataset.
Figure 3: Harmful score with different fine-tuning epochs (Left) and learning rates (Right).
Method
GSM8K HS ↓
GSM8K FA ↑
SFT
23.94
10.90
+ Align with λtLsharp(θt)
13.02 ( −10.92 )
16.00 ( +5.20 )
+ Fine-tune with wθt
4.44 ( −19.50 )
13.83 ( +2.93 )
+ Align with Lrefusal(θt) (Antibody)
1.24 ( −22.70 )
15.07 ( +4.17 )
Table 4: Ablation on our proposed components. Each row is a cumulative addition to the previous one. We color-code the performance change from the SFT baseline.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
Clock Time (Hours)
GPU Memory (GB)
(Llama-2-7B)
Alignment
Fine-tuning
Sum
Alignment
Fine-tuning
Max
SFT
1.00
0.26
1.26
53.68
40.10
53.68
Vaccine
1.98
0.16
2.12
54.70
40.10
54.70
Lisa
1.00
0.26
1.26
53.68
53.87
53.87
Booster
5.09
0.26
5.35
64.36
40.10
64.36
Antibody
5.81
0.46
6.27
64.55
42.53
64.55
Appendix
Table 9: System evaluation for different methods.
Figure 4: Effect of our flatness-regularized alignment from Section 4.1 . We plot the distribution of per-sample gradient norms at the beginning of the fine-tuning stage (before fine-tuning) for models aligned with Antibody’s alignment-stage solution (left) and with standard SFT (right).
Figure 5: The effect of our proposed fine-tuning method under a harmful fine-tuning attack with OOD harmful data. The plots illustrate the distribution of the score rθ (the pre-normalized weight) in Equation 7 for benign samples versus harmful samples before and after fine-tuning on a dataset poisoned with PureBad ( Qi et al., 2024 ) .
Fine-tuning-as-a-Service (FaaS) enables personalization of large language models (LLMs), but it can weaken safety-alignment under harmful fine-tuning attacks. Recent work has shown that activating harmful-behavior modules during fine-tuning can prevent models from learning undesired behaviors, but its mechanism remains unclear. In this paper, we revisit temporary jailbreaking as a defense against harmful fine-tuning and provide a gradient-level analysis showing that it saturates safety-degrading gradients while preserving benign task-relevant gradients. Based on this insight, we propose a Buffer-and-Reinforce fine-tuning framework that buffers harmful updates during user fine-tuning and reinforces safety after adaptation. Specifically, BufferLoRA induces temporary jailbreaking as a removable adapter to reduce harmful updates during user fine-tuning. After adaptation, ReinforceLoRA, trained to recover refusal behavior under the temporarily jailbroken state, is integrated with UserLoRA via QR decomposition-based merging to reinforce safety while preserving user-task performance. Extensive experiments show that our framework achieves superior safety and utility with no additional safety data during user fine-tuning and minimal computational cost.
Seokil Ham, Jaehyuk Jang, Wonjun Lee +1
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
Fine-tuning large language models often undermines their safety alignment, a problem further amplified by harmful fine-tuning attacks in which adversarial data removes safeguards and induces unsafe behaviors. We propose SPARD, a defense framework that integrates Safety-Projected Alternating optimization with Relevance-Diversity aware data selection. SPARD employs SPAG, which optimizes alternatively between utility updates and explicit safety projections with a set of safe data to enforce safety constraints. To curate safe data, we introduce a Relevance-Diversity Determinantal Point Process to select compact safe data, balancing task relevance and safety coverage. Experiments on GSM8K and OpenBookQA under four harmful fine-tuning attacks demonstrate that SPARD consistently achieves the lowest average attack success rates, substantially outperforming state-of-the-art defense methods, while maintaining high task accuracy. Code is available at https://github.com/shuhao02/SPARD.
Shuhao Chen, Weisen Jiang, Yeqi Gong +5
Department of Computer Science and Engineering, Southern University of Science and Technology · Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Department of Computer Science and Engineering, The Chinese University of Hong Kong +2
The safety alignment of Large Language Models (LLMs) remains vulnerable to Harmful Fine-tuning (HFT). While existing defenses impose constraints on parameters, gradients, or internal representations, we observe that they can be effectively circumvented under persistent HFT. Our analysis traces this failure to the inherent redundancy of the high-dimensional parameter space: attackers exploit optimization trajectories that are orthogonal to defense constraints to restore harmful capabilities while deceptively adhering to safety restrictions. To address this, we propose Safety Bottleneck Regularization (SBR). SBR shifts the defensive focus from the redundant parameter space to the unembedding layer, which serves as a geometric bottleneck. By anchoring the final hidden states of harmful queries to those of the safety-aligned model, SBR enables the model to maintain safe responses even under persistent HFT. Extensive experiments confirm SBR's effectiveness, demonstrating that utilizing just a single safety anchor is sufficient to reduce the Harmful Score to <10 while preserving competitive performance on benign downstream tasks.
Guoxin Lu, Letian Sha, Qing Wang +4
Nanjing University of Posts and Telecommunications, Nanjing, China.