Model Immunization Beyond Condition Numbers: The Importance of Plateau Regions
Authors: Quoc Nguyen, Trung Le, Dinh Phung, Mehrtash Harandi
Organizations: Department of Electrical and Computer Systems Engineering, Monash University, Australia · Department of Data Science and AI, Monash University, Australia
Harmful fine-tuning can rapidly compromise safety alignment in Large Language Models, underscoring the vulnerability of deployed models to post-release modification. While prior work motivates model immunization by inducing ill-conditioned curvature to slow harmful optimization, this approach is not scalable due to the high computational cost of Hessian estimation. We refine this paradigm by extending the theoretical framework to prioritize plateau regions alongside curvature. We further introduce an immunization algorithm based on efficient Hessian estimation and bounds on the extremal singular values of the Hessian. We demonstrate the proposed immunization framework in three applications: (i) Fine-tuning-as-a-service settings where benign user data may be mixed with poisoned harmful data, (ii) open-weight settings where an adversary can fine-tune on harmful-only data with varied training dynamics, and (iii) robust unlearning, where immunized models better resist relearning attacks after unlearning. Extensive experiments confirm that our approach achieves robust defense against adversarial fine-tuning while largely maintaining general utility and downstream trainability. Our code is available at https://github.com/minhquoc0712/BECON.
Figures & tables
Model
Acc (%)
FA (%)
MMLU
TruthfulQA
ARC
Hellaswag
SST2
AGNEWS
GSM8K
Non-immunized model
45.66
38.94
76.30
58.96
93.90
84.10
18.17
BECON
45.14
42.54
71.74
55.26
93.50
88.47
15.97
Table 5: Comparison of the utility and fine-tuning capabilities of the non-immunized model and model immunized with BECON.
Figure 1: Robustness evaluation of unlearning methods on WMDP against relearning attacks. Left: UE vs. relearning epochs. Right: UE vs. number of relearning samples.
Method
GSM8K HS ↓
GSM8K FA ↑
SFT
23.94
10.90
SFT + Lcn-min−Lcn-max
20.13 ( −3.81 )
15.00 ( +4.10 )
SFT + Lplateau
10.40 ( −13.54 )
15.07 ( +4.17 )
BECON
3.72 ( −20.22 )
15.43 ( +4.53 )
Table 6: Ablation of the proposed components on GSM8K. The intermediate rows add each component separately to SFT. BECON combines both. Colored values in parentheses indicate changes from the SFT baseline.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Method
SST2
AGNEWS
GSM8K
HS ↓
FA ↑
HS ↓
FA ↑
HS ↓
FA ↑
Booster
14.31
92.59
15.88
86.70
9.06
16.27
BECON ( harmful plateau only )
13.07
92.81
15.93
86.67
8.92
15.30
Appendix
Table 15: Effect of the harmful plateau promotion term relative to Booster in the FTaaS setting on Llama2-7B. Results are averaged over three random seeds. HS denotes Harmful Score, and FA denotes Fine-tuning Accuracy.
Method
HS ↓
FA ↑
Antibody ( alignment stage only )
10.87
15.40
BECON
3.72
15.43
Appendix
Table 17: Comparison with Antibody’s alignment-stage solution in the FTaaS setting on Llama2-7B and GSM8K.
Figure 2: 1D ASR safety landscapes. Note the smaller y-axis range for BECON compared to Booster.
Method (Qwen2-7B)
SST2 HS ↓
AGNEWS HS ↓
GSM8K HS ↓
SFT
33.86
31.00
14.54
BECON ( condition-number only )
7.29 ± 1.03
7.10 ± 0.93
3.62 ± 1.47
Appendix
Table 19: Harmful score of BECON ( condition-number only ) on Qwen2-7B across three random seeds, reported as mean ± standard deviation. SFT values are taken from Table 18 . HS denotes Harmful Score.
Figure 3: Evolution of σmin , σmax , and the condition number κ=σmax/σmin during the immunization process of our method. Due to the substantial computational cost of estimating the condition number, which involves Hessian computations, we conduct this experiment on the Qwen3-0.6B model and visualize the metrics for a single LoRA module in the final block, specifically the LoRA of the vproj matrix. The decrease in σmin and the increase in σmax together produce the desired effect of an increasing condition number.
Method (Llama2-7B)
Pre-attack HS ↓
Post-attack HS (GSM8K) ↓
SFT
0.81 ± 0.30
23.94
BECON
0.67 ± 0.22
3.72
Appendix
Table 20: Harmful score before and after the fine-tuning attack on Llama2-7B in the FTaaS setting of Section 5.1 , with GSM8K as the fine-tuning dataset. The pre-attack harmful score is measured right after alignment for SFT and right after immunization for BECON, and is reported as mean ± standard deviation over three random seeds. Post-attack values are taken from Table 1 . HS denotes Harmful Score.
Method
Clock Time (hours)
GPU Memory (GB)
Framework in [ 8 ]
1.53
79.53
BECON
5.70
59.96
Appendix
Table 23: System cost of different condition-number-based methods on Llama2-7B.
Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.
Fine-tuning-as-a-service introduces a threat to Large Language Models' safety when service providers fine-tune their models on poisoned user-submitted datasets, a process known as harmful fine-tuning attacks. In this work, we show that by regularizing the gradient contribution of harmful samples encountered during fine-tuning, we can effectively mitigate the impact of harmful fine-tuning attacks. To this end, we introduce Antibody, a defense strategy that first ensures robust safety alignment for the model before fine-tuning, and then applies a safety-preservation learning algorithm during fine-tuning. Specifically, in the alignment stage before fine-tuning, we propose optimizing the model to be in a flat loss region with respect to harmful samples, which makes the safety alignment more resilient to subsequent harmful fine-tuning. Then, in the fine-tuning stage, we design a fine-tuning algorithm that applies a weighting scheme to all samples in each training batch to inhibit the model from learning from harmful samples while encouraging learning from benign samples. Experimental results demonstrate that Antibody successfully mitigates harmful fine-tuning attacks while boosting fine-tuning performance on the user-submitted dataset.
Quoc Minh Nguyen, Trung Le, Jing Wu +2
Department of Electrical and Computer Systems Engineering, Monash University, Australia · Department of Data Science and AI, Monash University, Australia
Released aligned large language models remain vulnerable to malicious downstream finetuning. Existing defenses are largely designed for the fine-tuning-as-a-service (FTaaS) paradigm or rely on downstream users to follow additional safety procedures, and therefore do not directly address the setting we study: a provider controlled partially protected open-weight (PPOW) release setting in which most weights remain trainable while a small safety-critical component is preserved at release. We propose a Unidirectional Safety Gate (USG), instantiated as a Null Space Cubic Layer together with an Inverse Adapter inserted after the final Transformer layer. During downstream fine-tuning, the cubic layer suppresses or blocks gradients from harmful samples whose hidden states fall in a calibrated protected region, while the Inverse Adapter restores the base model's forward behavior. In practice, we calibrate a threshold using defender-held harmful data, allowing protection to generalize to nearby in-distribution harmful samples. Across six evaluated model-dataset settings, USG keeps post-finetuning attack success rate close to the pre-release level under a fixed release threshold, while maintaining high safe-pass rates on easier settings and exhibiting a clearer safety-utility trade-off on unsafe samples from BeaverTails. These results suggest that release-time representation-space blocking can raise the cost of malicious downstream adaptation without requiring downstream cooperation. The code is available at https://github.com/OpenCausaLab/Gradient-Immunity.
Yuxuan Huang, Xingyu Zeng, Tianhang Zheng +1
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Shenzhen University of Advanced Technology +1
Model providers increasingly release the weights of large language models. Although these models are safety-aligned before release, their safeguards can often be removed by fine-tuning on harmful data. A growing class of defenses aims to make alignment robust to such malicious fine-tuning, but these defenses are typically evaluated against attacks with a fixed training budget, even though an attacker who holds the weights can simply train for longer. We ask whether current defenses withstand this simplest escalation. Surveying fifteen recent defenses, we find that they share a common weakness: each is built around a limited model of the attacker, such as a bounded perturbation, a short simulated attack, or a trained link between harmful and benign behavior, and nothing enforces that protection once the weights are released. We then test six representative defenses on four open-weight models by continuing the same harmful-only fine-tuning attack for three epochs and measuring harmfulness and capability along the way. In all 72 defended runs, the model is more harmful at the end of training than at release, and on Llama-3.1 at the highest learning rate the defended models end almost as harmful as the undefended one. The defenses are not equally weak: one defense kept harmfulness low on one model, and some attacks recovered harmfulness only at the cost of general capability. Current defenses can delay or disrupt malicious fine-tuning, but in most cases their measured resistance does not persist under continued training, and they should not yet be treated as durable protection.
Itay Zloczower, Eyal Lenga, Gilad Gressel +1
Ben-Gurion University of the Negev · Amrita Vishwa Vidyapeetham, Amritapuri