How little of a pretrained large language model (LLM) has to change for it to acquire new capabilities? We find that zeroing fewer than 0.05% of its weights, with no other modification, matches full fine-tuning (FFT) at scale across three LLM post-training regimes: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. Our method, Deletion Tuning, learns a binary mask over the gate projection of transformer blocks. Two practical advantages follow: the trained mask is hundreds to thousands of times smaller than the corresponding FFT checkpoint, and Deletion Tuning forgets far less, which addresses a central challenge in continual learning. Under reinforcement learning, its prior-task accuracy stays near the base level while FFT, LoRA, and random sparse fine-tuning lose up to 5.4 points. To understand why masking alone suffices, we analyze the geometry of the learned weight changes under reinforcement learning using stable rank, spectrum drift, and principal-weight overlap. Compared with LoRA and random sparse fine-tuning, Deletion Tuning is closest to FFT across the three measures.
Figures & tables
Figure 1: Deletion Tuning learns which pretrained weights to delete and writes no new values. The pretrained weights W stay frozen, and each is paired with a trainable logit in G , the only parameters that post-training updates. During training, each weight is multiplied by σ(αG) , a smooth mask between 0 and 1 . The sharpness α rises over training until the sigmoid becomes a hard threshold at G=0 . At deployment, weights with positive logits ( blue ) keep their pretrained values and weights with negative logits ( red ) are set to zero (crossed out), so the trained adapter is one bit per weight.
Llama, GSM8K
Qwen, SimpleRL-Zoo
1B
3B
8B
0.5B
3B
7B
Method
Acc.
Size
Acc.
Size
Acc.
Size
Acc.
Size
Acc.
Size
Acc.
Size
Base
1.21
0
53.53
0
60.27
0
10.44
0
37.73
0
40.18
0
FFT
60.50
4714
82.49
12256
83.24
30633
19.01
1885
44.35
11772
51.89
29051
LoRA r=1
59.29
2.7
82.39
5.8
83.38
10.0
20.30
2.1
43.55
7.1
52.13
9.6
LoRA r=4
60.58
10.8
82.44
23.2
82.87
40.0
20.30
8.4
43.59
28.6
51.70
38.5
Table 1: Deletion Tuning has the smallest adapter at every scale and matches FFT at the largest backbone of each family. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO, on GSM8K for Llama and on the six-benchmark SimpleRL-Zoo mean for Qwen. The baselines are full fine-tuning (FFT), low-rank adapters (LoRA), and random sparse fine-tuning (MTH).
Llama8B
Qwen7B
Qwen14B
Qwen3-30B-A3B
GSM8K
SimpleRL-Zoo
SimpleRL-Zoo
SimpleRL-Zoo
Method
Acc.
Size
Acc.
Size
Acc.
Size
Acc.
Size
Base
60.27
0
40.18
0
44.37
0
49.41
0
FFT
83.24
30633
51.89
29051
54.87
56343
58.43
116471
Deletion Tuning
83.55
8.4
52.32
6.9
55.43
11.4
58.54
29.6
Table 2: Deletion Tuning matches FFT on dense and mixture-of-experts models from 7B to 30B with an adapter thousands of times smaller. Best-checkpoint validation accuracy (%) and adapter size (MB) under GRPO. Qwen14B and Qwen3-30B-A3B use the SimpleRL-Zoo setup of Qwen7B.
Llama8B, GSM8K
Qwen7B, SimpleRL-Zoo
Method
k=1
2
8
32
128
k=1
2
8
32
128
Base
53.20
66.75
83.64
92.40
95.29
27.35
38.75
56.18
66.26
74.01
Deletion Tuning
81.75
88.51
95.75
98.44
99.62
49.39
53.22
64.64
73.23
80.08
Table 3: Deletion Tuning acquires new capabilities. Pass@k (%), the share of problems solved within k attempts, estimated from 128 samples per problem at temperature 1.0 . A lead at large k means solving problems the base model misses even with many attempts.
Llama, GSM8K
Qwen, SimpleRL-Zoo
Method
1B
3B
8B
0.5B
3B
7B
Base
1.21
53.53
60.27
10.44
37.73
40.18
Wanda (best η )
1.59
52.46
60.65
11.42
38.70
42.49
Deletion Tuning
54.44
81.49
83.55
17.64
41.98
52.32
Table 4: Wanda alone stays near the base model. Validation accuracy (%) of the base model, of Wanda one-shot pruning at the best of eight sparsity levels, and of the best checkpoint of the Deletion Tuning mask trained with GRPO.
Figure 2: Deletion Tuning keeps prior-task accuracy at every Llama scale while the baselines drift. (a) Forward KL between the trained and base policies on GSM8K during GRPO training, the forgetting predictor of Shenfeld et al., 2026b () . (b) Change in mean accuracy on a six-task prior-task suite from the base model (dashed line).
Figure 3: Deletion Tuning (blue) tracks FFT (red) most closely across three update-geometry metrics. Values at the saved GRPO checkpoint for Llama8B (filled) and Qwen7B (open): (a) stable rank of ΔW , (b) spectrum drift, and (c) principal-weight overlap. Overlap below the random baseline of 0.5 is the RLVR signature reported by Zhu et al., (2026) .
MMLU-Pro
BBEH
1B
3B
1B
3B
Method
Acc.
Size
Acc.
Size
Acc.
Size
Acc.
Size
Base
10.37
0
12.63
0
2.17
0
8.87
0
FFT
16.97
4714
21.19
12256
8.35
4714
11.39
12256
Deletion Tuning
18.62
1.0
22.28
3.1
5.65
1.0
11.39
3.1
Table 5: Under SFT, Deletion Tuning matches FFT at 3B. Best-checkpoint evaluation accuracy (%), averaged over the 14 MMLU-Pro subjects and the 23 BBEH subtasks, and adapter size (MB) for Llama1B and Llama3B.
ToolAlpaca
Science
Method
Acc.
Size
Acc.
Size
Base
41.24
0
32.35
0
FFT
68.04
29051
71.20
29051
Deletion Tuning
69.07
27.1
70.61
47.1
Table 6: Under SDFT, Deletion Tuning matches FFT on Qwen7B. Best-checkpoint exact-match accuracy (%) and adapter size (MB) on ToolAlpaca tool use and Science.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Llama, GSM8K
Qwen, SimpleRL-Zoo
1B
3B
8B
0.5B
3B
7B
Earliest saved
0.0342%
0.0420%
0.0456%
0.0393%
0.0379%
0.0381%
Final saved
0.0347%
0.0422%
0.0459%
0.0405%
0.0418%
0.0415%
Appendix
Table 7: Fraction of all weights deleted by Deletion Tuning in the six GRPO runs. Fraction of all model parameters set to zero by the trained mask, at the earliest and final saved checkpoint. Every value is below 0.05% .
Method
GSM8K
MATH500
Minerva
OlympiadBench
AIME24
AMC23
Qwen0.5B
FFT
48.07
36.00
4.78
7.73
0.00
17.50
LoRA r=1
49.20
33.80
7.35
8.92
0.00
22.50
LoRA r=4
52.46
36.60
8.82
10.55
3.33
10.00
MTH 99%
50.34
35.20
10.29
9.36
3.33
22.50
MTH 99.95%
49.73
35.20
6.25
7.28
0.00
22.50
Appendix
Table 8: Qwen2.5 on SimpleRL-Zoo: per-benchmark validation accuracy (%) at the best checkpoint. Six benchmarks: GSM8K, MATH500, Minerva Math, OlympiadBench, AIME24, AMC23.
Category
Train
Test
Category
Train
Test
biology
537
180
law
825
276
business
591
198
math
1013
338
chemistry
849
283
other
693
231
computer science
307
103
philosophy
374
125
economics
633
211
physics
974
325
engineering
726
243
psychology
598
200
Appendix
Table 9: Per-task train/test split for MMLU-Pro under the 75/25 split. Total: 9,018 train / 3,014 test across 14 tasks.
Task group
Train
Test
22 tasks ( 200 examples each)
150 each
50 each
disambiguation_qa ( 120 examples)
90
30
Appendix
Table 10: Per-task train/test split for BBEH under the 75/25 split. Twenty-two tasks share the 200 -example pattern. The only outlier is disambiguation_qa with 120 examples. Total: 3,390 train / 1,130 test across 23 tasks.
Table 14Table 15
Method
Learning rate
KL coefficient
Llama8B, GSM8K
Qwen7B, SimpleRL-Zoo
FFT
5×10−7
0
83.23
50.55
FFT
5×10−7
10−4
82.55
51.89
FFT
10−6
0
83.24
51.93
FFT
10−6
10−4
81.89
50.79
Deletion Tuning
3×10−4
as in Table 1
83.55
52.32
Appendix
Table 15: FFT sweep under GRPO. Best-checkpoint validation accuracy (%), as in Table 1 .
Method
Learning rate
Accuracy
Base
–
12.63
FFT
10−5
19.35
FFT
3×10−6
21.19
FFT
10−6
18.35
Deletion Tuning
5×10−4
22.28
Appendix
Table 16: FFT learning-rate sweep under SFT, Llama3B MMLU-Pro. Best-checkpoint accuracy (%), as in Table 5 .
Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.
Large language models (LLMs) acquire new skills through fine-tuning, but it can destroy capabilities they already have: catastrophic forgetting remains a central obstacle to continual learning in LLMs. Recent works connect forgetting to how far, and along which directions, fine-tuning moves the weights. On-policy methods such as reinforcement learning and on-policy distillation avoid updating the principal singular directions of weight matrices and thus forget less. In contrast, SFT concentrates updates along these directions and is susceptible to catastrophic forgetting. In this work, we introduce Subspace Tuning (SST), which restricts each weight update to the bottom singular subspace, eliminating changes along the principal directions. SST is optimizer-agnostic and composes well with fine-tuning recipes including SFT and self-distillation. SST combines tight subspace protection, dense gradient estimation, and a simple refresh mechanism, enabling stronger learning than prior orthogonal fine-tuning methods. With these mechanisms, protecting as much as 99% of the weight subspace substantially improves retention while preserving strong target-task performance. This behavior strengthens with model and data size and nearly eliminates forgetting in the large-model regime. SST also keeps the model close to its base distribution, inducing 2--4× less KL drift than the baseline recipes. Overall, SST advances the learning--forgetting Pareto frontier, making it a practical default for fine-tuning pre-trained language models.
Recent work identified Super Weights, individual parameters whose removal degrades model performance by orders of magnitude. We show that this degradation due to pruning Super Weights does not universally apply to all LLMs. Furthermore, if these parameters are so important, Super Weight-aware training should be effective. We show the opposite. Training Super Weights in isolation (100 to 8,192 parameters) drops accuracy to random-guessing levels on both OLMo-1B and OLMo-7B, and expanding to local neighborhoods of up to 36K parameters provides no improvement. The failure is specific to Super Weight coordinates: training an equal number of randomly chosen positions in the same down_proj layers instead improves over the baseline, so the collapse comes from targeting Super Weights, not from sparsity itself. Vanilla LoRA, updating every position in attention weight matrices through low-rank structure, succeeds with only 0.16% of parameters, and applying the same low-rank update to down_proj succeeds as well. A 10-seed ablation confirms that constraining LoRA updates at positions corresponding to Super Weight coordinates yields statistically indistinguishable results. These findings establish that parameter importance does not imply parameter trainability in isolation, and that effective fine-tuning relies on structured decompositions over entire layers rather than targeting individually important weights.
The large language model (LLM) is typically integrated into the mainstream optimization protocol. However, it remains underexplored whether maintaining the model integrity is indispensable for promising performance. In this work, we introduce Mask Fine-Tuning (MFT), a novel LLM fine-tuning paradigm demonstrating that carefully breaking the model's structural integrity can surprisingly improve performance without updating model weights. MFT learns and applies binary masks to well-optimized models, using the standard LLM fine-tuning objective as supervision. Based on fully fine-tuned models, MFT uses the same fine-tuning datasets to achieve consistent performance gains across domains and backbones (e.g., an average gain of 2.70/4.15 on IFEval with LLaMA2-7B/3.1-8B). Detailed ablation studies and analyses examine the proposed MFT from different perspectives, including the sparse ratio and the loss surface. Additionally, when deployed on well-trained models, MFT is compatible with other LLM optimization procedures to improve overall model performance. Furthermore, this study extends the masking operation beyond its conventional use in network pruning for model compression to encompass a broader range of model capabilities.
Mingyuan Zhang, Yue Bai, Huan Wang +4
College of Engineering, Northeastern University · Khoury College of Computer Science, Northeastern University