cs.CVSep 26, 2026

PruneForget: Joint Unlearning and Pruning of Vision Models

Authors: Yu-Shan Tai, Amber Yijia Zheng, Raymond A. Yeh

Organizations: Department of Computer Science Purdue University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Machine unlearning and model pruning are increasingly coupled in the real world. Models must support unlearning requests, e.g., for safety concerns, while also meeting requirements in latency and memory budget. Until recently, existing works have studied each aspect as an independent problem, e.g., running unlearning and pruning sequentially. In this work, we show that unlearning and pruning are naturally aligned and should be solved jointly to be made aware of each other. Intuitively, parameters that encode information of the unlearned samples are natural pruning targets, as unlearning and pruning both call for the "deletion" of such parameters. We propose PruneForget, a method that uses the unlearn set as a guide for pruning, so that unlearning and pruning mutually benefit each other. Extensive experiments on image classifiers and generative models show that PruneForget removes the influence of the unlearned samples while producing a more compact model with reduced inference cost. It achieves a negligible performance gap relative to an oracle that retrains from scratch for unlearning and then prunes.

Figures & tables

Figure 1: Sequential methods vs . PruneForget. Model pruning typically consists of three stages: sparse training, pruning, and fine-tuning. Sequential baselines ( Unlearn →\to Prune or Prune →\to Unlearn ) treat unlearning and pruning as independent tasks. Our proposed PruneForget integrates unlearning awareness into the three stages by introducing unlearn-aware sparsification, sensitivity-guided pruning, and post-pruning unlearning; achieving a joint unlearning and pruning framework.
MethodRandom Unlearn 10%Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (6.03 ±0.56 )0.00 (100.00 ±0.00 )0.00 (93.74 ±0.32 )0.000.0022150.00 (7.60 ±0.22 )0.00 (100.00 ±0.00 )0.00 (92.20 ±0.30 )0.000.001518
Pretrain →\to 3SP0.89 (5.15 ±0.38 )0.00 (100.00 ±0.00 )0.41 (94.15 ±0.20 )0.430.209841.54 (6.06 ±0.17 )0.00 (100.00 ±0.00 )0.81 (93.01 ±0.27 )0.780.23556
3SP →\to SFRon14.84 (20.88 ±0.77 )1.88 (98.12 ±0.36 )3.38 (90.37 ±0.37 )6.700.2411415.39 (12.99 ±0.54 )0.74 (99.25 ±0.11 )3.42 (88.79 ±0.56 )3.190.27593
SFRon →\to 3SP0.16 (5.87 ±0.57 )0.00 (100.00 ±0.01 )0.04 (93.70 ±0.31 )0.070.2211520.15 (7.45 ±0.21 )0.00 (100.00 ±0.00 )0.34 (91.87 ±0.28 )0.160.27627
ℓ1\ell_{1} -MU →\to 2SP5.00 (11.03 ±0.46 )5.19 (94.81 ±0.26 )4.90 (88.85 ±0.21 )5.030.278134.95 (12.55 ±0.20 )3.43 (96.57 ±0.27 )5.05 (87.16 ±0.16 )4.480.34478
Table 1: CIFAR-10 (ResNet-18) random unlearning on classification averaged over 10 trials. PruneForget achieves performance comparable to the oracle (Retrain →\to 3SP). Baselines include SFRon Huang et al. (2024) , ℓ1\ell_{1} -MU Jia et al. (2023) , Un-pruning Xiao et al. (2025) , Bilevel Shirkavand et al. (2025) , and LLM-Eraser Zhang et al. (2025) . Performance gaps ( Δ\Delta ) relative to the oracle are reported first, with the raw UA, RA, and TA values provided in parentheses. Avg Δ\text{Avg }\Delta reflects overall performance. Best results are in blue , while second best are italicized .
MethodUnlearn 1 Class (10%)Unlearn 5 Classes (50%)
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.00 (94.27 ±0.82 )0.000.0022410.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.00 (96.38 ±1.39 )0.000.001554
Pretrain →\to 3SP0.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.44 (94.72 ±0.57 )0.150.259890.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.22 (96.60 ±1.23 )0.070.36566
3SP →\to SFRon0.00 (100.00 ±0.00 )2.77 (97.22 ±1.03 )2.34 (91.94 ±1.31 )1.700.4610780.00 (100.00 ±0.00 )3.46 (96.54 ±2.84 )3.53 (92.85 ±3.20 )2.331.54619
SFRon →\to 3SP0.00 (100.00 ±0.00 )0.01 (99.99 ±0.01 )0.67 (93.60 ±0.68 )0.230.3310740.00 (100.00 ±0.00 )0.01 (99.99 ±0.01 )0.89 (95.48 ±2.00 )0.300.47615
ℓ1\ell_{1} -MU →\to 2SP0.00 (100.00 ±0.00 )4.93 (95.07 ±0.70 )4.44 (89.84 ±0.78 )3.120.318050.00 (100.00 ±0.00 )1.74 (98.26 ±0.86 )3.18 (93.20 ±1.94 )1.640.45453
Table 2: CIFAR-10 (ResNet-18) class-wise unlearning on classification averaged over 10 classes. PruneForget achieves close performance to the oracle and the lowest Avg Δ\text{Avg }\Delta in both scenarios.
MethodRandom Unlearn 10%Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 2SP0.00 (19.88 ±0.00 )0.00 (98.66 ±0.00 )0.00 (80.18 ±0.00 )0.000.0088140.00 (22.61 ±0.00 )0.00 (99.24 ±0.00 )0.00 (77.56 ±0.00 )0.000.005704
Pretrain →\to 2SP1.64 (18.24 ±0.36 )0.05 (98.71 ±0.03 )0.09 (80.27 ±0.21 )0.590.2562782.32 (20.30 ±0.13 )0.07 (99.32 ±0.04 )0.26 (77.82 ±0.18 )0.880.303771
2SP →\to SFRon7.26 (27.14 ±0.87 )0.62 (98.04 ±0.18 )4.83 (75.35 ±0.78 )4.240.4569684.61 (27.22 ±0.72 )0.28 (98.96 ±0.08 )5.64 (71.92 ±0.59 )3.510.534779
SFRon →\to 2SP1.63 (18.25 ±0.31 )0.03 (98.69 ±0.04 )0.20 (80.38 ±0.22 )0.620.3070392.51 (20.11 ±0.22 )0.09 (99.34 ±0.05 )0.44 (78.00 ±0.27 )1.010.344865
ℓ1\ell_{1} -MU →\to 2SP29.54 (49.42 ±2.31 )39.17 (59.49 ±2.60 )29.36 (50.82 ±2.10 )32.691.36741233.71 (56.33 ±0.48 )45.53 (53.71 ±0.78 )34.04 (43.51 ±0.50 )37.761.554486
Table 3: TinyImageNet (Swin-T) random unlearning on classification averaged over 10 trials. PruneForget achieves near-oracle performance across all metrics with the lowest Avg Δ\text{Avg }\Delta .
StageStageStageRandom Unlearn 50%
123Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
   1.54 (6.06 ±0.17 )0.00 (100.00 ±0.00 )0.81 (93.01 ±0.27 )0.780.23556
✓  0.93 (6.67 ±0.17 )0.00 (100.00 ±0.00 )0.29 (92.50 ±0.24 )0.410.25588
✓✓ 0.80 (6.80 ±0.16 )0.00 (100.00 ±0.00 )0.22 (92.43 ±0.20 )0.340.24590
✓✓✓0.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
Table 4: Ablation of pipeline stages. Evaluating the contribution of each optimization stage in PruneForget across 50% random unlearning on CIFAR-10. Integrating all three stages yields the optimal balance between unlearning efficacy and model utility.
Design of S\mathcal{S}Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
11 (Uniform)1.49 (6.11 ±0.33 )0.00 (100.00 ±0.00 )0.71 (92.91 ±0.21 )0.730.23575
Random2.29 (5.31 ±0.45 )0.00 (100.00 ±0.00 )0.98 (93.19 ±0.19 )1.090.23574
Unlearn Only0.15 (7.44 ±0.15 )0.00 (100.00 ±0.00 )0.05 (92.25 ±0.22 )0.070.26587
Retain Only1.05 (8.64 ±0.43 )0.00 (100.00 ±0.00 )1.09 (91.11 ±0.50 )0.710.27589
Fisher Score0.15 (7.74 ±0.58 )0.00 (100.00 ±0.00 )0.15 (92.05 ±0.36 )0.100.25620
Table 5: Sensitivity score design comparison. We evaluate the impact of various designs of S\mathcal{S} on 50% random unlearning on CIFAR-10. Ours achieves the lowest Avg Δ\text{Avg }\Delta among all methods.
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime (s)
Retrain →\to 3SP0.00 (7.60 ±0.22 )0.00 (100.00 ±0.00 )0.00 (92.20 ±0.30 )0.000.001518
Updating Sensitivity0.41 (8.00 ±0.49 )0.00 (100.00 ±0.00 )0.51 (91.70 ±0.35 )0.300.26604
Static (PruneForget)0.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
Table 6: Comparison of updating vs. static sensitivity score. We compare an updating sensitivity score versus a static one. The static approach achieves a smaller gap to the oracle across all metrics.
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (7.60 ±0.22 )0.00 (100.00 ±0.00 )0.00 (92.20 ±0.30 )0.000.001518
ζt=1\zeta_{t}=10.20 (7.80 ±0.33 )0.00 (100.00 ±0.00 )0.17 (92.03 ±0.22 )0.120.26612
Dynamic ζt\zeta_{t}0.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
Table 7: Ablation of dynamic weighted unlearning. We evaluate dynamic weighted unlearning on 50% random unlearning of CIFAR-10, demonstrating that it effectively improves Avg Δ\text{Avg }\Delta .
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime (s)
Retrain →\to 3SP0.00 (7.60 ±0.22 )0.00 (100.00 ±0.00 )0.00 (92.20 ±0.30 )0.000.001518
R=1R=1 (Ours)0.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
R=2R=20.57 (7.02 ±0.14 )0.00 (100.00 ±0.00 )0.49 (92.69 ±0.22 )0.350.25599
R=5R=50.67 (6.93 ±0.14 )0.00 (100.00 ±0.00 )0.57 (92.77 ±0.16 )0.410.25646
R=10R=100.45 (7.15 ±0.18 )0.00 (100.00 ±0.00 )0.48 (92.69 ±0.14 )0.310.25702
Table 8: Ablation on iterative pruning rounds. We evaluate the impact of the number of iterative pruning rounds ( RR ). A single round ( R=1R=1 ) achieves the closest oracle alignment.
MethodAverage (5 Classes)Time
Δ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrow
Retrain →\to 2SP0.00 (98.39 ±1.65 )0.00 (16.73 ±0.63 )398622
Pretrain →\to 2SP0.45 (97.94 ±1.70 )0.30 (16.95 ±0.47 )21304
2SP →\to SFRon0.39 (98.00 ±1.52 )0.22 (16.92 ±0.70 )25652
SFRon →\to 2SP0.55 (97.83 ±1.77 )0.23 (16.96 ±0.71 )26674
ℓ1\ell_{1} -MU →\to 2SP0.20 (98.19 ±1.76 )0.76 (17.49 ±0.73 )22675
Table 9: CIFAR-10 (DDPM) class-wise unlearning results. Averaged across 5 classes, PruneForget obtains close performance to the oracle with over 10 ×\times acceleration.
Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Related WorkUnlearn-aware Stages
SparsityPruningFine-tuning
ℓ1\ell_{1} -MU  ✓
Un-pruning  ✓
Bilevel  ✓
LLM-Eraser ✓✓
PruneForget (Ours)✓✓✓
Appendix
Table A1: Comparison with related works. Typical pruning methods consist of three stages of sparsity training, pruning, and post-pruning fine-tuning. Prior works have mainly focused on making the fine-tuning stage unlearn-aware.
ModelDatasetInputppMACsParamsSpeedup
ResNet-18CIFAR-10 / CIFAR-10032×3232\times 320556.6M11.17M1.00×1.00\times
0.3269.4M5.46M2.07×2.07\times
0.5139.9M2.80M3.98×3.98\times
0.749.8M1.00M11.17×11.17\times
Swin-TTinyImageNet224×224224\times 22404.36G27.67M1.00×1.00\times
0.23.56G20.19M1.22×1.22\times
Appendix
Table A2: Model compute reduction after pruning. We report MACs, parameter counts, and theoretical speedup across pruned models (with different pruning ratios pp ). Here, we follow Fang et al. (2023a) and report the theoretical speedup ≜MACs(base)MACs(pruned)\triangleq\frac{\text{MACs}(\text{base})}{\text{MACs}(\text{pruned})} .
fbasef_{base}Random Unlearn 10%Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
20200.03 (6.01 ±0.28 )0.00 (100.00 ±0.00 )0.01 (93.73 ±0.22 )0.010.229981.43 (9.03 ±1.45 )0.01 (99.99 ±0.01 )0.67 (91.54 ±0.75 )0.700.30609
50500.06 (6.09 ±0.42 )0.00 (100.00 ±0.00 )0.25 (93.49 ±0.26 )0.100.229960.57 (8.17 ±0.23 )0.00 (100.00 ±0.00 )0.21 (92.00 ±0.28 )0.260.27603
80800.12 (5.92 ±0.49 )0.00 (100.00 ±0.00 )0.07 (93.67 ±0.24 )0.060.229860.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
1001000.20 (5.84 ±0.40 )0.00 (100.00 ±0.00 )0.16 (93.59 ±0.19 )0.120.229730.19 (7.41 ±0.36 )0.00 (100.00 ±0.00 )0.04 (92.24 ±0.22 )0.070.25579
Appendix
Table A3: Hyperparameter search of fbasef_{base} on CIFAR-10 (ResNet-18) random unlearning. We use grid search to determine the most suitable unlearning frequencies and identify fbase=20f_{base}=20 and fbase=80f_{base}=80 as the optimal settings for 10% and 50% unlearning, respectively.
fbasef_{base}Unlearn 1 Class (10%)Unlearn 5 Classes (50%)
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
20200.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.51 (93.76 ±0.68 )0.170.289920.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.42 (95.96 ±1.23 )0.140.40602
50500.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.45 (93.82 ±0.83 )0.150.289840.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.27 (96.10 ±1.29 )0.090.39600
80800.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.41 (93.86 ±0.71 )0.140.289840.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.25 (96.13 ±1.28 )0.080.40600
1001000.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.41 (93.86 ±0.71 )0.140.289840.00 (100.00 ±0.00 )0.00 (100.00 ±0.00 )0.20 (96.17 ±1.21 )0.070.38596
Appendix
Table A4: Hyperparameter search of fbasef_{base} on CIFAR-10 (ResNet-18) class unlearning. Grid search results indicate that fbase=100f_{base}=100 achieves the best performance for both settings.
MethodRandom Unlearn 10%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrow
Task: Pure Unlearning (No Pruning)
Retrain0.00 (4.38 ±0.25 )0.00 (100.00 ±0.00 )0.00 (95.34 ±0.08 )0.000.00
FT4.28 (0.10 ±0.05 )0.01 (99.99 ±0.00 )0.39 (94.94 ±0.15 )1.560.26
GA1.71 (6.09 ±1.67 )6.24 (93.76 ±1.89 )8.34 (87.00 ±1.64 )5.430.36
SalUn4.38 (0.00 ±0.01 )0.01 (99.99 ±0.01 )0.45 (94.89 ±0.09 )1.610.27
Appendix
Table A5: Comparison with pure unlearning. We compare pure unlearning (baselines include FT Warnecke et al. (2023) , GA Thudi et al. (2022) , SalUn Fan et al. (2024) , BT Chundawat et al. (2023) , SCRUB Kurmanji et al. (2023) , and SFRon Huang et al. (2024) ) against unlearning with pruning on a 10% random subset of CIFAR-10. Note that these are inherently distinct tasks evaluated against different respective oracles .
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime (s)
Retrain →\to 3SP0.00 (7.60 ±0.22 )0.00 (100.00 ±0.00 )0.00 (92.20 ±0.30 )0.000.001518
Pretrain →\to 3SP1.54 (6.06 ±0.17 )0.00 (100.00 ±0.00 )0.81 (93.01 ±0.27 )0.780.23556
Prune →\to retrain2.05 (9.65 ±0.19 )0.01 (99.99 ±0.01 )3.05 (89.15 ±0.24 )1.700.35598
PruneForget0.16 (7.44 ±0.32 )0.00 (100.00 ±0.00 )0.02 (92.22 ±0.23 )0.060.25594
Appendix
Table A6: Comparison with pruning then retraining from scratch. We evaluate the Prune →\to retrain baseline on CIFAR-10 50% random unlearning. The results demonstrate that pruned models are inherently difficult to optimize from a random initialization.
MethodRandom Unlearn 10%Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (5.15 ±0.31 )0.00 (100.00 ±0.00 )0.00 (94.75 ±0.18 )0.000.0022120.00 (6.98 ±0.20 )0.00 (100.00 ±0.00 )0.00 (92.78 ±0.20 )0.000.001494
Pretrain →\to 3SP1.14 (4.01 ±0.10 )0.00 (100.00 ±0.00 )0.05 (94.80 ±0.14 )0.400.1410572.40 (4.58 ±0.12 )0.00 (100.00 ±0.00 )1.04 (93.82 ±0.16 )1.150.20638
3SP →\to SFRon15.17 (20.32 ±0.61 )1.78 (98.22 ±0.13 )4.10 (90.65 ±0.23 )7.020.2111715.30 (12.29 ±0.29 )0.41 (99.59 ±0.05 )2.90 (89.88 ±0.15 )2.870.25698
SFRon →\to 3SP1.10 (6.25 ±0.41 )0.00 (100.00 ±0.00 )1.35 (93.40 ±0.23 )0.820.2011670.14 (6.84 ±0.16 )0.00 (100.00 ±0.00 )0.22 (92.55 ±0.29 )0.120.25707
ℓ1\ell_{1} -MU →\to 2SP5.94 (11.09 ±0.26 )5.20 (94.80 ±0.29 )5.95 (88.80 ±0.31 )5.690.279315.46 (12.44 ±0.18 )3.25 (96.75 ±0.26 )5.41 (87.37 ±0.21 )4.710.33545
Appendix
Table A7: CIFAR-10 (ResNet-18) random unlearning on classification with p=0.3p=0.3 . Among all baselines, PruneForget achieves the lowest Avg Δ\text{Avg }\Delta under a low pruning ratio.
MethodRandom Unlearn 10%Random Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeΔ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (8.39 ±0.31 )0.00 (99.89 ±0.00 )0.00 (91.34 ±0.18 )0.000.0021310.00 (9.71 ±0.20 )0.00 (99.98 ±0.00 )0.00 (90.09 ±0.20 )0.000.001454
Pretrain →\to 3SP0.49 (7.90 ±0.38 )0.03 (99.92 ±0.02 )0.40 (91.74 ±0.23 )0.310.149910.05 (9.65 ±0.32 )0.01 (99.97 ±0.01 )0.19 (89.90 ±0.31 )0.080.20597
3SP →\to SFRon13.88 (22.26 ±0.94 )7.48 (92.41 ±1.08 )4.91 (86.43 ±0.93 )8.760.2110976.47 (16.18 ±0.58 )5.39 (94.59 ±0.97 )5.09 (85.00 ±0.46 )5.650.25618
SFRon →\to 3SP1.28 (9.66 ±0.74 )0.50 (99.39 ±0.37 )1.11 (90.23 ±0.70 )0.960.2011202.43 (12.14 ±0.42 )0.33 (99.65 ±0.09 )2.62 (87.47 ±0.46 )1.790.25668
ℓ1\ell_{1} -MU →\to 2SP3.17 (11.56 ±0.46 )5.76 (94.13 ±0.33 )3.05 (88.29 ±0.32 )3.990.278593.38 (13.09 ±0.25 )4.17 (95.81 ±0.34 )3.42 (86.68 ±0.30 )3.660.33505
Appendix
Table A8: CIFAR-10 (ResNet-18) random unlearning on classification with p=0.7p=0.7 . PruneForget achieves a small gap to the oracle under a high pruning ratio.
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTime
Retrain →\to 3SP0.00 (32.48 ±0.31 )0.00 (99.99 ±0.01 )0.00 (67.40 ±0.18 )0.000.002551
Pretrain →\to 3SP3.22 (29.27 ±0.71 )0.00 (99.99 ±0.01 )1.50 (68.90 ±0.69 )1.570.20567
3SP →\to SFRon5.39 (37.87 ±0.31 )0.44 (99.55 ±0.05 )1.67 (65.72 ±0.37 )2.500.25631
SFRon →\to 3SP2.12 (30.37 ±0.28 )0.00 (99.99 ±0.01 )1.48 (68.88 ±0.30 )1.200.25635
ℓ1\ell_{1} -MU →\to 2SP6.28 (38.76 ±0.36 )7.59 (92.40 ±0.74 )6.13 (61.27 ±0.53 )6.670.33467
Appendix
Table A9: CIFAR-100 (ResNet-18) random unlearning on classification. PruneForget achieves performance comparable to the oracle (Retrain →\to 3SP), demonstrating its generalizability across more complex datasets.
MethodAutomobileCatDogHorseTruckTime
Δ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrow
Retrain →\to 2SP0.00 (98.87)0.00 (17.10)0.00 (95.50)0.00 (15.96)0.00 (98.79)0.00 (16.26)0.00 (99.12)0.00 (17.54)0.00 (99.66)0.00 (16.78)398622
Pretrain →\to 2SP0.65 (98.22)0.10 (17.20)0.55 (94.95)0.61 (16.57)0.38 (98.41)0.07 (16.33)0.13 (98.99)0.21 (17.33)0.55 (99.11)0.52 (17.30)21304
2SP →\to SFRon0.66 (98.21)0.34 (17.44)0.15 (95.35)0.13 (16.09)0.36 (98.43)0.07 (16.19)0.19 (98.93)0.02 (17.56)0.59 (99.07)0.53 (17.31)25652
SFRon →\to 2SP0.81 (98.06)0.30 (17.40)0.78 (94.72)0.11 (16.07)0.25 (98.54)0.05 (16.31)0.16 (98.96)0.21 (17.75)0.77 (98.89)0.47 (17.25)26674
ℓ1\ell_{1} -MU →\to 2SP0.21 (98.66)1.03 (18.13)0.39 (95.11)0.55 (16.51)0.16 (98.63)0.63 (16.89)0.07 (99.05)0.49 (18.03)0.15 (99.51)1.11 (17.89)22675
Appendix
Table A10: CIFAR-10 (DDPM) class-wise unlearning on generation. PruneForget consistently achieves close performance to the oracle .
MethodCacatua galeritaGolden retrieverWhite wolfArctic foxOtterAverageTime
UA ↑\uparrowFID ↓\downarrowUA ↑\uparrowFID ↓\downarrowUA ↑\uparrowFID ↓\downarrowUA ↑\uparrowFID ↓\downarrowUA ↑\uparrowFID ↓\downarrowUA ↑\uparrowFID ↓\downarrow
Pretrain →\to 2SP4.212.512.012.652.612.749.812.603.212.824.36 ±3.1512.66 ±0.1235296.4
2SP →\to SFRon100.0115.01100.0115.68100.021.62100.0316.17100.022.25100.0 ±0.00118.15 ±120.1535930.8
SFRon →\to 2SP5.412.362.812.738.813.185.012.8710.812.806.56 ±3.2012.79 ±0.2937440.7
LLM-Eraser100.0564.48100.0359.76100.0564.48100.0357.60100.0315.15100.0 ±0.00432.29 ±121.97824133.9
PruneForget100.012.95100.013.68100.013.40100.013.48100.013.84100.0 ±0.0013.47 ±0.3435765.0
Appendix
Table A11: ImageNet-1K (DiT-XL/2) class-wise unlearning on generation. PruneForget uniquely achieves perfect forgetting while maintaining generative quality. Averages (shaded) across the 5 classes are reported in the summary section. We omit the oracle (Retrain →\to 2SP) due to computational constraints. Best results are in blue , while second best are italicized .
MethodAutomobileCatDogHorseTruckTime
Δ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrowΔ\Delta UA ↓\downarrowΔ\Delta FID ↓\downarrow
Retrain →\to 2SP0.00 (98.87)0.00 (17.10)0.00 (95.50)0.00 (15.96)0.00 (98.79)0.00 (16.26)0.00 (99.12)0.00 (17.54)0.00 (99.66)0.00 (16.78)398622
w/o Early Stop5.60 (93.27)3.21 (20.31)1.00 (96.50)6.64 (22.60)0.15 (98.64)4.82 (21.08)0.20 (99.32)8.41 (25.95)0.04 (99.62)1.19 (17.97)26396
PruneForget0.17 (98.70)0.38 (17.48)0.13 (95.37)0.13 (15.83)0.04 (98.75)0.10 (16.36)0.04 (99.08)0.04 (17.50)0.26 (99.40)0.63 (17.41)26254
Appendix
Table A12: Ablation of early stopping on generation. We evaluate the impact of the early stopping mechanism on DDPM for CIFAR-10 class-wise unlearning. PruneForget consistently maintains tighter Δ\Delta UA and Δ\Delta FID gaps relative to the oracle .
MethodUnlearn classRetain classes
 I1I2I3I4I5C1C2C3C4C5C6C7C8C9
Pretrain               
Retrain →\to 2SPAutomobile              
PruneForget               
Pretrain               
Retrain →\to 2SPCat              
Appendix
Figure A1: Qualitative results of CIFAR-10 (DDPM) class-wise unlearning. We show generated samples for the unlearn classes ( Automobile , Cat , Dog , Horse , and Truck ) alongside several retain classes. We observe that PruneForget successfully unlearns the class concept, e.g . , when the pre-trained model successfully generated a car, PruneForget and the oracle instead generate an ambiguous object. Overall, we observe that PruneForget consistently generates images that closely match the oracle (Retrain →\to 2SP). These observations are consistent with the quantitative metrics.
Figure A2: Qualitative results of ImageNet (DiT-XL/2) class-wise unlearning. We show generated samples for the 5 evaluated unlearn classes ( Cacatua galerita , Golden retriever , White wolf , Arctic fox , and Otter ) alongside several retain classes. PruneForget achieves perfect forgetting (by generating noise for the unlearn class) while maintaining generative quality in the retain classes (matching the pre-trained model). We omit the oracle due to computational constraints.
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeMACsSpeedup
Retrain →\to 3SP0.00 (7.21 ±0.25 )0.00 (100.00 ±0.00 )0.00 (92.54 ±0.29 )0.000.001471175.5M3.17 ×\times
Pretrain →\to 3SP1.79 (5.42 ±0.14 )0.00 (100.00 ±0.00 )0.99 (93.53 ±0.16 )0.930.23596186.9M2.98 ×\times
PruneForget0.05 (7.26 ±0.29 )0.00 (100.00 ±0.00 )0.12 (92.42 ±0.29 )0.060.24648137.4M4.05 ×\times
Appendix
Table A13: CIFAR-10 (ResNet-18) under global pruning. Evaluating PruneForget under global pruning on 50% random unlearning demonstrates superior alignment to the oracle ( Avg Δ\Delta ) and higher speedup compared to the unlearning-unaware baseline.
MethodRandom Unlearn 50%
Δ\Delta UA ↓\downarrowΔ\Delta RA ↓\downarrowΔ\Delta TA ↓\downarrowAvg Δ\Delta ↓\downarrowDKLD_{\text{KL}} ↓\downarrowTimeMACsSpeedup
Retrain →\to 2SP0.00 (22.95 ±0.18 )0.00 (98.68 ±0.07 )0.00 (77.29 ±0.21 )0.000.0054292962.6M1.50 ×\times
Pretrain →\to 2SP2.37 (20.58 ±0.14 )0.22 (98.90 ±0.05 )0.27 (77.56 ±0.26 )0.950.3136632960.4M1.51 ×\times
PruneForget1.16 (21.79 ±0.17 )0.35 (99.02 ±0.05 )0.12 (77.17 ±0.20 )0.540.3745062942.9M1.51 ×\times
Appendix
Table A14: TinyImageNet (Swin-T) under global pruning. Evaluating PruneForget under global pruning on 50% random unlearning shows lower Avg Δ\Delta than the baseline under the same speedup.
Figure A3: Channel sparsity distribution of pruned ResNet-18. In (a), the pruning process operates independently of the unlearning phase; conversely, in (b), the proposed PruneForget integrates unlearning awareness into the pruning stage. This distinction leads to noticeably different parameter distributions across the network. Specifically, PruneForget tends to preserve significantly more channels in the final stage.
Figure A4: Channel sparsity distribution of pruned Swin-T. In (a), the pruning process operates independently of the unlearning phase; conversely, in (b), the proposed PruneForget integrates unlearning awareness into the pruning stage. This results in the difference in pruned parameter distributions across the model. Specifically, we observe that PruneForget prunes the initial QKV layer more aggressively than in (a). While the majority of pruning for both methods occurs toward the end of the model, (b) more significantly prunes the QKV layer at the end.
More in this topic

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 354 papers

Explore similar work

CardsList

  1. Models Designed to Forget: Machine Unlearning via Key Deletion

    Mar 16, 2026Sonia Laguna, Jorge da Silva Goncalves, Moritz Vandenhirtz +3Machine Unlearning

    Department of Computer Science, ETH Zurich, Switzerland

  2. Reference-Guided Machine Unlearning

    Mar 11, 2026Jonas Mirlach, Sonia Laguna, Julia E. VogtMachine Unlearning

    Department of Computer Science, ETH Zurich

  3. Unmerge: Efficient Machine Unlearning via Task Arithmetic

    Sep 30, 2026Haoran Tang, Andrew Tan, Rajiv KhannaMachine Unlearning

    Department of Computer Science Purdue University