IDSISA: Influence-Distilled Sharded Training for Efficient Machine Unlearning
Organizations: The authors are with the Sardar Vallabhbhai National Institute of Technology Surat, Surat, Gujarat, India.
Abstract
Machine unlearning removes the influence of specific training examples to comply with privacy regulations. Full retraining gives strong guarantees but is computationally prohibitive at scale; approximate methods are efficient but lack formal guarantees. SISA (Sharded, Isolated, Sliced, and Aggregated training) achieves certified deletion through data sharding and independent retraining, but random partitioning scatters forget requests across most shards, and isolated retraining degrades utility. We introduce IDSISA (Influence-Distilled SISA), which uses influence functions to concentrate forget-prone samples into few shards and distillation from unaffected shards to recover utility lost to isolation. The deletion guarantee inherited from SISA's retraining step remains exact and requires no probabilistic qualification; the distillation-based utility recovery is a separately quantified, empirically bounded approximation ( in expectation). We further formalize certification under sequences of repeated deletion requests, where cross-shard distillation can create dependencies between shards; we prove that a closure-based cascading retraining procedure restores the deletion guarantee to the full aggregated model under any deletion sequence. Experiments on ResNet-18/CIFAR-10, ResNet-18/CIFAR-100, BiLSTM/IMDB, TCN/IMDB, and a fully connected network on Fashion-MNIST show better selective forgetting, up to better utility retention in relative terms (with consistent percentage-point gains on the other architectures, where a relative baseline is undefined), and unlearning-time changes ranging from to depending on architecture, relative to SISA.
Figures & tables
| Method Class | Certified? | Selective Forgetting | Knowledge Pres. | Efficiency |
| Full Retraining | Yes | Strong | Strong | Low |
| Approx. Scrubbing | No | Moderate | Poor | High |
| Descent-to-Delete | No | Moderate | Poor | High |
| SISA (Random Shards) | Yes | Weak | Poor | High |
| Federated Unlearning | Yes | Moderate | Moderate | High |
| IDSISA (Proposed) | Yes † | Strong | Strong | High |
| Symbol | Definition |
| Full training dataset | |
| Forget set: samples to be unlearned | |
| Retain set: | |
| Data assigned to shard | |
| Model parameters for shard | |
| Checkpoint after slice of shard |
| Operation | Theoretical | F-MNIST (ANN) | CIFAR-10 (CNN) | IMDB (LSTM) |
| Influence Calculation | 12s | 38s | 28s | |
| PCA Reduction | 1s | 3s | 2s | |
| K-means Clustering | 8s | 15s | 11s | |
| Initial Training (SISA) | 8.9 min | 59.9 min | 120.16s | |
| Initial Training (IDSISA) | 8.9 min | 66.1 min | 138.32s | |
| Unlearning (IDSISA) | 9.5 min | 81.2 min | 93s |
| Architecture | Dataset | Params | Forgetting Improv. | Preservation Improv. ‡ | Unlearn Speedup |
| LSTM (BiLSTM) | IMDB Sentiment | 4.2M | +91% | +3.75 pts | +2.1% |
| (3.94 vs 2.06%) | (+0.65 vs -3.10%) | (93 vs 95s) | |||
| CNN (ResNet-18) | CIFAR-10 Image | 11.2M | +33.9% | +50% | +6.4% |
| (12.3 vs 18.6%) | (0.8 vs 1.6%) | (81.2 vs 86.7m) | |||
| ANN (Fully-Conn.) | Fashion-MNIST | 203K | +14.3% | +0.03 pts | -6.7% |
| (-9.57 vs -8.37%) | (+0.03 vs 0.00%) | (534 vs 570s) |
| Method | Certified? | Forget Acc (%) | Retain Acc (%) | Test Acc (%) | MIA on |
| Full Retrain (Oracle) | Yes | 10.0 | 93.4 | 93.4 | Calibrated |
| SalUn [ 21 ] | No † | 0–6 | 88–92 | 88–92 | Low |
| SCRUB [ 17 ] | No † | 5–8 | 83–85 | 83–85 | Moderate |
| SSD [ 18 ] | No † | 2–8 | 82–86 | 82–86 | Calibrated |
| SISA [ 1 ] | Yes | 18.6 | 83.2 | 82.3 | 0.134 |
| IDSISA | Yes ‡ | 12.3 | 84.7 | 83.1 | 0.132 |
| Metric | SISA | IDSISA |
| Initial Training | ||
| Test Acc. (Pre-Unlearn) (%) | 83.8 | 84.6 |
| Training Time (min) | 59.9 | 66.1 |
| Forgetting Quality | ||
| Forget Class Acc (%) | 18.6 | 12.3 |
| Effectiveness (% removed) | 80.1 | 86.4 |
| Metric | SISA | IDSISA |
| Initial Training | ||
| Test Acc. (Pre-Unlearn) (%) | 79.55 | 79.07 |
| Training Time (s) | 120.16 | 138.32 |
| Forgetting Quality | ||
| Target Class Accuracy (%) | 77.94 | 74.74 |
| Forgetting Gap (%) | 2.06 | 3.94 |
| Metric | Random SISA | IDSISA |
| Test Acc. (Pre-Unlearn) (%) | 85.86 | 84.02 |
| Training Time (s) | 534.0 | 546.0 |
| Forgetting Gap (%) | -8.37 | -9.57 |
| Post-Unlearn Accuracy (%) | 85.86 | 84.02 |
| Unlearning Time (s) | 534.0 | 570.0 |
| Metric | SISA | IDSISA |
| Pre-Unlearn Positive Acc (%) | 69.08 | 70.15 |
| Post-Unlearn Positive Acc (%) | 30.60 | 28.19 |
| Forgetting Gap (%) | 38.48 | 41.96 |
| Retained Negative Acc (%) | 71.99 | 73.08 |
| Affected Shards | 10/10 | 8/10 |
| Unlearning Time (s) | 68.23 | 67.83 |
| Metric | SISA | IDSISA |
| Pre-Unlearn Test Acc (%) | 20.68 | 21.13 |
| Forget Superclass Acc (%) | 0.0 | 0.0 |
| Retain Acc (%) | 21.15 | 21.89 |
| Post-Unlearn Test Acc (%) | 20.09 | 20.95 |
| Affected Shards | 10/10 | 7/10 |
| Unlearning Time (s) | 920.03 | 915.04 |
| Metric | SISA-RNN | IDSISA-RNN | BiLSTM |
| Pre-Unlearn Acc (%) | 52.3 | 52.3 | 79.6 |
| Post-Unlearn Acc (%) | 53.4 | 52.3 | 77.6 |
| Retained Class Acc (%) | 66.0 | 51.8 | 80.4 |
| Forgetting Gap (%) | 25.3 † | 0.1 | 3.94 |
| Influence Time (s) | N/A | 342 | 1.34 |
| Clip Events (train) | 5,355 | High | Low |
| Metric | SISA | IDSISA |
| Affected Shards | 10/10 | 8/10 |
| Forget Accuracy (%) | 15.9 | 13.1 |
| Post-Unlearn Acc (%) | 83.0 | 83.6 |
| Preservation Loss (%) | 1.4 | 0.9 |
| Unlearning Time (s) | 5,210 | 3,720 |
| Configuration | Forget (%) | Retain (%) | Time (min) |
| SISA (Baseline) | 18.6 | 83.2 | 86.7 |
| + Influence Only | 16.4 | 83.8 | 84.3 |
| + Influence + Distill. | 12.3 | 84.7 | 81.2 |
| 14.1 | 84.2 | 82.6 | |
| (default) | 12.3 | 84.7 | 81.2 |
| 13.8 | 84.1 | 81.8 |
| Architecture | (mean) | Max Gap (99%ile) | Theoretical Bound (mean) |
| CNN (CIFAR-10) | 0.011 | 0.024 | 0.015 |
| LSTM (IMDB) | 0.008 | 0.019 | 0.015 |
| ANN (Fashion-MNIST) | 0.013 | 0.027 | 0.015 |
| Average | 0.011 | 0.023 | 0.015 |
| Model | Mean Conf. | TPR | TPR | Threshold |
| SISA (Unlearned) | 0.134 | 0.42 | 0.60 | 0.139 |
| IDSISA (Unlearned) | 0.132 | 0.39 | 0.59 | 0.139 |
| Full Retrain ( only) | 0.681 | 0.83 | N/A | 0.738 |
| Method | |||
| Full Retrain | 8–10 h | 40–50 h | 80–100 h |
| SISA | 146.6 min | 493.4 min | 926.9 min |
| IDSISA | 147.3 min | 472.1 min | 878.1 min |
| IDSISA + Reset ( ) | 147.3 min | 532.0 min | 997.9 min |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Req. | Cascade? | Scratch | Cum. | ||
| fallback? | edges | ||||
| 1 | 6 | 6 | No | No | 0 |
| 2 | 5 | 7 | Yes | No | 2 |
| 3 | 7 | 10 | Yes | Yes | 0 |
| 4 | 8 | 8 | No | No | 0 |
| 5 | 5 | 9 | Yes | No | 4 |
| Method | Certified Deletion | Selective Forgetting | Utility Pres. | Multi- Arch | NLP Support | MI Safe | Amor- tized | Influence Guided | Reference |
| Full Retraining | Strong | Strong | Oracle | ||||||
| Gradient Ascent [ 25 ] | Moderate | Poor | 2022 | ||||||
| Fine-Tuning [ 6 ] | Weak | Good | 2020 | ||||||
| Fisher Forgetting [ 6 ] | (DP) | Moderate | Poor | 2020 | |||||
| NegGrad+ [ 17 ] | Moderate | Moderate | 2023 | ||||||
| SCRUB [ 17 ] | Strong | Moderate | 2023 |
| Method | Type | Certified? | Forget Acc (%) | Retain Acc (%) | Test Acc (%) | MIA Conf. on |
| Full Retraining | Exact | Yes | 10.0 | 93.4 | 93.4 | 0.68 |
| Gradient Ascent [ 25 ] | Weight mod. | No † | 0–6 ∗ | 74–80 ∗ | 63–75 ∗ | Low |
| Fine-Tuning [ 6 ] | Weight mod. | No † | 60–80 ∗ | 83–85 ∗ | 82–84 ∗ | High |
| NegGrad+ [ 17 ] | Gradient | No † | 5–10 ∗ | 82–84 ∗ | 82–84 ∗ | Moderate |
| Fisher Forgetting [ 6 ] | DP noise | Approx. DP | 3–8 ∗ | 78–82 ∗ | 78–82 ∗ | Low |
| SSD [ 18 ] | FIM dampening | No † | 2–8 ∗ | 82–86 ∗ | 82–86 ∗ | Calibrated |
| Model | Conf. | TPR | TPR | Threshold | Val Acc |
| SISA (Unlearned) | 0.134 | 0.42 | 0.60 | 0.139 | 0.62 |
| IDSISA (Unlearned) | 0.132 | 0.39 | 0.59 | 0.139 | 0.62 |
| Full Retrain ( ) | 0.681 | 0.83 | N/A | 0.738 | N/A |
Machine Unlearning
Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.
See all 372 papers