Do unlearning algorithms learn the right features?
Organizations: CISPA Helmholtz Center for Information Security · Saarland University · Department of Computer Science, University of Copenhagen
Abstract
Machine unlearning is commonly evaluated with standard post-deletion metrics that compare an unlearned model to one retrained on the remaining data. Such snapshot evaluations can conceal consequential differences in the features the two models encode. We prove that if an unlearned and a retrained model agree on their outputs but encode different features, a single gradient step on their prediction heads can break this agreement. We further show that deleting an arbitrarily small fraction of the data can cause retraining to select qualitatively different features, so matching retraining can require learning features that the full model never had. We then introduce a measure of feature recovery that separates missing from extraneous features. Across eight unlearning methods, models that perform well on standard metrics often fail to recover the retrained features. This mismatch predicts deviations from retraining under subsequent relearning and downstream adaptation, and directly correcting it systematically reduces these deviations.
Figures & tables
| Retrain | Initial model | IU | ReprUnl | LTD | SalUn | FT | NegGrad+ | AMUN | RL | |
| Feature Discrepancy | 0.000 | 0.049 | 0.050 | 0.079 | 0.099 | 0.134 | 0.139 | 0.144 | 0.151 | 0.160 |
| Mean Accuracy (%) | 90.29 | 89.71 | 89.69 | 85.61 | 85.49 | 80.40 | 79.62 | 84.44 | 78.00 | 75.46 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Method | Retain Acc. | Forget Acc. | Test Acc. |
| Retrain | |||
| FT | |||
| AMUN | |||
| NegGrad+ | |||
| IU | |||
| LTD |
| Method | Retain Acc. | Forget Acc. | Test Acc. |
| Retrain | |||
| FT | |||
| AMUN | |||
| NegGrad+ | |||
| IU | |||
| LTD |
Machine Unlearning
Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.
See all 372 papersWe Should Distinguish Unlearning From Untraining
delete'' specific data points or behaviours from a trained model, a goal referred to as machine unlearning''. We argue that the umbrella term unlearning'' actually spans two distinct problem formulations, but the distinction between them has not yet been observed in literature. This causes ambiguity around when an unlearning algorithm is expected to work, leads to the use of inappropriate metrics and baselines when comparing algorithms to one another, difficulty in interpreting results, and missed opportunities for pursuing critical research directions. In this paper, we argue for the position that a fundamental distinction must be made between two notions that we refer to as unlearning and untraining. On one hand, untraining aims to reverse the effect of having trained on a given forget set'', i.e. to remove the influence that that specific set of examples had on the model during training. On the other hand, unlearning aims not just to remove the influence of those given examples, but also of the entire underlying subdistribution from which they were sampled (and thus e.g. the concept or model behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each notion. By disambiguating technical definitions, our work aims to accelerate progress in this important field.