cs.LGNov 2, 2026

Forget What Comes Next: LLM Unlearning via Statistical Independence

Authors: Andrii Shkabrii, Claudia Plant, Martin Pawelczyk

Organizations: Faculty of Computer Science, University of Vienna · Doctoral School Computer Science, University of Vienna · Research Network Data Science, University of Vienna

Abstract

Large language models (LLMs) can memorize and reproduce fragments of their training data, raising privacy and legal concerns that motivate machine unlearning. Yet poisoned training examples can remain statistically detectable after unlearning, which raises the question of whether current unlearning objectives remove what the model has memorized at all. In this work, we introduce PINT, a novel algorithm for machine unlearning that uses the Hilbert-Schmidt Independence Criterion (HSIC) to reduce the statistical dependence between tokens to be forgotten and the internal representations used to predict them. We formally relate our HSIC-based objective to Exact Memorization through a computable upper bound, connecting the training objective directly to the model's ability to reproduce the forgotten tokens. In extensive experiments, we compare forgetting-utility trade-offs across eight settings with models up to 7B parameters and demonstrate scalability to 70B parameter models. Our evaluations cover unlearning Gaussian poisons, personal information, hazardous knowledge in biology and cybersecurity, and copyrighted books and news articles. PINT achieves the strongest forgetting-utility trade-off on average and is the only method to make Gaussian poisons undetectable while maintaining near-original model utility. Code is available at: https://github.com/shkabrii/PINT

Figures & tables

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

    Sep 9, 2026Ravi Ranjan, Olivera Kotevska, Agoritsa PolyzouMemorization in Language ModelsLanguage Model Robustness

  2. SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion

    May 8, 2026Zizhao Hu, Ameya Godbole, Johnny Tian-Zheng Wei +3Machine UnlearningLLM Unlearning