cs.AINov 4, 2026

Auditable Unlearning for LLMs

Authors: Piyush Tiwary, Anushka Srivastava, Prathosh A. P

Organizations: Department of Electrical Communication Engineering, Indian Institute of Science, Bengaluru, KA 560012, India · Independent Researcher · Department of Electrical Communication Engineering, Bengaluru, KA 560012, India

Abstract

Machine unlearning aims to remove a user's data from a trained model after a deletion request. Existing methods are majorly evaluated on three axes - how well they forget, how much capability they retain, and how robust they are to membership inference attacks, but not on whether the provider can offer the user any evidence that the deletion occurred. To fill this gap, we introduce auditability as a fourth axis, defined as the separation between requested (forget set) and retained (retain set) data under a statistic the provider computes. We show that across eleven unlearning methods on TOFU, MUSE and RWKU no method satisfies all four axes, with different methods falling short on different axes. To address this, we propose a two stage method that can be used on top of any unlearning method, in the the first stage we propose GapWiden{GapWiden} which uses a hinge loss to increase the difference between audit statistics of forget set and retain set. In the second stage we propose GapVeil{GapVeil}, an inference time rejection-sampler which uses the base model as the drafter and re-samples based on the audit statistics of the drafted response. We observe that using our method, auditability and leakage reach oracle parity for every unlearning method, and forget performance converges to the oracle; retain performance improves for nine of eleven methods but remains short of parity. Our results show that evidence of deletion or auditability can be constructed by design, and that treating it as an explicit requirement changes the viability of existing unlearning methods.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Auditing of Unlearning Algorithms

    Jul 7, 2026Sahasrajit Sarmasarkar, Anastasia Koloskova, Sanmi KoyejoMachine UnlearningMachine Unlearning Evaluation

  2. Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning

    Aug 11, 2026Zirui Song, Huaxing Liu, Xiang Wang +8LLM AuditingMachine Unlearning