Machine Unlearning Evaluation

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 58

All topics

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget
Jun 15, 2026Dayong Ye, Tianqing Zhu, Ruiding Huang +5Machine Unlearning LLM Unlearning
Behavioral Audit of Machine Unlearning Has a Privacy Cost
Jun 12, 2026Liou Tang, James Joshi, Ashish KunduPrivacy Auditing Machine Unlearning
University of Pittsburgh, Pittsburgh, PA, USA · Cisco Research, San Jose, CA, USA
MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
Jun 11, 2026He Li, Haoang Chi, Qizhou Wang +6VLM Unlearning Multimodal Large Language Models
College of Computer Science and Technology, National University of Defense Technology, Changsha, China · TMLR Group, Hong Kong Baptist University, Hong Kong, China · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China +2
REMEDI: A Benchmark for Retention and Unlearning Evaluation in Multi-label Clinical Disease Inference
Jun 5, 2026Anurag Sharma, Sai Teja Chunchu, Prasenjit Mitra +2Multi-Label Classification Machine Unlearning
IIT Kharagpur · India · Carnegie Mellon University +4
SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation
May 29, 2026Petros Andreou, Jamie Lanyon, Axel Finke +1Machine Unlearning Machine Unlearning Evaluation
Department of Computer Science, School of Science, Loughborough University, Loughborough, UK · School of Mathematics, Statistics and Physics, Newcastle University, Newcastle upon Tyne, NE1 7RU, UK
Forgetting Has Neighbors: Localized Collateral Forgetting in Machine Unlearning
May 29, 2026Polina Dolgova, Sebastian U. StichSelf-Supervised Knowledge Distillation Machine Unlearning
CISPA Helmholtz Center for Information Security · Universität des Saarlandes · Saarbrücken, Germany
AMNESIA: A Large Scale Medical Unlearning Benchmark Suite with Disease-Informed Analysis
May 28, 2026Saeedeh Davoudi, Reihaneh Iranmanesh, Ophir Frieder +1Healthcare Privacy Leakage in Language Models
1IR Lab, Computer Science Department, Georgetown University, Washington D.C.
MAAT: Multi-phase Adapter-Aware Targeted Unlearning
May 28, 2026Suryash Yagnik, Shubham Gaur, Saksham Thakur +3Machine Unlearning LLM Unlearning
1Indian Institute of Information Technology, Bhopal, India · University of California, Santa Cruz, USA · 3Independent Researcher +2
RULER: Representation-Level Verification of Machine Unlearning
May 26, 2026Georgina Cosma, Axel FinkeMachine Unlearning Machine Unlearning Evaluation
Department of Computer Science, Loughborough University, UK · School of Mathematics, Statistics and Physics, Newcastle University, UK
On the Robustness of Machine Unlearning for Vision-Language Models
May 26, 2026Yujie Lin, Kaidi Jia, Jiayao Ma +2VLM Evaluation VLM Unlearning
1Xiamen University
Measuring the Depth of LLM Unlearning via Activation Patching
May 23, 2026Jaeung Lee, Dohyun Kim, Jaemin JoLLM Auditing LLM Unlearning
Sungkyunkwan University Republic of Korea
Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models
May 20, 2026Divyaksh Shukla, Ashutosh ModiConfidence Estimation in Language Models Language Model Calibration
Indian Institute of Technology Kanpur (IIT Kanpur)
Form and Function: Machine Unlearning as a Problem of Misaligned States
May 17, 2026Kennon StewartQuasi-Newton Methods Machine Unlearning
Second Street Labs, Detroit, MI, USA · Department of Statistics, University of Michigan, Ann Arbor, MI, USA
Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries
May 17, 2026Yanhang Li, Zhichao Fan, Zexin ZhuangMemorization in Language Models LLM Auditing
Northeastern University, USA · University of Illinois Urbana-Champaign, USA · Southern Methodist University, USA
Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution
May 14, 2026Saisab Sadhu, Pratinav Seth, Vinay Kumar SankarapuMachine Unlearning Post-Training Quantization
Lexsi Labs
Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation
May 14, 2026Kyomin Hwang, Hyeonjin Kim, Sangyeon Cho +1Multilingual Language Model Evaluation Privacy Leakage in Language Models
GSCST, Seoul National University · Department of Artificial Intelligence, Chung-Ang University · Korean Surgical Researcher Foundation, Republic of Korea +1
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5VLM Evaluation VLM Unlearning
Harbin Institute of Technology, Shenzhen, China · Sichuan University, Chengdu, China · Harbin Institute of Technology, Weihai, China +3
Unlearners Can Lie: Evaluating and Improving Honesty in LLM Unlearning
May 9, 2026Renjie Gu, Jiazhen Du, Yihua Zhang +1LLM Unlearning LLM Honesty
Fudan University · Central South University · Michigan State University
Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation
May 9, 2026Weidong Zheng, Kongyang Chen, Yuanwei Guo +1Classification Machine Unlearning
School of Computer Science and Cyber Engineering, Guangzhou University, Guangzhou 510335, China · School of Artificial Intelligence, Guangzhou University, Guangzhou 510006, China · Pazhou Lab, Guangzhou 510006, China +1
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Privacy-Preserving Language Models Multimodal Large Language Models
School of Computer Science and Technology Xidian University
Not Every Subject Should Stay: Machine Unlearning for Noisy Engagement Recognition
May 6, 2026Alexander VedernikovMachine Unlearning Machine Unlearning Evaluation
Helsinki, Finland
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score
May 4, 2026Abdullah Ahmad Khan, Hamid Laga, Ferdous SohelVLM Evaluation VLM Unlearning
Murdoch University
Towards Reliable Testing of Machine Unlearning
Apr 16, 2026Anna Mazhar, Sainyam GalhotraData Leakage Automated Software Testing
Cornell University Ithaca, NY, USA
Suppression Is Not Forgetting: Residual Recoverability in Visual Concept Unlearning for VLMs
Apr 3, 2026Zhangyun Tan, Zeliang Zhang, Jiani Liu +4VLM Evaluation VLM Unlearning
University of Rochester
RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories
Dec 3, 2025Roy Rinberg, Usha Bhalla, Igor Shilov +2LLM Evaluation LLM Unlearning
Harvard University · MATS · Imperial College London +1
CAFÉ: Causal Black-Box Testing of Machine Unlearning
Sep 20, 2025Anna Mazhar, Sainyam GalhotraMachine Unlearning Evaluation Black-Box Model Auditing
Cornell University, USA
The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
Date pendingAaryaman Kalani, Murari Mandal, Dhruv Kumar +2Machine Unlearning Machine Unlearning Evaluation
BITS Pilani · KIIT Bhubaneswar · National University of Singapore

Jun 15, 2026, cs.LGJ/K move · Enter open · S save

Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget

Dayong Ye, Tianqing Zhu, Ruiding Huang, Xinbo Fu+4

Machine Unlearning LLM Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Machine unlearning has been extensively studied in response to growing privacy concerns and regulatory requirements. However, auditing whether unlearning algorithms have truly erased the influence of specific data remains an open challenge. The lack of reliable and practical auditing mechanisms can lead to critical privacy risks, such as residual information leakage. This paper initiates a systematic investigation into whether existing unlearning algorithms can truly forget the designated data. We propose the first practical and general-purpose auditing framework for machine unlearning, inspired by the concept of proof of ignorance. Our framework addresses the key practicality limitations of existing methods by eliminating the need for retraining-from-scratch baselines, avoiding the training of large numbers of shadow models, and requiring no intrusive intervention in the original training process. To evaluate the effectiveness of our framework, we first conduct validation experiments to verify its soundness and completeness. We then perform comprehensive experiments across six datasets and ten representative unlearning methods. The results demonstrate that our framework reliably distinguishes between successful and failed unlearning. In particular, we observe that retraining-based and fine-tuning-based methods can achieve effective unlearning, even when the target data remain in the original dataset. In contrast, de-optimization-based methods fail to achieve true unlearning and instead degrade the model's performance. Fisher/Hessian-based methods also fail to unlearn requested data, even formal certification is provided. Moreover, we show that our framework is robust against fake unlearning attempts and generalizes well to large language models.

Momentum

Related topics

Latest papers 58