Machine Unlearning Evaluation

Follow this topic Ask about this topic

Machine unlearning evaluation asks whether a model has genuinely lost the influence of specified training examples, facts, or users—not merely stopped producing them on familiar prompts. Common approaches compare behavior and utility with retrained models, measure membership or extraction risk, probe hidden representations, and test recovery through adversarial queries, fine-tuning, or sequential updates. Benchmarks span classifiers, language and vision-language models, generative systems, and agents, with checks for collateral damage to related knowledge and retained capabilities. A central tension remains: behavioral tests are practical but can miss recoverable information, while stronger audits may require costly reference models or risk exposing private data themselves.

Latest papers

105 papers20 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning
Oct 28, 2026Xingjian Zhao, Mohammad Mohammadi Amiri, Malik Magdon-IsmailMachine Unlearning Machine Unlearning Evaluation
Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
DecisionUnlearn-Bench: Unlearning Probabilistic Decisions under Changing Schemas
Oct 28, 2026Tiantong Wu, Wei Yang Bryan LimMachine Unlearning Machine Unlearning Evaluation
Nanyang Technological University, Singapore.
Risk-Aware Output Control Should be the Focus of Machine Unlearning
Oct 26, 2026Yahya Alkhatib, Yuyang Xiao, Wee Peng TayMachine Unlearning Machine Unlearning Evaluation
School of Electrical and Electronic Engineering Nanyang Technological University, Singapore
Approximate Unlearning in Prototype-based-classifiers
Oct 25, 2026Sreejita Ghosh, Mohammad MohammadiMachine Unlearning Machine Unlearning Evaluation
UMC Groningen, Groningen, The Netherlands · Technical University of Eindhoven, Eindhoven, The Netherlands · Tilburg University, Tilburg, The Netherlands
Does Unlearning Survive Collaboration? Robust Unlearning for Shared-Model LLM Teams
Oct 19, 2026Hengsong Liu, Liangyu Teng, Jingnan Cai +5LLM Unlearning Machine Unlearning Evaluation
Fudan Institute on Networking Systems of AI
Lethe: Link Inference Attacks For Evaluation of Edge Unlearning Methods
Oct 15, 2026Andrea D'Angelo, Francesco Gullo, Davide Mottin +2Graph Unlearning Machine Unlearning
University of Aarhus Aarhus, Denmark · University of L’Aquila L’Aquila, Italy · Indian Institute of Technology Delhi Delhi, India +1
Behavioral Forgetting Is Not Privacy in Machine Unlearning
Oct 15, 2026Tyler Lizzo, Larry HeckMachine Unlearning LLM Unlearning
AI Virtual Assistant (AVA) Lab, Georgia Institute of Technology
Not All Change Is Damage: Update-Matched Unlearning for Large Language Models
Oct 15, 2026Jie Li, Youming Tao, Bowei He +1Machine Unlearning LLM Unlearning
University of Massachusetts Boston · Technische Universität Berlin · Mohamed bin Zayed University of Artificial Intelligence
Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs
Oct 14, 2026Oscar Chew, Po-Yi Lu, Yi-Jie Cheng +4MLLM Unlearning Machine Unlearning
Texas A&M University · National Taiwan University · Columbia University
RA $^2$ : Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning
Oct 14, 2026Zinan Ling, Yang Xiao, Ruimeng Ye +2Machine Unlearning LLM Unlearning
The University of Tulsa · Michigan State University
Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents
Oct 13, 2026Bofan Xie, Chaokun Wang, Guanchen Ge +4LLM Agent Memory Memory Agent Benchmarks
Tsinghua University · Sina Weibo
Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents
Oct 13, 2026Junkai Lin, Ge Xu, Zheng Xu +2Machine Unlearning LLM Unlearning
Institute of Automation, Chinese Academy of Sciences · The Chinese University of Hong Kong · Nanyang Technological University +2
Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning
Oct 10, 2026Haodong Chen, Shengtao Wen, Dong Liang +2MLLM Unlearning Refusal Behavior in Language Models
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · School of Intelligence Science and Technology, Nanjing University
Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning
Oct 8, 2026Xunlei Chen, Qinghui Gong, Jingkun Xue +3Machine Unlearning LLM Unlearning
University of Electronic Science and Technology of China · Southwest Jiaotong University
Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
Oct 7, 2026Junkai Chen, Yuhao He, Qianshan Wei +19MLLM Unlearning Machine Unlearning Evaluation
Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · ByteDance +9
Exact Unlearning via Quantized Sufficient Statistics
Oct 5, 2026Ami Tavory, Shripad Gade, Tal Sarig +2Machine Unlearning Machine Unlearning Evaluation
Meta Platforms, Inc.
Your Unlearning Gives You Away: Identifying Erased Concepts in Diffusion Models
Oct 4, 2026Kaiyuan Deng, Yuchen Li, Yang Xiao +3Diffusion Model Unlearning Concept Erasure
The University of Arizona · Peking University · The University of Tulsa +1
From Mathematical to Executable Certificates for Machine Unlearning
Oct 1, 2026Ziyu Zhao, Xinyu Wang, Xiaowen Chang +1Machine Unlearning Machine Unlearning Evaluation
Arizona State University · McGill University
Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning
Sep 30, 2026Tyler Skow, Shravan Chaudhari, Rama Chellappa +1Machine Unlearning LLM Unlearning
Johns Hopkins University
Do LLMs Really Forget? Hidden-State Leakage in Model Unlearning and How to Fix it
Sep 29, 2026Hadi Reisizadeh, Jiajun Ruan, Sijia Liu +1Machine Unlearning LLM Unlearning
University of Minnesota · Michigan State University · IBM Research
Bounding Retraining Equivalence and the Deletion Floor in Materials Machine Unlearning
Sep 28, 2026Can Polat, Mustafa Kurban, Erchin Serpedin +1Machine Unlearning Machine Unlearning Evaluation
Department of Electrical and Computer Engineering, Texas A&M University, College Station, Texas, USA · Department of Electrical and Computer Engineering, Texas A&M University at Qatar, Doha, Qatar · Department of Prosthetics and Orthotics, Ankara University, Ankara, Turkey +1
eval-unlearn: Benchmarking unlearning in Text-to-Image Diffusion Models
Sep 28, 2026Mansi, Nikhil Raghavan, Zixia Huang +4Diffusion Model Unlearning T2I Generation Evaluation
Imperial College London
Evaluating Machine Unlearning in ASR
Sep 28, 2026Diogo Dinis, Francisco Teixeira, Bhiksha Raj +2Automatic Speech Recognition Machine Unlearning
INESC-ID · Instituto Superior T´ecnico, Universidade de Lisboa · LTI, Carnegie Mellon University
LLM Unlearning Evaluation with TRIAGE
Sep 26, 2026Danial Ataee, Peter TriantafillouMachine Unlearning Evaluation Machine Unlearning
Department of Computer Science University of Warwick
Mitigating Sequential Reappearance in Diffusion Data-Point Unlearning
Sep 21, 2026Donghyun Kim, Taehyuk Lee, Jinyeong Kim +4Memorization in Generative Models Diffusion Model Unlearning
Department of Computer Science and Engineering, Konkuk University Trustworthy Machine Learning Lab Seoul, Republic of Korea
$μ^2$ -Bench: A Multilingual Machine Unlearning Benchmark
Sep 17, 2026Kyomin Hwang, Hyeonjin Kim, Hyunho Lee +3Machine Unlearning LLM Unlearning
GSCST, Seoul National University · AIIS, Seoul National University
K-Bench: A Benchmark for LLM Unlearning in Agentic Deployments
Sep 11, 2026Guangsheng Yu, Yanna Jiang, Qin Wang +2LLM Agent Evaluation Privacy Leakage in Language Models
University of Technology Sydney, Australia · CSIRO, Australia
Do Influence-Derived Data Perturbations Enable Machine Unlearning? A Controlled Study of Three Plausible Roles
Sep 11, 2026Chenkai Wu, Chrispine Kambimbi, Qinyang Zeng +1Machine Unlearning Machine Unlearning Evaluation
Monash University · LibrAI · Tongji University +1
The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation
Sep 8, 2026Aaryaman Kalani, Murari Mandal, Dhruv Kumar +2Machine Unlearning Machine Unlearning Evaluation
BITS Pilani · KIIT Bhubaneswar · National University of Singapore
Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate
Sep 6, 2026Vishwajith RameshLLM Unlearning Machine Unlearning Evaluation
Vy Labs, Inc.

Oct 28, 2026J/K move · Enter open · S save

Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning

Xingjian Zhao, Mohammad Mohammadi Amiri, Malik Magdon-Ismail

Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180, USA

Machine Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Gradient-based machine unlearning methods typically ``unlearn'' by optimizing forget and retain losses, but similar losses do not necessarily imply similar output-space behavior. Prior work has shown that high forget loss alone does not certify that unlearning has produced retraining-like output behavior. We therefore ask: when a gradient-based unlearner differs from the model that would have been learned on only the retain data, what structure does the resulting output-space gap have? We derive closed-form expressions for this gap in linear regression, with an extension to nonlinear models via a generalized Gauss-Newton and local linearity approximations. Our analysis puts various gradient-based methods under a unified framework parametrized by geometry, reweighting, and projection. Two leverage matrices emerge that capture the forget-retain entanglement under the retrained model geometry: forget self-leverage and retain cross-leverage. We show both theoretically and empirically that these two leverage matrices drive the impact of the forget residual on the forget and retain outputs, and existing methods account for this geometry only partially. Our framework yields precomputable leverage diagnostics for auditing forget sets to identify what type of gradient-based unlearning is likely to be successful.