MLLM Unlearning

MLLM: Multimodal Large Language Model

MLLM unlearning aims to remove specific information or behaviors—such as private identities, copyrighted content, or unsafe associations—from models that combine language with images, audio, or other inputs. Methods include fine-tuning and adapter edits, activation steering, representation projection, and distillation, often designed to target a concept while protecting related knowledge. Benchmarks test forgetting alongside retained utility, using varied prompts, cross-modal queries, held-out examples, and adversarial recovery attempts. A central challenge is distinguishing genuine removal from refusal or suppression, while avoiding collateral damage and cumulative degradation during repeated unlearning.

Latest papers

37 papers5 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs
Oct 14, 2026Oscar Chew, Po-Yi Lu, Yi-Jie Cheng +4MLLM Unlearning Machine Unlearning
Texas A&M University · National Taiwan University · Columbia University
CUSE: Fine-Grained Multimodal Unlearning with Counterfactual Guidance and On-Policy Self-Distillation
Oct 10, 2026Chenlu Ding, Jiancan Wu, Yanchen Luo +4MLLM Unlearning Machine Unlearning
University of Science and Technology of China · Hong Kong Polytechnic University
Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning
Oct 10, 2026Haodong Chen, Shengtao Wen, Dong Liang +2MLLM Unlearning Refusal Behavior in Language Models
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · School of Intelligence Science and Technology, Nanjing University
Open-MMUnlearning: Unifying Methods and Evaluation for MLLM Unlearning
Oct 7, 2026Junkai Chen, Yuhao He, Qianshan Wei +19MLLM Unlearning Machine Unlearning Evaluation
Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · ByteDance +9
SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning
Oct 1, 2026Si Qi Goh, Cap Dang Xuan Kiet, Tat-Jen Cham +1VLM Unlearning Privacy-Preserving Language Models
Digital Trust Centre, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models
Aug 31, 2026Kangwook Ko, Jaehyuk Jang, Wonjun Lee +2MLLM Unlearning Privacy Protection in VLMs
KAIST
AIM: Anchor Identity Features, Then Match for Multimodal Large Language Model Unlearning
Aug 28, 2026Wonjun Lee, Jaehyuk Jang, Kangwook Ko +2MLLM Unlearning Privacy Protection in VLMs
Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea
Machine Unlearning for Speech Question Answering in Large Audio-Language Models
Aug 13, 2026Zhe LiuMachine Unlearning MLLM Unlearning
Meta Platforms, Inc. Menlo Park, USA
LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection
Aug 12, 2026Xinhao Zhong, Yuxia Qiao, Junhao Li +3VLM Unlearning Multimodal Large Language Models
3Pengcheng Laboratory · 1Harbin Institute of Technology, Shenzhen · 2Tsinghua University
PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Models
Aug 11, 2026Huafeng Chen, Yueming Lyu, Ziyuan Chen +4MLLM Unlearning Machine Unlearning
School of Intelligence Science and Technology, Nanjing University · Department of Computing, Imperial College London
A Model Merging Approach for Continual MLLM Unlearning
Aug 5, 2026Yuhang Wang, Linlin Zhang, Haoxuan Ji +3Multimodal Large Language Models MLLM Unlearning
1Xidian University, Xi’an, China · 2Xi’an Jiaotong University, Xi’an, China
Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation
Aug 4, 2026Chunlin Liu, Junnian Chen, Haitong Jiang +7VLM Evaluation VLM Unlearning
1Shenzhen University · 2Hebei University of Technology · 3Ludwig Maximilian University of Munich +1
Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
Aug 3, 2026Junxiang You, Junkai Chen, Yuhao He +3Multimodal Large Language Models MLLM Unlearning
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · 3Cloudspace Technology
Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
Aug 2, 2026Junkai Lin, Junkai Chen, Siqi Hou +6VLM Unlearning Privacy Leakage in Language Models
Institute of Automation, Chinese Academy of Sciences · The Chinese University of Hong Kong · The Australian National University +3
Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
Jul 23, 2026Lorenzo Orsingher, Thomas De Min, Massimiliano Mancini +2VLM Unlearning Algorithmic Fairness
University of Trento, Trento, Italy · Fondazione Bruno Kessler, Trento, Italy
Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning
Jul 21, 2026Zijie Liu, Jinhao Duan, Gaowen Liu +2VLM Unlearning Vision-Language Models
UNC at Chapel Hill · Cisco Research · Michigan State University
One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models
Jul 17, 2026Sudharshan Balaji, Yili Ren, Guangjing Wang +2VLM Unlearning VLM Robustness
University of South Florida · USA · University of Massachusetts Lowell
Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
Jul 8, 2026Nobin Sarwar, Shubhashis Roy Dipta, Zheyuan Liu +1VLM Unlearning MLLM Unlearning
University of Maryland, Baltimore County · University of Notre Dame · UNC Chapel Hill
POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking
Jul 7, 2026Zhangheng LI, Jianing Zhu, Junyuan Hong +4Multimodal Large Language Models Privacy Leakage in Language Models
University of Texas at Austin · DEVCOM Army Research Laboratory
PPE-Bench: A Benchmark for Evaluating MLLM Unlearning under Private-Public Entanglement
Jul 3, 2026Xianren Zhang, Delvin Ce Zhang, Dongwon Lee +1Privacy Leakage in Language Models MLLM Unlearning
The Pennsylvania State University · University of Sheffield
MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs
Jun 11, 2026He Li, Haoang Chi, Qizhou Wang +6VLM Unlearning Multimodal Large Language Models
College of Computer Science and Technology, National University of Defense Technology, Changsha, China · TMLR Group, Hong Kong Baptist University, Hong Kong, China · School of Statistics and Data Science, Shanghai University of Finance and Economics, Shanghai, China +2
SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs
Jun 1, 2026Zhijing Zhang, Jiaqi Ding, Qianshan Wei +5VLM Unlearning Multimodal Large Language Models
School of Computer Science and Engineering, Southeast University · Institute of Automation, Chinese Academy of Sciences
On the Robustness of Machine Unlearning for Vision-Language Models
May 26, 2026Yujie Lin, Kaidi Jia, Jiayao Ma +2VLM Evaluation VLM Unlearning
1Xiamen University
Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning
May 26, 2026Junkai Chen, Yuhao He, Junxiang You +3VLM Unlearning Multimodal Large Language Models
Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, UCAS
CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic
May 18, 2026Shen Lin, Junhao Dong, Rongjie Chen +3VLM Unlearning Vision-Language Models
1Fujian Normal University · 2Nanyang Technological University · 3Xidian University
ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models
May 15, 2026Jiahui Guang, Haiyan Wang, Yingjie Zhu +4VLM Unlearning Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · The Hong Kong Polytechnic University, Hong Kong, China +1
ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition
May 14, 2026Shen Lin, Jing Lin, Junhao Dong +2VLM Unlearning MLLM Unlearning
1Fujian Normal University · 2Nanyang Technological University · University of New South Wales +1
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5VLM Evaluation VLM Unlearning
Harbin Institute of Technology, Shenzhen, China · Sichuan University, Chengdu, China · Harbin Institute of Technology, Weihai, China +3
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Privacy-Preserving Language Models Multimodal Large Language Models
School of Computer Science and Technology Xidian University
Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning
May 7, 2026Yuhang Wang, Zhenxing Niu, Haoxuan Ji +3VLM Unlearning Multimodal Large Language Models
School of Computer Science and Technology, Xidian University · Xi’an Jiaotong University

Oct 14, 2026J/K move · Enter open · S save

Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs

Oscar Chew, Po-Yi Lu, Yi-Jie Cheng, Kunal Jain+3

Texas A&M University · National Taiwan University · Columbia University

MLLM Unlearning Machine Unlearning Machine Unlearning Evaluation Multimodal Model Evaluation

Open paper AskPDF arXiv

As machine unlearning for multimodal large language models (MLLMs) rapidly advances, rigorous evaluation of its effectiveness becomes increasingly important. However, existing benchmarks largely inherit evaluation protocols from text-only unlearning and overlook the unique challenges of unlearning in multimodal settings. In this work, we identify three major pitfalls that may lead to misleading conclusions: the lack of heldout evaluation to distinguish genuine concept-wise forgetting from instance-wise overfitting, the reliance on synthetic, fine-tuned knowledge rather than real-world knowledge acquired during pre-training, and unfair method comparisons caused by applying the same training settings to different unlearning methods. To address these challenges, we introduce GEMMU (Generalizable Evaluation for Multimodal Machine Unlearning), a benchmark built on real-world knowledge and a principled evaluation protocol. Re-evaluating representative unlearning methods under GEMMU reveals substantial generalization gaps and challenges conclusions drawn from previous benchmarks. Our findings highlight the importance of rigorous evaluation for advancing multimodal machine unlearning and provide guidelines for future benchmark design.