VLM Unlearning

VLM: Vision-Language Model

Vision-language model unlearning asks how to remove targeted information—such as sensitive identities, copyrighted content, or spurious associations—from models that connect images and text without losing useful capabilities. Methods include refusal fine-tuning and reinforcement learning, activation or representation editing, concept- and layer-selective updates, null-space constraints, and distillation. Evaluations use forget and retain sets across visual question answering, identity, concept, and domain tasks, increasingly testing cross-modal transfer, alternate prompts, recoverability, fairness, and collateral damage. A central challenge is that suppressing an answer may leave knowledge recoverable through another query or modality, while stronger or repeated edits can degrade general utility and multimodal alignment.

Latest papers

31 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4VLM Unlearning Multimodal Large Language Models
Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands

Nov 25, 2025J/K move · Enter open · S save

Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

Zhen Zeng, Leijiang Gu, Zhangling Duan, Feng Li+3

Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands

VLM Unlearning Multimodal Large Language Models Privacy Leakage in Language Models MLLM Unlearning Machine Unlearning LLM Unlearning

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) can inadvertently memorize privacy-sensitive information during training. While existing unlearning methods can remove such content, they often severely degrade the model's foundational capabilities, such as general image understanding. This critical shortfall motivates our investigation into benign memory forgetting, the precise removal of targeted, privacy-sensitive knowledge while rigorously preserving unrelated capabilities. To pioneer and evaluate progress toward this objective, we introduce S-MLLMUn Bench, the first benchmark designed to jointly and quantitatively assess an unlearning method's efficacy in knowledge erasure and the preservation of image understanding. Furthermore, we propose the Sculpted Memory Forgetting Adapter (SMFA), a new framework that enables benign memory forgetting. SMFA confines forgetting to designated memory regions, maintaining overall model performance. By initially fine-tuning the model to replace sensitive outputs with refusals, SMFA generates a memory forgetting adapter, followed by a retaining anchor-guided masking mechanism that safeguards unrelated knowledge. Extensive experiments on S-MLLMUn Bench demonstrate that existing methods fail to achieve benign forgetting, whereas our proposed SMFA serves as an effective baseline, successfully achieving targeted knowledge erasure without compromising the model's foundational visual capabilities. Code and data are available at https://github.com/zeng-zhen/S-MLLMUn.