cs.LGOct 5, 2025

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

Authors: Chenlu Ding, Jiancan Wu, Leheng Sheng, Fan Zhang, Yancheng Yuan, Xiang Wang, Xiangnan He

Organizations: University of Science and Technology of China · National University of Singapore · Huawei · The Hong Kong Polytechnic University

Abstract

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and harmful content. Existing unlearning approaches for MLLMs typically adapt training-based strategies such as gradient ascent or preference optimization, but these methods are computationally expensive, irreversible, and often distort retained knowledge. In this work, we propose MLLMEraser, an input-aware, training-free framework for test-time unlearning. Our approach leverages activation steering to enable dynamic knowledge erasure without parameter updates. Specifically, we construct a multimodal erasure direction by contrasting adversarially perturbed, knowledge-recall image-text pairs with knowledge-erasure counterparts, capturing both textual and visual discrepancies. To prevent unnecessary interference, we further design an input-aware steering mechanism that adaptively determines when and how the erasure direction should be applied, preserving utility on retained knowledge while enforcing forgetting on designated content. Experiments on LLaVA-1.5 and Qwen-2.5-VL demonstrate that MLLMEraser consistently outperforms state-of-the-art MLLM unlearning baselines, achieving stronger forgetting performance with lower computational cost and minimal utility degradation.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearning

    May 26, 2026Junkai Chen, Yuhao He, Junxiang You +3VLM UnlearningMultimodal Large Language Models

  2. Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

    Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4VLM UnlearningMultimodal Large Language Models

  3. Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models

    Aug 3, 2026Junxiang You, Junkai Chen, Yuhao He +3Multimodal Large Language ModelsMLLM Unlearning