cs.LGNov 2, 2026

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning

Authors: Abudukelimu Wuerkaixi, Qizhou Wang, Sen Cui, Wutong Xu, Bo Han, Gang Niu, Masashi Sugiyama, Changshui Zhang

Organizations: Institute for Artificial Intelligence, Tsinghua University (THUAI); Beijing National Research Center for Information Science and Technology (BNRist); Department of Automation, Tsinghua University, Beijing, P.R.China · TMLR Group, Department of Computer Science, Hong Kong Baptist University · RIKEN · The University of Tokyo

Abstract

With the growing deployment of large language models (LLMs) across diverse domains, concerns regarding their safety have grown substantially. LLM unlearning has emerged as a pivotal approach to removing harmful or unlawful content while maintaining utility. Despite increasing interest, the challenges of continual unlearning, which is common in real-world scenarios, remain underexplored. Successive unlearning tasks often lead to intensified utility degradation. To effectively unlearn targeted knowledge while preserving LLM utility, it is essential to minimize changes in model parameters by selectively updating those linked to the target knowledge, thereby ensuring other knowledge remains unaffected. Building on the task vector framework, we propose a new method named ALKN (Adaptive Localization of Knowledge Negation), which uses dynamic masking to sparsify training gradients and adaptively adjusts unlearning intensity based on inter-task relationships. Comprehensive experiments across three well-established LLM unlearning datasets demonstrate that our approach consistently outperforms baseline methods in both unlearning effectiveness and utility retention under continual unlearning settings.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Preserving Utility in Continual LLM Unlearning via Orthogonal Projections

    Oct 29, 2026Juan Belieni, Ana Carolina Erthal, Eliezer de Souza da Silva +1LLM Unlearning

  2. Disentangling Forgetting from Utility: Lightweight LLM Unlearning via Orthogonal Null-Space Projection

    Nov 13, 2026Xiuyuan Wang, Weiming Liu, Zongxin Yang +7LLM UnlearningSafety-Utility Trade-Off in Language Models

  3. CALIBURN: Self-Calibrated LLM Unlearning Alignment

    Feb 2, 2026Zhengbang Yang, Yisheng Zhong, Junyuan Hong +1LLM Unlearning