cs.LGNov 13, 2026

Disentangling Forgetting from Utility: Lightweight LLM Unlearning via Orthogonal Null-Space Projection

Authors: Xiuyuan Wang, Weiming Liu, Zongxin Yang, Fan Wang, Liemiao Jin, Rubin Qin, Wei Chen, Bin Shi, 

Organizations: Zhejiang University, Hangzhou, China. · University of Illinois Chicago, Chicago, IL, USA. · Zhejiang Tailong Commercial Bank, China.

Abstract

Large Language Models (LLMs) pose significant security risks by retaining private or hazardous information, creating an urgent need for unlearning techniques that excise specific knowledge without compromising general utility. However, prevailing fine-tuning-based unlearning methods struggle to achieve this balance, often suffering from computational inefficiency and unintended utility degradation due to the entangled nature of knowledge within model parameters. To disentangle forgetting from utility, we propose UUE, a theoretically grounded framework that reformulates unlearning as controllable model editing. First, UUE projects parameter updates into the orthogonal left null space of retained knowledge representations, strictly ensuring that the editing process causes minimal interference to the model's general capabilities. Second, distinct from targeted mapping, we introduce a convex untargeted objective that derives a closed-form analytical solution to effectively disrupt undesired knowledge by pushing forget-set embeddings toward their antipodal points. We implement UUE via lightweight, pluggable adapters (including a low-rank variant, UUE-L), enabling efficient deployment without modifying the frozen backbone. Extensive experiments on TOFU and WMDP benchmarks demonstrate that UUE and UUE-L achieve superior unlearning efficacy, significantly outperforming existing methods.

Figures & tables

Explore similar work

CardsList
  1. Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

    Oct 7, 2026Wei Zhai, Xiang Liu, Qiang Huang +6LLM UnlearningPrivacy Leakage in Language Models

  2. Towards Mitigating Excessive Forgetting in LLM Unlearning via Entanglement-Guidance with Proxy Constraint

    Aug 28, 2025Zhihao Liu, Jian Lou, Yuke Hu +6Machine UnlearningLLM Unlearning

  3. Representation-Guided Parameter-Efficient LLM Unlearning

    Apr 19, 2026Zeguan Xiao, Lang Mo, Yun Chen +4Machine UnlearningLLM Unlearning