Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

Wei Zhai; Xiang Liu; Qiang Huang; Rui Qian; Lemao Liu; Ziwei Li; Ziqi Wang; Zhitao Huang

cs.LGOct 7, 2026

Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

Authors: Wei Zhai, Xiang Liu, Qiang Huang, Rui Qian, Lemao Liu, Ziwei Li, Ziqi Wang, Zhitao Huang, +1 more

Organizations: College of Computer Science and Artificial Intelligence, Fudan University · BEDI Cloud · School of Electrical and Information Engineering, Tianjin University · King Abdullah University of Science and Technology (KAUST), Saudi Arabia · School of Software Technology, Zhejiang University · School of Transnational Law, Peking University

Abstract

Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.

Figures & tables

Method	TF	NWM	GC	HFQ	UP
GradDiff	✗	✗	✗	✗	✗
SimNPO	✗	✗	✗	✓	✗
LUNAR	✗	✓	✗	✗	✗
Nullify	✓	✓	✓	✓	✓

Table 1: Attribute comparison of representative LLM unlearning methods. TF: Training-Free (no backpropagation); NWM: No (base) Weight Modification; GC: Geometric Constraint isolating retained knowledge; HFQ: High Forget Quality; UP: Utility Preservation.

Panel of Figure 1 — Figure 1: Pareto view of LLM unlearning on TOFU forget10 (Llama-2-7B-chat-hf). Nullify (orange star) dominates all baselines, achieving the highest forget quality at the highest model utility (a) and the lowest wall-clock cost (b); the red arrow marks the preferred direction. Wall-clock in (b) refers to the cost benchmark of Table 6 .

Method	Unlearning Efficacy				Utility Preservation
	Forget Set			FQ $\uparrow$	Real Authors $\uparrow$			World Facts $\uparrow$			Retain Set $\uparrow$			MU $\uparrow$
	1-RL	1-P	TR	FQ $\uparrow$	RL	P	TR	RL	P	TR	RL	P	TR	MU $\uparrow$
Forget05
Original ‡	0.04	0.01	0.49	0.00	0.93	0.44	0.58	0.91	0.43	0.55	0.98	0.99	0.48	0.62
Retrain †,‡	0.61	0.85	0.66	1.00	0.92	0.44	0.57	0.90	0.43	0.54	0.97	0.99	0.48	0.62
GA ‡	1.00	1.00	0.66	$<$ 0.01	0.00	0.20	0.40	0.00	0.30	0.28	0.00	0.00	0.15	0.00

Table 2: Results on TOFU Maini et al. (2024) ( forget05 / forget10 , Llama-2-7B-chat-hf). Unlearning Efficacy : forget-set 1

-

ROUGE-L, 1

-

Prob., Truth Ratio, and the summary metric FQ. Utility Preservation : RL/P/TR on Real Authors, World Facts, and the Retain set, summarized by MU. † Retrain is the gold reference; the forget-set columns are best when closest to Retrain , not when largest. Rows marked with ‡ are taken from Fan et al. (2025) . Bold : best FQ / MU among unlearning methods.

Method	Unlearning	Privacy	Utility
Method	KM-D ${}_{\text{f}}$	PrivLeak	KM-D ${}_{\text{r}}$
Original ‡	0.63	$-$ 98.71	0.54
Retrain †,‡	0.33	$-$ 0.00	0.54
GA ‡	0.00	$-$ 20.14	0.00
GradDiff ‡	0.31	108.12	0.28
Task Vector ‡	0.59	$-$ 100.00	0.48

Table 3: Results on MUSE-News Shi et al. (2025) . KM-D

{}_{\text{f}}

/KM-D

{}_{\text{r}}

: forget-/retain-set knowledge memorization (best when closest to Retrain † ,

0.33

0.54

); PrivLeak: privacy leakage (ideal

=0

, bold ). Rows marked with ‡ are taken from Fan et al. (2025) .

Attack family	Original	Retrain	Nullify
Prefix injection	0.633	0.087	0.222
Roleplay	0.434	0.071	0.111
Few-shot elicitation	0.388	0.074	0.014
Partial-answer elicitation	0.202	0.042	0.065
Multi-turn extraction	0.405	0.085	0.128
Cloze / multiple choice	0.285	0.210	0.030

Table 4: Exact-span leakage under adversarial extraction (lower is better). The mean is unweighted across families; multi-turn extraction is scored on the final extraction turn.

Prompt type	Original	Retrain	Nullify
Plain QA	0.975	0.085	0.253
Paraphrased question	0.478	0.128	0.218
Cloze completion	0.358	0.153	0.060

Table 5: Exact-span leakage under paraphrasing and reformatting (lower is better).

Method	#Train	Mem	Wall-clock	$\times$ vs
	params	(GB)	(min)	Nullify
Retrain †	6.6 B	$\sim$ 79	$\sim$ 300	57.7 $\times$
GA	6.6 B	79	14.5	2.8 $\times$
GradDiff	6.6 B	79	13.5	2.6 $\times$
IDK	6.6 B	79	$\sim$ 13	2.5 $\times$
NPO	6.6 B	79	58.2	11.2 $\times$

Table 6: End-to-end cost on TOFU forget05 / Llama-2-7B-chat-hf, single NVIDIA H800. #Train params: parameters updated by gradient descent. Wall-clock: per {hyperparameter, seed} configuration at each method’s paper-recipe epoch budget.

Variant	best $\alpha$	FQ	MU
Nullify (full)	$-2.0$	0.965	0.622
w/o null-space proj. ( $\hat{\mathbf{P}}{=}\mathbf{I}$ )	$-0.3$	0.270	0.360
w/o IDK-contrastive dir. (random $\mathbf{r}$ )	$-2.0$	0.466	0.600

Table 7: Ablation on TOFU forget05 . The

\hat{\mathbf{P}}{=}\mathbf{I}

row is shown at its own best

\alpha

(

-0.3

); by

\alpha{=}-1.0

it has already collapsed to FQ

0.00

/ MU

0.01

Figure 2: Forget-set truth-ratio density under three models on TOFU. Dashed gray: Original (memorized); red: Nullify; blue: Retrain (oracle). Nullify collapses the memorization peak back onto the Retrain envelope; the Kolmogorov–Smirnov

p

-value in each title is the Forget Quality used in Table 2 .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda$	best $\alpha$	FQ	MU
0.3	$-1.5$	0.000	0.556
0.5	$-1.5$	0.000	0.585
1	$-1.5$	0.068	0.606
3	$-1.5$	0.866	0.621
5	$-1.5$	0.628	0.623
10	$-2.0$	0.965	0.622

Appendix

Table 8:

\lambda

sweep on TOFU forget05 (

p{=}0.75

Question	Original output	Nullify-steered output
Forget set — Moshe Ben-David
Has Moshe Ben-David received any awards for Islamic literature?	Yes, the Distinguished Author Award in Islamic Literature.	This information is not provided in the given text.
Are Moshe Ben-David’s works fundamental reads of the Islamic genre?	Yes, Miracles & Merits of Allah’s Messenger and others are seminal.	This information is not provided in the reference.
Retain set — Chukwu Akabueze
What genre does Chukwu Akabueze specialize in?	Biography.	(unchanged)
Has Chukwu Akabueze won any awards?	Yes, the prestigious Nigerian Writers Award among others.	(unchanged)

Appendix

Table 12: Representative case study on TOFU forget05 (Llama-2-7B-chat-hf). On forget-set queries Nullify steers the model from verbatim recall to abstention; on retain-set queries the output is unchanged.

$\alpha$	FQ	MU
$-1.6$	0.545	0.625
$-1.8$	0.713	0.623
$-1.9$	0.924	0.622
$\mathbf{-2.0}$	0.965	0.622
$-2.1$	0.924	0.620
$-2.2$	0.924	0.617

Appendix

Table 9:

\alpha

curve on TOFU forget05 (

p{=}0.75

\lambda{=}10

$p$	best $\alpha$	FQ	MU
0.40	$-2.2$	0.793	0.624
0.60	$-2.0$	0.394	0.625
0.75	$-2.0$	0.965	0.622
0.85	$-2.1$	0.713	0.613
0.90	$-1.8$	0.545	0.614

Appendix

Table 10:

p

sweep on TOFU forget05 (

\lambda{=}10

). MU is essentially flat (

0.61

–

0.63

) across the entire sweep—the direct signature of the null-space utility guarantee.

Backbone	Method	FQ $\uparrow$	MU $\uparrow$	F-RL (target: Retrain)	R-RL $\uparrow$
Qwen3-8B	Original	$<0.001$	0.603	1.000	0.999
	Retrain †	1.000	0.596	0.434	1.000
	Nullify	0.641	0.574	0.385	0.801
Llama-2-7B-chat	Nullify (main)	0.581	0.620	0.410	0.950

Appendix

Table 11: Cross-backbone results on TOFU forget10 . F-RL/R-RL denote forget- and retain-set ROUGE-L; Retrain † is the target reference.

Explore similar work

Less is More: Geometric Unlearning for LLMs with Minimal Data Disclosure
May 3, 2026Chenchen Tan, Xinghao Li, Shujie Cui +3Privacy-Preserving Language Models Machine Unlearning
Faculty of Information Technology, Monash University, Clayton, Victoria, Australia. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.
Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
Sep 9, 2026Ravi Ranjan, Olivera Kotevska, Agoritsa PolyzouMemorization in Language Models Language Model Robustness
Florida International University (FIU), Miami, FL, USA · Oak Ridge National Laboratory (ORNL), Oak Ridge, TN, USA
Cascade: Hierarchical Recoverability Control for Large Language Model Unlearning
Sep 15, 2026Qingchen Yu, Shiying Duan, Xiaodong Li +5Privacy Leakage in Language Models Machine Unlearning
1Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University · Center for Applied Statistics, School of Statistics, Renmin University of China +1

May 3, 2026, cs.CLJ/K move · Enter open · S save

Less is More: Geometric Unlearning for LLMs with Minimal Data Disclosure

Chenchen Tan, Xinghao Li, Shujie Cui, Youyang Qu+2

Faculty of Information Technology, Monash University, Clayton, Victoria, Australia. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.

Privacy-Preserving Language Models Machine Unlearning LLM Unlearning

Open paper AskPDF arXiv

As large language models (LLMs) are increasingly deployed in real-world systems, they must support post-hoc removal of specific content to meet privacy and governance requirements. This motivates selective unlearning, which suppresses information about a particular entity or topic while preserving the LLM's general utility. However, most existing LLM unlearning methods require access to the original training corpus and rely on output-level refusal tuning or broad gradient updates, creating a tension among unlearning strength, non-target preservation, and data availability. We propose Geometric Unlearning (GU), an approach that operates directly on the model's prompt-conditioned hidden states without access to the original training corpus. Specifically, GU distills a compact, low-rank safe-behavior subspace from a small set of safe reference prompts and uses lightweight anchor-in-context synthetic prompts to trigger localized, projection-based alignment of hidden representations to this safe subspace. A teacher-distillation regularizer on synthetic non-target anchors further reduces collateral drift. Across privacy-oriented unlearning benchmarks (ToFU and UnlearnPII), GU achieves strong target suppression with minimal impact on non-target performance, demonstrating that effective unlearning can be achieved with minimal synthetic data.

Abstract

Explore similar work

Less is More: Geometric Unlearning for LLMs with Minimal Data Disclosure

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Cascade: Hierarchical Recoverability Control for Large Language Model Unlearning