Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning

Xingjian Zhao; Mohammad Mohammadi Amiri; Malik Magdon-Ismail

cs.LGOct 28, 2026

Loss is Not Behavior: A Unified Output-Space Analysis of Gradient-Based Machine Unlearning

Authors: Xingjian Zhao, Mohammad Mohammadi Amiri, Malik Magdon-Ismail

Organizations: Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY 12180, USA

Abstract

Gradient-based machine unlearning methods typically ``unlearn'' by optimizing forget and retain losses, but similar losses do not necessarily imply similar output-space behavior. Prior work has shown that high forget loss alone does not certify that unlearning has produced retraining-like output behavior. We therefore ask: when a gradient-based unlearner differs from the model that would have been learned on only the retain data, what structure does the resulting output-space gap have? We derive closed-form expressions for this gap in linear regression, with an extension to nonlinear models via a generalized Gauss-Newton and local linearity approximations. Our analysis puts various gradient-based methods under a unified framework parametrized by geometry, reweighting, and projection. Two leverage matrices emerge that capture the forget-retain entanglement under the retrained model geometry: forget self-leverage and retain cross-leverage. We show both theoretically and empirically that these two leverage matrices drive the impact of the forget residual on the forget and retain outputs, and existing methods account for this geometry only partially. Our framework yields precomputable leverage diagnostics for auditing forget sets to identify what type of gradient-based unlearning is likely to be successful.

Figures & tables

Figure 1: Loss and weights are not behavior. MNIST [ 20 ] with multi-layer perceptron (MLP) model for class forgetting with

10

seeds for various gradient-based methods and a Newton step unlearning. From left to right: forget cross-entropy (matched to the retrained reference, dashed), weight distance

\|\boldsymbol{\theta}-\boldsymbol{\theta}_{r}^{\star}\|_{2}

, KL

(p_{\boldsymbol{\theta}_{r}^{\star}}\|p_{\boldsymbol{\theta}})

\mathcal{D}_{f}

, and Top-1 forget accuracy. Setup is detailed in Appendix B .

Method	Output gap $\;\boldsymbol{X}_{a}(\boldsymbol{\theta}_{\rho}^{\text{method}}-\boldsymbol{\theta}_{r}^{\star})$	Geometry
GradDiff	$\tfrac{\rho}{n}\,\boldsymbol{K}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n}\boldsymbol{K}_{ff}\big)^{-1}\boldsymbol{e}_{f}^{-}$	retrained Hessian
NPO	$\tfrac{\rho}{n}\,\boldsymbol{K}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n}\boldsymbol{W}^{-}_{\mathrm{NPO}}\boldsymbol{K}_{ff}\big)^{-1}\boldsymbol{W}^{-}_{\mathrm{NPO}}\boldsymbol{e}_{f}^{-}$	retrained Hessian
SimNPO	$\tfrac{\rho}{n}\,\boldsymbol{K}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n}\boldsymbol{W}^{-}_{\mathrm{SimNPO}}\boldsymbol{K}_{ff}\big)^{-1}\boldsymbol{W}^{-}_{\mathrm{SimNPO}}\boldsymbol{e}_{f}^{-}$	retrained Hessian
Pure GA	$-\,\boldsymbol{X}_{a}\boldsymbol{\theta}_{r}^{\star}-\tfrac{\rho}{n\lambda}\,\boldsymbol{G}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n\lambda}\boldsymbol{G}_{ff}\big)^{-1}\boldsymbol{y}_{f}$	Euclidean Gram
IMU	$-\,\boldsymbol{X}_{a}\boldsymbol{\theta}_{r}^{\star}-\tfrac{\rho}{n\lambda}\,\boldsymbol{G}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n\lambda}\boldsymbol{A}_{\mathrm{IMU}}\boldsymbol{G}_{ff}\big)^{-1}\boldsymbol{A}_{\mathrm{IMU}}\boldsymbol{y}_{f}$	Euclidean Gram
ProjGD (fixed)	$\tfrac{\rho}{n}\,\boldsymbol{K}^{\mathrm{P}}_{af}\,\big(\boldsymbol{I}-\tfrac{\rho}{n}\boldsymbol{K}^{\mathrm{P}}_{ff}\big)^{-1}\boldsymbol{e}_{f}^{-}$	retrained Hessian (projected)

Table 1: Theoretical output gaps from

\boldsymbol{\theta}_{r}^{\star}

in the linear setting. Each row assumes that its displayed inverse exists. The GradDiff, Pure GA, fixed-weight IMU, fixed-projector ProjGD, and Newton Step rows are exact for the displayed linear objectives; NPO and SimNPO are frozen-weight local forms, with weights evaluated at

\boldsymbol{\theta}_{r}^{\star}

. The geometry column distinguishes methods that route forget perturbations through

\boldsymbol{H}_{r}^{-1}

(top group: GradDiff, NPO, SimNPO, fixed-projector ProjGD) from those that use raw Euclidean Gram structure (Pure GA, IMU). Newton Step recovers

\boldsymbol{\theta}_{r}^{\star}

exactly in linear regression. Proofs in Appendix A.6 .

Per-method output-space deviation from $\boldsymbol{\theta}_{r}^{\star}$ (mean $\pm$ std)
Forget-set geometry under $\boldsymbol{\theta}_{r}^{\star}$		$\lambda_{\max}(\boldsymbol{K}_{ff})$	$\\|\boldsymbol{K}_{rf}\\|_{F}$	$\\|\boldsymbol{e}_{f}^{-}\\|$
Random ( $300$ runs)		$1.45\!\times\!10^{6}$	$4.74\!\times\!10^{5}$	$5.28$
Class ( $30$ runs)		$2.27\!\times\!10^{6}$	$6.26\!\times\!10^{5}$	$24.25$
Method	Regime	RMS f	RMS r	RMS f /RMS r
GradDiff	random	$0.010{\scriptstyle\,\pm\,0.016}$	$0.009{\scriptstyle\,\pm\,0.016}$	$1.12$
GradDiff	class	$0.038{\scriptstyle\,\pm\,0.037}$	$0.031{\scriptstyle\,\pm\,0.029}$	$1.23$

Table 2: Class vs. random forgetting on MNIST. Top: mean leverage estimates using retain-normalized GGN curvature at

\boldsymbol{\theta}_{r}^{\star}

. Bottom: per-coordinate logit RMS gaps from

\boldsymbol{\theta}_{r}^{\star}

\mathcal{D}_{f}

and

\mathcal{D}_{r}

and their ratio. Class-regime forget accuracy is zero for GradDiff, NPO, SimNPO, and ProjGD, and

0.005\pm 0.020

for Newton Step. Loss ranges are summarized in the text; setup details in Appendix B .

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 356 papers

Erased, but Not Gone: Output Forgetting Is Not True Forgetting
Jun 23, 2026Teresa Pui Yee Yong, Win Kent Ong, Chee Seng ChanMachine Unlearning Machine Unlearning Evaluation
Universiti Malaya, Kuala Lumpur, Malaysia · VinUniversity, Hanoi, Vietnam
Variance-Reduced $(\varepsilon,δ)-$ Unlearning using Forget Set Gradients
Feb 16, 2026Martin Van Waerebeke, Marco Lorenzi, Kevin Scaman +2Machine Unlearning
INRIA Paris · INRIA Sophia Antipolis · CMAP - Ecole polytechnique
Unmerge: Efficient Machine Unlearning via Task Arithmetic
Sep 30, 2026Haoran Tang, Andrew Tan, Rajiv KhannaMachine Unlearning
Department of Computer Science Purdue University

Jun 23, 2026J/K move · Enter open · S save

Erased, but Not Gone: Output Forgetting Is Not True Forgetting

Teresa Pui Yee Yong, Win Kent Ong, Chee Seng Chan

Universiti Malaya, Kuala Lumpur, Malaysia · VinUniversity, Hanoi, Vietnam

Machine Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced membership inference. But if output-level success can coexist with retraining-inconsistent residuals in representation space, what kind of forgetting are current evaluations actually certifying? We study this question through retraining-consistent representation forgetting, using a population of retrained models (i.e., trained from scratch without the forget data) as an operational reference distribution for ideal forgetting. Across multiple unlearning methods, datasets, and models, our theoretical analysis and empirical results show that standard output-level evaluation can systematically overestimate the success of unlearning. Under this stronger lens, current methods often appear forgotten at the output layer while exhibiting a structured mismatch relative to retraining. They partially align with retraining on forget samples, remain more inconsistent on retain samples, and tend to leave residual discrepancy localized along retraining-related directions rather than diffuse in representation space. This structured mismatch is characterized by forget/retain asymmetry, directional mismatch, and concentrated residuals along retraining-related directions. In many cases, the discrepancies exceed the variability of independent retraining and lead to a risk of information recovery during relearning attacks. Overall, the results suggest that current MU is often evaluated for apparent forgetting rather than retraining-consistent forgetting. More broadly, retraining reveals what output forgetting hides.

Abstract

Machine Unlearning

Explore similar work

Erased, but Not Gone: Output Forgetting Is Not True Forgetting

Variance-Reduced (ε,δ)−(\varepsilon,δ)-(ε,δ)−Unlearning using Forget Set Gradients

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Variance-Reduced $(\varepsilon,δ)-$ Unlearning using Forget Set Gradients