UnAct: Gradient-Free Unlearning via Targeted Activation Intervention

Saeed Abdul Muizz; Aayat Rafiq; Iqra Altaf Gillani; Janibul Bashir

cs.LGOct 3, 2026

UnAct: Gradient-Free Unlearning via Targeted Activation Intervention

Authors: Saeed Abdul Muizz, Aayat Rafiq, Iqra Altaf Gillani, Janibul Bashir

Organizations: Gaash Lab National Institute of Technology Srinagar Srinagar, Jammu and Kashmir, India

Abstract

Machine unlearning seeks to remove the influence of designated training data from a trained model without retraining from scratch. Retrain-free methods such as Selective Synaptic Dampening (SSD) and its label-free variant LFSSD avoid full retraining but still require backpropagation and parameter importance computed over the entire dataset. We ask: what happens when a deletion request arrives with only a few images of the class to be forgotten? To answer this question, we introduce UnAct, a gradient-free class-unlearning method that needs only forward passes over the forget images. UnAct scores late-layer units by their responses, attenuates the most responsive connections, and repeats this for up to 20 rounds using no gradients, no labels, and no retained data. On ResNet-18 trained with CIFAR-10, CIFAR-20, and CIFAR-100, UnAct is competitive with SSD and LFSSD when forgetting entire classes and, unlike them, never collapses the network when forget data is scarce. On ResNet-18, across all tested sizes, UnAct's retain accuracy stays within 2.5 points of retraining, while SSD and LFSSD, at their full-class operating points, lose up to 86 points on some classes. With five forget images on CIFAR-10, UnAct's distance to retraining is 0.21 points, against 67 for LFSSD and 90 for SSD, and re-selecting SSD's threshold at each size with an oracle does not close the gap. In preliminary transfer to ViT-B/16, UnAct's distance to retraining is 11.5 against 33.7 for SSD, and a request is 19x faster than SSD when SSD computes its importance at request time. The code is available at https://github.com/abdulmuizz0903/UnAct

Figures & tables

Method	Requirements	Limitations
Retraining-based ( Ginart et al., 2019 ; Bourtoule et al., 2021 ; Graves et al., 2021 )	Retained data and full or partial retraining; SISA and Amnesiac also need shards, checkpoints or logged updates	Compute and storage cost; SISA and Amnesiac must be adopted at training time
Optimization-based ( Chundawat et al., 2023a ; Kurmanji et al., 2023 ; Jia et al., 2023 ; Fan et al., 2024 )	Labels and/or retained data, many gradient steps	Fine-tuning cost, data access at request time
SSD ( Foster et al., 2024b )	Labels, backward passes, Fisher information over the full dataset	Approximate method, unreliable with few forget images
LFSSD ( Foster et al., 2024c )	Backward passes, sensitivity over the full dataset	Approximate method
Unit pruning ( Wang et al., 2022 ; Pochinkov and Schoots, 2024 ; Chang et al., 2024 )	Retain or reference data, often fine-tuning	Data access, permanent removal of units
UnAct (ours)	Forward passes over the forget images	Approximate method

Table 1: Summarized Comparison of unlearning families upon deletion request

Figure 1: Overview of UnAct. Forward passes on the forget set

\mathcal{D}_{f}

score every unit of the target layer (1). Units above the

p

-th percentile are selected (2), and their incoming and outgoing weights are multiplied by

\gamma

(3). The round is repeated

k

times, re-scoring the edited model each time (4). No labels, backward passes or retained data are used.

	ResNet-18									ViT-B/16
	CIFAR-10			CIFAR-20			CIFAR-100			CIFAR-10
Method	$A_{r}$	$A_{f}$	$\Delta\!\downarrow$	$A_{r}$	$A_{f}$	$\Delta\!\downarrow$	$A_{r}$	$A_{f}$	$\Delta\!\downarrow$	$A_{r}$	$A_{f}$	$\Delta\!\downarrow$
Original model	95.16	93.28	93.72	85.15	88.96	89.50	77.31	81.00	81.39	98.33	97.92	98.11
Retrain	95.60	0.00	—	85.69	0.00	—	77.71	0.00	—	98.46	0.00	—
SSD	95.44	0.00	0.29	85.43	0.00	0.27	77.20	0.00	0.50	78.89	14.16	33.74
LFSSD	95.66	0.00	0.11	85.45	0.00	0.24	77.33	0.00	0.38	n/r	n/r	n/r

Table 2: Removing one class with the full forget set, mean over the five forget classes.

A_{r}

A_{f}

: test accuracy on the retained and forgotten classes;

\Delta

: distance to the retrained model (lower is better; best method in bold). Each method runs at its selected operating point; n/r: not run. ResNet-18 standard deviations, MIA, Avg. Gap and modified-parameter fractions are in Table 8 ; ViT-B/16 details are in Table 16 .

Figure 2: Forgetting from few images. Each method runs at its selected configuration at every forget-set size

n

(log scale; the last

n

is the whole class). (a)

\Delta

, mean over the five forget classes. (b) Retain and (c) forget accuracy: solid, mean; dotted, the worst of the five classes (lowest

A_{r}

, highest

A_{f}

); dashed, the retrained models. Read (b) and (c) together: a collapsed network also reaches

A_{f}=0

. The shaded band marks

n\leq 10

. CIFAR-100 and a log-scale

\Delta

are in Figure 5 .

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

$\alpha$	64	256	512
2	32.75	60.06	75.43
4	18.41	20.66	28.09
6	2.99	8.53	11.87
8	31.23	0.29	0.61
10	90.96	93.49	93.36
12	93.72	93.73	93.74

Appendix

Table 3: Complete SSD grid,

\Delta

averaged over the five forget classes, at each dataset’s selected

\lambda

(rows:

\alpha

, columns: importance batch size). The selected operating point is in bold; the full

\lambda

axis is in the results file.

\alpha<5

lies outside the range SSD’s paper reports and was never selected.

$\alpha$	64	256	512
2	83.98	84.22	84.49
4	10.54	54.27	74.67
6	0.11	0.78	6.54
8	3.84	8.29	7.31
10	93.72	93.72	93.72
12	93.72	93.72	93.72

Appendix

Table 4: Complete LFSSD grid,

\Delta

averaged over the five forget classes, at each dataset’s selected

\lambda

(rows:

\alpha

, columns: importance batch size). The selected operating point is in bold; the full

\lambda

axis is in the results file.

\alpha<5

lies outside the range SSD’s paper reports and was never selected.

$p$	rounds $k$
	1	5	10	20
50	0.58	5.90	7.10	7.22
75	0.15	0.36	0.48	0.46
90	61.56	0.12	0.15	0.15
95	80.79	0.14	0.12	0.11
99	89.96	78.66	52.65	0.08

Appendix

Table 5: Complete UnAct grid,

\Delta

averaged over the five forget classes, at each dataset’s selected

\gamma

(rows:

p

, columns: rounds

k

). Selected operating point in bold. The usable region runs diagonally in

(p,k)

: raising

p

and raising

k

trade off at roughly constant coverage (Eq. 1 ). The full

\gamma

axis is in the results file.

Figure 3: Complete search grids, $\Delta$ averaged over the five forget classes, with stars on the selected configurations. (a) SSD and (b) LFSSD at their selected $\lambda$ ; (c) UnAct at its selected $\gamma$ . UnAct’s usable region runs diagonally, as Eq. ( 1 ) predicts: raising $p$ and raising $k$ trade off against one another at fixed coverage.

Figure 4: One-dimensional sweeps along each method’s own axes,

\Delta

averaged over the five forget classes, log scale; stars mark the selected configurations. (a) SSD and (b) LFSSD against

\alpha

, one line per importance batch size; the shaded region

\alpha<5

lies outside the range the SSD paper reports. (c) UnAct against the attenuation factor

\gamma

at its selected

p

, one line per number of rounds

k

: flat below a threshold and degrading above it.

Method	Selection rule	CIFAR-10	CIFAR-20	CIFAR-100
SSD	Oracle ( $\min\Delta$ , needs retraining)	0.29	0.27	0.51
	Retrain-free ( $A_{f}{=}0$ , $\max A_{r}$ )	0.29	0.27	0.51
	Transfer from the other two datasets	31.23	0.54	32.42
	One configuration for all three	2.99	0.54	12.29
LFSSD	Oracle ( $\min\Delta$ , needs retraining)	0.11	0.24	0.38
	Retrain-free ( $A_{f}{=}0$ , $\max A_{r}$ )	0.11	0.24	0.38

Appendix

Table 6: Choosing the operating point without a retrained model.

\Delta

(mean over the five forget classes) of the configuration each rule picks from the method’s full grid. For class removal the retrained model’s forget accuracy is 0 by construction, so the retrain-free rule keeps the configurations with mean forget accuracy 0 and picks the one with the highest retain accuracy; it uses a labelled retain evaluation set (here the test split) but no retrained model. Transfer: the configuration with the lowest mean

\Delta

on the other two datasets. One configuration: the lowest mean

\Delta

over all three datasets.

	Retrain	SSD			LFSSD			UnAct (ours)
Class	$A_{r}$	$A_{r}/A_{f}$	$\Delta$	oracle	$A_{r}/A_{f}$	$\Delta$	oracle	$A_{r}/A_{f}$	$\Delta$	oracle
CIFAR-100 baby	77.79	77.28/0.00	0.51	0.51	77.39/0.00	0.39	0.22	76.54/0.00	1.25	0.74
CIFAR-100 lamp	77.37	77.32/0.00	0.05	0.04	77.45/0.00	0.08	0.01	77.33/0.00	0.04	0.04
CIFAR-20 electrical dev.	86.18	85.80/0.00	0.38	0.29	85.78/0.00	0.40	0.35	85.19/0.20	1.19	0.98

Appendix

Table 7: Held-out classes on which SSD’s own paper reports failures. Each method runs at the configuration it selected on the five main forget classes (transfer, the headline), with

A_{r}/A_{f}

and

\Delta

; “oracle” is its best

\Delta

over its whole grid on that class. These classes were chosen because a paper reports SSD failing on them, so they are kept out of every five-class mean. At SSD’s published default (

\alpha=10

\lambda=1

) the failure reproduces at some importance batch sizes: baby, batch 64:

A_{r}

72.8; baby, batch 256:

A_{r}

6.1; baby, batch 512:

A_{r}

2.0; lamp, batch 64:

A_{r}

73.8; lamp, batch 256:

A_{r}

54.9; lamp, batch 512:

A_{r}

34.2; electrical dev., batch 64:

A_{r}

80.3; electrical dev., batch 256:

A_{r}

85.7; electrical dev., batch 512:

A_{r}

85.6.

	Method	Configuration	$A_{r}$	$A_{f}$	MIA	$\Delta\downarrow$	Avg. Gap $\downarrow$	Params (%)	Eval
CIFAR-10	Original model	---	$95.16$ $\,\pm\,0.37$	$93.28$ $\,\pm\,3.32$	$90.4$ $\,\pm\,2.2$	93.72 $\,\pm\,3.10$	51.25	—	L4
	Retrain	100 epochs	$95.60$ $\,\pm\,0.60$	$0.00$ $\,\pm\,0.00$	$30.4$ $\,\pm\,10.1$	—	—	—	L4
	SSD	a8.0, l1.0, b256	$95.44$ $\,\pm\,0.38$	$0.00$ $\,\pm\,0.00$	$0.9$ $\,\pm\,1.8$	0.29 $\,\pm\,0.47$	9.94	3.63	L4
	LFSSD	a6.0, l0.1, b64	$95.66$ $\,\pm\,0.52$	$0.00$ $\,\pm\,0.00$	$0.0$ $\,\pm\,0.1$	0.11 $\,\pm\,0.06$	10.17	3.09	L4
	UnAct (ours)	p99.0, g0.01, k20	$95.59$ $\,\pm\,0.56$	$0.00$ $\,\pm\,0.00$	$0.0$ $\,\pm\,0.0$	0.08 $\,\pm\,0.07$	10.17	4.80	L4
CIFAR-20	Original model	---	$85.15$ $\,\pm\,0.17$	$88.96$ $\,\pm\,3.19$	$94.6$ $\,\pm\,1.2$	89.50 $\,\pm\,3.08$	55.64	—	L4

Appendix

Table 8: Full version of Table 2 : mean

\pm

s.d. over the five forget classes. Avg. Gap follows Fan et al. (2024) over the three terms we record,

|\Delta A_{f}|

|\Delta A_{r}|

and

|\Delta\mathrm{MIA}|

with MIA in points; it omits their training-set retain accuracy. Because every unlearning method drives MIA to about 0 against a retrained model at 7–30, the MIA term dominates Avg. Gap. “Params” is the percentage of parameters modified. LFSSD rows on CIFAR-20 and CIFAR-100 were evaluated on an A10 rather than the L4; the two devices differ by at most one test image on the same checkpoint.

	CIFAR-10				CIFAR-20				CIFAR-100
Method	$A_{f}$	$\Delta$	Agr f	Probe	$A_{f}$	$\Delta$	Agr f	Probe	$A_{f}$	$\Delta$	Agr f	Probe
Original model	93.3	93.73	5.3	0.992	89.0	89.49	7.2	0.991	81.0	81.39	10.0	0.981
Retrain	0.0	—	100.0	0.977	0.0	—	100.0	0.972	0.0	—	100.0	0.956
Logit mask (trivial)	0.0	0.09	62.4	0.992	0.0	0.25	46.0	0.991	0.0	0.33	45.0	0.981
SSD	0.0	0.29	65.5	0.992	0.0	0.26	48.8	0.990	0.0	0.51	39.6	0.979
LFSSD	0.0	0.11	62.5	0.991	0.0	0.24	47.4	0.990	0.0	0.38	43.2	0.979

Appendix

Table 9: What

\Delta

measures (ResNet-18, full forget set, mean over the five forget classes, selected configurations).

A_{f}

: forget test accuracy. Agr f : top-1 agreement with the retrained model on forget-class test images, i.e. whether the forgotten images go to the same classes as under retraining. Probe: test AUROC of a class-balanced linear probe for the forget class, trained on the model’s own penultimate training features. The logit mask zeroes the classifier row of the class and sets its bias to

-10^{4}

. The last two rows replay UnAct’s per-round channel masks on only the incoming (

\mathrm{In}_{j}

: last block’s conv2 filters and BatchNorm affine) or only the outgoing (

\mathrm{Out}_{j}

: classifier columns) parameters. Evaluated on the A10.

Figure 5: As Figure 2 , for the three ResNet-18 datasets, with

\Delta

on a log scale (a) so that differences below one point are visible. Lines: mean over the five forget classes; small faint markers: each class (slightly offset in

n

for legibility). SSD and LFSSD are bimodal at small

n

: on some classes retain accuracy collapses and forget accuracy falls with it, while others are barely changed. Values of

\Delta

below 0.05 are drawn at 0.05.

	CIFAR-10			CIFAR-20			CIFAR-100
$n$	SSD	LFSSD	UnAct	SSD	LFSSD	UnAct	SSD	LFSSD	UnAct
1	91.85	84.34	1.72	84.53	54.05	7.50	33.79	18.97	12.19
5	89.85	67.07	0.21	72.95	7.55	0.77	46.14	0.44	1.37
10	77.02	39.40	0.19	57.46	2.56	0.73	45.57	0.39	1.24
25	85.35	0.74	0.10	58.06	0.35	0.79	27.77	0.39	1.22
50	68.73	0.40	0.15	54.16	0.32	0.81	12.85	0.36	1.23

Appendix

Table 10: Distance to the retrained model

\Delta

against the number of forget images

n

each method is shown; the best method in each cell group is in bold. Every method runs at its single selected configuration at every

n

. The class being removed and the retain set are fixed, and SSD and LFSSD always receive their importance pass over the full training set, so

n

measures only how much evidence a method needs to locate the class. The last

n

of each dataset is the whole class.

n^{\star}

is the smallest

n

at which a method comes within

2\times

of its own best

\Delta

	CIFAR-10			CIFAR-20			CIFAR-100
$n$	UnAct	SSD	SSD- $n$	UnAct	SSD	SSD- $n$	UnAct	SSD	SSD- $n$
1	1.72	91.85	89.26 (6)	7.50	84.53	83.68 (6)	12.19	33.79	33.79 (50)
5	0.21	89.85	87.47 (6)	0.77	72.95	64.62 (50)	1.37	46.14	46.14 (50)
10	0.19	77.02	77.02 (8)	0.73	57.46	55.91 (20)	1.24	45.57	25.17 (15)
25	0.10	85.35	77.46 (20)	0.79	58.06	57.13 (6)	1.22	27.77	16.04 (30)
50	0.15	68.73	65.45 (10)	0.81	54.16	38.42 (12)	1.23	12.85	7.39 (30)

Appendix

Table 11: SSD with

\alpha

re-selected at every forget-set size (ResNet-18,

\Delta

, mean over the five forget classes). UnAct and SSD run at their full-class configurations, as in Figure 2 . SSD-

n

picks, at each

n

separately, the

\alpha

in the range reported by the SSD paper (

\alpha\in[5,50]

; value in parentheses) with the lowest mean

\Delta

, which requires a retrained model at every

n

;

\lambda

and the importance batch stay at their selected values. UnAct receives no per-

n

tuning. From the existing forget-set-size runs; “all” is the whole class (500 images on CIFAR-100).

Figure 6: Sequential deletion requests. Five one-class requests applied one after another to the same model, each method at its selected single-class configuration with no re-tuning. (a) Forget accuracy on all classes requested so far. (b) Retain accuracy. Thick lines: mean over five class orders; thin lines: individual orders. The dashed line is the original model’s test accuracy before any request, not a retrained model: most prefixes have no retrained reference. Per-request values are in Table 12 .

		SSD		UnAct (ours)
Dataset	Request	$A_{f}$	$A_{r}$	$A_{f}$	$A_{r}$
CIFAR-10 5 orders orig. 94.97	1	$0.0$ $\,\pm\,0.0$	$95.3$ $\,\pm\,0.4$	$0.0$ $\,\pm\,0.0$	$95.2$ $\,\pm\,0.3$
	2	$47.7$ $\,\pm\,0.7$	$95.2$ $\,\pm\,0.4$	$0.0$ $\,\pm\,0.0$	$95.4$ $\,\pm\,0.4$
	3	$63.3$ $\,\pm\,1.8$	$95.4$ $\,\pm\,0.7$	$0.0$ $\,\pm\,0.0$	$96.1$ $\,\pm\,1.0$
	4	$71.0$ $\,\pm\,1.5$	$95.5$ $\,\pm\,0.9$	$0.1$ $\,\pm\,0.1$	$96.4$ $\,\pm\,1.0$
	5	$76.1$ $\,\pm\,1.1$	$95.3$ $\,\pm\,1.1$	$0.3$ $\,\pm\,0.3$	$96.6$ $\,\pm\,1.1$

Appendix

Table 12: Sequential deletion requests over five class orders per dataset, drawn at random before running (one class per request, each applied to the already-edited model). Mean

\pm

sd over the class orders of the forget accuracy on all classes requested so far (

A_{f}

) and the retain accuracy (

A_{r}

), both in %. Both methods run at their selected single-class configuration, with no re-tuning; SSD recomputes its importances on the current model at every request. Most prefixes have no retrained model, so the reference is the original model’s test accuracy over all classes (“orig.”); its accuracy on the remaining classes shifts as classes are removed. The first order is the one of Table 13 (evaluated on an A10); the others were evaluated on an L4. Seed 42.

		SSD					UnAct (ours)
	Dataset	$q{=}1$	2	3	5	$A_{r}/A_{f}$	$q{=}1$	2	3	5	$A_{r}/A_{f}$
sequential	CIFAR-10	0.00	46.67	62.21	75.60	96.8/74.4	0.00	0.16	0.13	0.60	98.1/0.4
	CIFAR-20	0.43	43.73	56.37	69.07	85.0/68.9	0.63	0.39	2.84	15.21	70.1/0.2
	CIFAR-100	0.72	0.46	0.76	2.79	77.3/2.6	1.48	4.59	4.17	7.63	71.4/1.6
simultaneous	CIFAR-10	0.00	94.80	93.67	94.54	96.7/93.3	0.00	55.29	72.41	84.76	98.4/84.3
	CIFAR-20	0.43	86.83	88.74	89.97	84.1/89.0	0.63	13.64	31.10	60.51	85.3/60.4

Appendix

Table 13: Removing several classes, with each method at its selected single-class configuration (no re-tuning for the task). Sequential : one class per request, applied to the already-edited model; the column is the number of requests so far. Simultaneous : all

q

classes in one request.

\Delta

is measured against a model retrained without all

q

classes; four-class retrained models were not trained, so

q=4

is omitted. The last column of each method is

A_{r}/A_{f}

after five classes. Sequential: the first of the five class orders of Table 12 , the only one with retrained references. Seed 42.

Dataset	Classes	Selectivity $\rho$	Forget mass (%)	Retain mass (%)	Ratio
CIFAR-10	10	$0.574$ $\,\pm\,0.053$	$83.7$ $\,\pm\,3.5$	$19.1$ $\,\pm\,0.5$	$4.38$ $\,\pm\,0.24$
CIFAR-20	20	$0.414$ $\,\pm\,0.052$	$74.6$ $\,\pm\,0.5$	$23.4$ $\,\pm\,0.4$	$3.19$ $\,\pm\,0.03$
CIFAR-100	100	$0.298$ $\,\pm\,0.066$	$64.0$ $\,\pm\,4.6$	$25.2$ $\,\pm\,0.3$	$2.54$ $\,\pm\,0.16$

Appendix

Table 14: Class selectivity of the selected channels at a fixed

p=75

, measured on the

512

-channel output of the last residual block. These are forward-pass statistics and no weight is modified. Per channel,

\rho=(a_{f}-a_{r})/(a_{f}+a_{r})

, where

a_{f}

and

a_{r}

are mean activations on the forget and retain sets. Forget and retain mass are the fractions of each set’s total activation that the selected channels carry. Mean

\pm

s.d. over the five forget classes.

Figure 7: Class selectivity of the selected channels, from forward passes only. Error bars are one standard deviation over the five forget classes. (a) Selectivity rises with the percentile

p

. (b) The forget-to-retain activation mass ratio of the selected channels. (c) At fixed

p=75

, as more classes share the same 512 channels, the selected channels carry less forget mass and more retain mass.

Figure 8: Cost of one request (full forget class, class 3, one L4 GPU). Distance to retraining against wall-clock time. UnAct runs the best configuration of its grid for

k=1

and

k=5

rounds and its selected configuration (filled). SSD runs its selected configuration with its full-training-set importance precomputed (filled) or computed at request time (cold, open); whiskers span the timing sessions. LFSSD’s

\Delta

is drawn as a line; it was not timed and has SSD’s two-pass structure. Paired per-session timings are in Table 15 .

Dataset	$k$	config	$\Delta$	(s)	amortised	cold	share (%)	vs. amort.	vs. cold
				UnAct	SSD (s)		full- $D$	UnAct speed-up
CIFAR-10	1	p75, $\gamma$ 0.01	0.15	0.58	1.91	20.4	91	3.31 $\times$	35 $\times$
	5	p90, $\gamma$ 0.03	0.11	3.48	2.72	30.5	91	0.78 $\times$	9 $\times$
	20 †	p99, $\gamma$ 0.01	0.08	18.27	3.08	34.5	91	0.17 $\times$	2 $\times$
CIFAR-20	1	p75, $\gamma$ 0.1	0.84	0.38	1.14	26.3	96	3.01 $\times$	69 $\times$
	5	p95, $\gamma$ 0.1	0.82	1.98	1.31	29.2	95	0.66 $\times$	15 $\times$

Appendix

Table 15: Wall-clock cost on one L4, serialised, full forget class (class 3), warm-up repeat discarded, mean of the remaining repeats. For each

k

, UnAct runs the best configuration of its grid with that number of rounds ( † : the selected configuration); its

\Delta

is the five-class grid value. SSD runs its selected configuration in the same session, so each ratio is paired. “Amortised” credits SSD’s importance pass over the full training set as precomputed and reusable, which its paper allows; that pass is the listed share of SSD’s cold cost. Ratios above 1 mean UnAct is faster; bold marks the settings where it is faster even than amortised SSD.

Method	$A_{r}$	$A_{f}$	$\Delta\!\downarrow$	worst $A_{r}$	closer	Params (%)
Original model	98.33	97.92	98.11	98.2	—	—
Retrain	98.46	0.00	—	98.1	—	—
SSD	78.89	14.16	33.74	9.6	2/5	6.44
UnAct (ours)	91.93	4.96	11.49	85.3	3/5	9.75

Appendix

Table 16: Removing one class from ViT-B/16 on CIFAR-10, mean over the five forget classes. UnAct uses the class-token score (Section 3.3 ;

p=85

\gamma=0.1

k=5

, chosen on class 3); SSD uses its best grid setting (

\alpha=5

\lambda=0.1

, importance batch 128). Worst

A_{r}

: lowest retain accuracy over the five classes; closer: classes on which the method has the smaller

\Delta

. Per-class results are in Table 17 .

Figure 9: Forget-set size on ViT-B/16 (CIFAR-10), UnAct and SSD at their selected ViT configurations at every

n

, fp32 on the full test split. (a)

\Delta

, mean over the five forget classes. (b) Retain accuracy: solid, mean; dotted, the lowest of the five classes; dashed, the retrained models. (c) Forget accuracy: solid, mean; dotted, the highest of the five classes; dashed, the retrained models. Below the whole class, SSD leaves its worst class almost entirely unforgotten, while UnAct’s mean forget accuracy is below 10% from

n=5

	Units and score	Config	$A_{r}$	$A_{f}$	$\Delta$	c0	c2	c3	c5	c8	Params (%)
UnAct	MLP, blocks 9–12, class-token logit score (reported)	p85, $\gamma$ 0.1, $k$ 5	91.93	4.96	11.49	4.6	13.1	3.0	16.1	20.7	9.753
UnAct	MLP, blocks 11–12, class-token logit score	p75, $\gamma$ 0.1, $k$ 5	95.45	41.22	44.24	50.7	33.1	22.5	63.7	51.1	5.558
UnAct	MLP, blocks 9–12, class-token positive activation	p90, $\gamma$ 0.1, $k$ 10	94.48	91.42	95.41	102.9	99.6	66.7	106.1	101.8	7.210
SSD	importance-weighted dampening	$\alpha$ 5, $\lambda$ 0.1, b128	78.89	14.16	33.74	43.4	1.8	25.8	9.1	88.5	6.437

Appendix

Table 17: ViT-B/16 score variants. Each UnAct variant’s configuration was chosen on class 3 (bf16, 2,000 retain images) and then run on all five classes at fp32 on the full test split; the rows here are that second stage. Columns 7–11 are

\Delta

per forget class. Scoring by the positive part of the activation at the class token fails (row 3), which is the reason for the logit score of Section 3.3 . SSD per class (

A_{r}/A_{f}

): 0: 98.1/43.2, 2: 98.2/1.6, 3: 96.1/23.1, 5: 92.5/2.9, 8: 9.6/0.0.

	$m=1$				$m=5$				$m=25$
Dataset	Retrain	SSD	UnAct	Ret. $A_{r}$	Retrain	SSD	UnAct	Ret. $A_{r}$	Retrain	SSD	UnAct	Ret. $A_{r}$
CIFAR-10	0.0	47.2	25.9	68.4	0.0	68.1	36.8	87.5	0.0	87.4	47.5	92.7
CIFAR-20	0.0	28.3	12.0	67.4	0.0	41.3	33.2	79.6	0.0	52.2	59.7	83.5
CIFAR-100	1.0	53.2	54.0	63.8	1.2	40.2	83.6	71.8	6.6	52.2	95.2	75.2

Appendix

Table 18: Forget accuracy (%) after 10 relearning steps, each on

m

forget images mixed

4{:}1

with retain images (SGD, lr 0.01), mean over the five forget classes. The retrained model is the reference: it never saw the class, so any forget accuracy it gains is learned from the

m

images alone. “Ret.

A_{r}

” is the retrained model’s own retain accuracy at step 10; at

m=1

the probe degrades it substantially, so small-

m

curves mix relearning with damage (Figure 10 ).

Figure 10: Forget accuracy against gradient steps of relearning on

m

forget images per step, mixed

4{:}1

with retain images (SGD, learning rate 0.01), ResNet-18. The dotted line is the retrained model’s own retain accuracy: at

m=1

the probe degrades every model, so the late steps at

m=1

mix relearning with damage. Both unlearned models regain much of the class within 10 to 20 steps, while the retrained model is still at or near zero forget accuracy at step 10.

Figure 11: Relearning on ViT-B/16 (CIFAR-10), learning rate

5\times 10^{-4}

m

forget images per step mixed

4{:}1

with retain images, evaluated in bfloat16, with retain accuracy on 2,000 retain test images and forget accuracy on the full forget test split. Mean over the five forget classes. UnAct’s edit is undone within a few steps, faster than SSD’s; the retrained model stays at zero forget accuracy for the first five steps.

Figure 12: Cumulative coverage

\Gamma_{k}

organises UnAct’s hyperparameters. (a) Across the grid at

\gamma\leq 0.1

\Delta

is U-shaped in coverage. (b) A coverage window per dataset (bar) against configurations with

\Delta\leq 2

(dots) and above (crosses); the percentage is the share of configurations the window classifies correctly. The windows were fitted on this grid, so they describe it rather than forecast a held-out one. (c) Measured coverage divided by two bounds: the naive

\min(1,k(1-p/100))

is exceeded in 489 of 1500 runs, because selection uses

\geq

against an interpolated percentile and so keeps slightly more than a

(1-p/100)

share of units, while the bound

km/N

of Eq. ( 1 ) is exceeded in 0 runs.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 349 papers

Sep 30, 2026

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector

τ_m

that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component

τ_F

to recover the retain task vector

τ_R

. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize

τ_F

in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

Haoran Tang, Andrew Tan, Rajiv Khanna

Department of Computer Science Purdue University

Machine Unlearning

Ask PDF

Apr 16, 2026

Class Unlearning via Depth-Aware Removal of Forget-Specific Directions

Machine unlearning aims to remove targeted knowledge from a trained model without the cost of retraining from scratch. In class unlearning, however, reducing accuracy on forget classes does not necessarily imply true forgetting: forgotten information can remain encoded in internal representations, and apparent forgetting may arise from classifier-head suppression rather than representational removal. We show that existing class-unlearning methods often exhibit weak or negative selectivity, preserve forget-class structure in deep representations, or rely heavily on final-layer bias shifts. We then introduce DAMP (Depth-Aware Modulation by Projection), a one-shot, closed-form weight-surgery method that removes forget-specific directions from a pretrained network without gradient-based optimization. At each stage, DAMP computes class prototypes in the input space of the next learnable operator, extracts forget directions as residuals relative to retain-class prototypes, and applies a projection-based update to reduce downstream sensitivity to those directions. To preserve utility, DAMP uses a parameter-free depth-aware scaling rule derived from probe separability, applying smaller edits in early layers and larger edits in deeper layers. The method naturally extends to multi-class forgetting through low-rank subspace removal. Across MNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet, and across convolutional and transformer architectures, DAMP more closely resembles the retraining gold standard than some of the prior methods, improving selective forgetting while better preserving retain-class performance and reducing residual forget-class structure in deep layers.

Arman Hatami, Romina Aalishah, Ilya E. Monosov

Johns Hopkins University

Machine Unlearning

Ask PDF

Sep 2, 2026

Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning

Class unlearning aims to remove a model's ability to recognize designated forget classes while preserving performance on retain classes. However, low forget accuracy after unlearning does not necessarily mean the class structure has been erased. Approximate unlearning methods can alter classifier decision boundaries while leaving recoverable structure in the representation. Prior work has shown that forget classes can be recovered, but existing approaches require real forget or retain samples, auxiliary data, or reference checkpoints. We study class relearning in a strictly source-free setting, asking whether a forget class can be recovered through a classifier-head update using only the unlearned model. Our approach rests on a theoretical analysis establishing a sufficient alignment condition under which a single gradient step on a synthetic probe set increases the expected logit margin of the forget class. Building on this, we propose a white-box Source-Free Relearning Audit (SFRA), which generates candidate embeddings in representation space and uses model-guided confidence filtering to construct high-confidence retain probes and low-confidence boundary-adjacent probes that are relabelled as the forget class. Gaussian sampling and Softmax confidence are used by default, while ablations with alternative proposal distributions and uncertainty criteria show that recoverability is not specific to these choices. To quantify recoverability, we introduce the Relearning Score (RS), which jointly measures forget-class recovery and retain-accuracy preservation, and report class-matched

Δ

RS relative to a retrained reference. Experiments on CIFAR-10, CIFAR-100, and TinyImageNet with ResNet-18, ViT-B/16, and Swin-T show that several unlearning methods exhibit substantial source-free recoverability, and that for a subset of methods this recoverability exceeds the matched retrained reference.

Zahra Dehghani, Pablo Piantanida, Mohammadhadi Shateri

LIVIA · ILLS · ´ETS Montreal +2

Machine Unlearning Image Classification

Ask PDF

Abstract

Machine Unlearning

Explore similar work

Unmerge: Efficient Machine Unlearning via Task Arithmetic

Class Unlearning via Depth-Aware Removal of Forget-Specific Directions

Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning