Beyond the Frame: Redundancy, Not Trajectory, Carries the Privacy Leak in Video Unlearning

Kavyansh Tyagi; Paolo Russo

cs.CVOct 20, 2026

Beyond the Frame: Redundancy, Not Trajectory, Carries the Privacy Leak in Video Unlearning

Authors: Kavyansh Tyagi, Paolo Russo

Abstract

Class unlearning for video segmentation aims to ensure that a removed category cannot be reconstructed from a deployed model's outputs. Current leakage evaluation algorithms inspect one frame at a time, and we show that this understates leakage by a wide margin: an attacker that averages a model's output field over a two-second clip recovers the forgotten class from every method we evaluate on VSPW and most on Virtual KITTI2, including models whose per-frame predictions are visually indistinguishable from a retrain oracle (a model trained from scratch without the forgotten class). We attribute the recoverable signal to cross-frame redundancy rather than object trajectory, establish this with a time-shuffled control that leaves leakage intact, and introduce PACE, an unlearning method that suppresses redundancy directly using an exactly permutation-invariant pooled adversarial discriminator and a two-sided temporal consistency objective. Across VSPW and Virtual KITTI2, PACE lowers pooled attacker AP to $18.79$ (person) and $12.10$ (car) on VSPW and $9.11$ (car) on Virtual KITTI~2, achieving on the latter the strongest suppression among verdict-safe methods retaining usable utility, while retaining $19.8$ mIoU on VSPW car against at most $8.3$ for any baseline achieving comparable erasure.

Figures & tables

Figure 1: The failure mode, and the control that identifies it. (a) A model can assign negligible probability to the forgotten class in every frame and still leak: a weak but temporally consistent residual survives averaging while the noise around it does not, and per-frame audits cannot observe this quantity. (b) Two mechanisms explain a temporal attacker’s advantage. Trajectory leakage requires frame order, so permuting frames destroys it; redundancy leakage requires only repetition, so permuting frames leaves it intact. Scoring both arms separates the two: in every configuration we test, the order-dependent component remains at retrain-oracle level.

Panel of Figure 2 — Figure 2: The method family. All three variants train one generator against a discriminator that differs only in what it is permitted to observe: PACE-F’s $D_{F}$ reads a single frame $O_{t}$ ; PACE-T’s $D_{T}$ reads the ordered clip $O_{1:T}$ , paired with the one-sided bound $\mathcal{L}_{tc}^{1}$ (Eq. 6 ); PACE’s $D_{P}$ reads only the time average $\bar{O}$ (Eq. 7 ), exactly order-invariant by construction, paired with the two-sided bound $\mathcal{L}_{tc}^{2}$ (Eq. 9 ) that removes the frozen-region solution.

Model	frame	shuf.	ord.	$\Delta_{\mathrm{red}}$	synergy
oracle (floor)	24.73	26.77	26.99	$+2.04$	$+0.22$
original	91.56	94.94	94.94	$+3.38$	$+0.00$
PACE-F	29.44	45.35	45.26	$\mathbf{+15.91}$	$-0.09$
PACE-T	16.42	22.62	21.26	$+6.20$	$-1.36$
SalUn [ 13 ]	49.27	83.54	84.74	$\mathbf{+34.27}$	$+1.20$

Table 1: Adversary AP by arm at

T{=}3

. The pooling gain

\Delta_{\mathrm{red}}

is largest for the methods that appear most private under a per-frame audit, while synergy remains small relative to the corresponding redundancy gain.

Model	3D CNN	ConvGRU	Sel. SSM
oracle (floor)	$-0.99$	$+2.02$	$+5.49$
original	$+2.09$	$+3.24$	$+1.63$
PACE-F	$+12.69$	$+19.21$	$+17.49$
PACE-T	$\mathbf{+2.53}$	$\mathbf{+5.43}$	$\mathbf{+7.02}$

Table 2: Pooling gain

\Delta_{\mathrm{red}}

T{=}16

under three attacker architectures. PACE-T reduces the gain by

2.5

5\times

relative to PACE-F under every attacker, and its residual tracks the irreducible floor as attacker strength increases.

Model	retain mIoU $\uparrow$	person IoU	pooled AP $\downarrow$	Verdict
oracle (floor)	30.8	0.0	30.82 $\pm$ 0.14	floor
original	33.5	81.2	93.82 $\pm$ 0.77	leak
FT [ 19 ]	17.4	0.0	48.48 $\pm$ 1.71	leak
NegGrad [ 42 ]	0.1	0.1	88.70 $\pm$ 0.39	leak
NegGrad+ [ 30 ]	4.9	0.0	68.14 $\pm$ 1.28	leak
RL [ 22 ]	19.0	0.0	86.08 $\pm$ 0.40	leak

Table 3: Person class , all thirteen models under one protocol, threshold

\mu_{\text{floor}}{+}2\sigma_{\text{floor}}

from oracle floor. SSD is the only baseline scoring safe , and it does so by destroying the model (retain mIoU

0.4

); SCRUB is the one baseline that barely erases the class at all (person IoU

36.7

) and leaks accordingly. PACE-F, PACE-T and PACE are the only leak /safe/safe sequence that comes from one continuously improved objective rather than three unrelated methods, and PACE is the only method in the table combining a safe verdict, retain mIoU above

21

, and the lowest pooled AP.

Model	retain mIoU $\uparrow$	car IoU	pooled AP $\downarrow$	Verdict
oracle (floor)	31.6	0.0	18.10 $\pm$ 5.60	floor
original	31.9	33.5	80.45 $\pm$ 1.62	leak
FT [ 19 ]	6.6	0.0	32.55 $\pm$ 3.21	leak
NegGrad [ 42 ]	0.1	0.0	6.97 $\pm$ 0.27	safe
NegGrad+ [ 30 ]	3.6	0.0	16.79 $\pm$ 2.99	safe
RL [ 22 ]	3.7	0.0	57.61 $\pm$ 4.23	leak

Table 4: Car class , all twelve models under one protocol, threshold

\mu_{\text{floor}}{+}2\sigma_{\text{floor}}

AP from oracle floor. Three baselines are verdict-safe (NegGrad, NegGrad+, SSD); all three reach it with retain mIoU

\leq 3.6

, i.e. by damaging the model rather than by selectively forgetting car . PACE-T and PACE are the only methods that are simultaneously verdict-safe and retain mIoU above

8.3

Figure 3: Car class, privacy against utility. Error bars give trial spread. Low leakage carries meaning only at usable utility, and the two least leaky baselines sit at the left edge because the model has been degraded. PACE and PACE-T occupy the useful corner.

Model	retain mIoU $\uparrow$	car IoU	pooled AP $\downarrow$	Verdict
oracle (floor)	45.0	0.0	50.71 $\pm$ 0.71	floor
original	50.1	82.8	88.67 $\pm$ 1.27	leak
FT [ 19 ]	30.9	0.0	41.43 $\pm$ 2.32	safe
NegGrad [ 42 ]	1.8	0.0	9.67 $\pm$ 1.41	safe
NegGrad+ [ 30 ]	4.9	0.0	33.57 $\pm$ 6.40	safe
RL [ 22 ]	31.7	0.0	89.17 $\pm$ 1.08	leak

Table 5: Virtual KITTI 2, car class , all twelve models under one protocol, threshold

\mu_{\text{floor}}{+}2\sigma_{\text{floor}}

AP from oracle floor. FT is verdict-safe here while retaining

30.9

mIoU, substantially lower pooled AP than the strongest-leaking baselines in this setting; its synergy of

+10.24

is also the single clearly non-zero value across every model and dataset we test. PACE attains the lowest AP of any safe method but at a lower retain mIoU (

7.6

) than PACE-T (

21.1

Figure 4: Two person examples: the model’s own per-frame prediction (predicted row) paired with the pooled attacker’s reconstruction of the same outputs (recovered row). Red marks the ground-truth class, blue the predicted class; brightness in the recovered rows is the attacker’s probability of the forgotten class. RL and SalUn appear to erase the class per frame, yet the pooled attacker recovers a clear silhouette (Tab. 3 ); Original and SCRUB already leak visibly in their own predictions, while PACE-T and PACE yield substantially weaker silhouettes. Attackers use the protocol of Tab. 3 with a reduced single-run budget.

Machine Unlearning

Machine unlearning removes the influence of selected data or concepts from trained models while preserving useful behavior and avoiding full retraining.

See all 349 papers

Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning
Feb 18, 2026Yurim Jang, Jaeung Lee, Dohyun Kim +2Machine Unlearning Machine Unlearning Evaluation
Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon, Republic of Korea
Graph Propagated Projection Unlearning: A Unified Framework for Vision and Audio Discriminative Models
Apr 13, 2026Shreyansh Pathak, Jyotishman DasMachine Unlearning Machine Unlearning Evaluation
Indian Institute of Technology Jodhpur
SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation
May 29, 2026Petros Andreou, Jamie Lanyon, Axel Finke +1Machine Unlearning Machine Unlearning Evaluation
Department of Computer Science, School of Science, Loughborough University, Loughborough, UK · School of Mathematics, Statistics and Physics, Newcastle University, Newcastle upon Tyne, NE1 7RU, UK

Feb 18, 2026J/K move · Enter open · S save

Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning

Yurim Jang, Jaeung Lee, Dohyun Kim, Jaemin Jo+1

Department of Artificial Intelligence, Sungkyunkwan University, Suwon, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon, Republic of Korea

Machine Unlearning Machine Unlearning Evaluation

Open paper AskPDF arXiv

As pretrained models are increasingly shared on the web, ensuring that models can forget or delete sensitive, copyrighted, or private information upon request has become crucial. Machine unlearning has been proposed to address this challenge. However, current evaluations for unlearning methods rely on output-based metrics, which cannot verify whether information is completely deleted or merely suppressed at the representation level, where suppression is insufficient for true unlearning. To address this gap, we propose a novel restoration-based analysis framework that uses Sparse Autoencoders to identify class-specific expert features in intermediate layers and applies inference-time steering to quantitatively distinguish between suppression and deletion. Applying our framework to 12 major unlearning methods in image classification tasks, we find that most methods achieve high restoration rates of unlearned information, indicating that they only suppress information at the decision-boundary level, while preserving semantic features in intermediate representations. Notably, even retraining from pretrained checkpoints shows high restoration, revealing that robust semantic features inherited from pretraining are not removed by retraining. These results demonstrate that representation-level retention poses significant risks overlooked by output-based metrics, highlighting the need for new unlearning evaluation criteria. We propose new evaluation guidelines that prioritize representation-level verification, especially for privacy-critical applications in the era of pre-trained models.

Abstract

Machine Unlearning

Explore similar work

Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning

Graph Propagated Projection Unlearning: A Unified Framework for Vision and Audio Discriminative Models

SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation