Rethinking Demonstration Unlearning in Imitation Learning for Robotics

Jiazhuo Li; Yu Zhang; Yiming Fei; Kangkang Dong; Xiaojun Zhu; Houde Liu; Jinze Tao

cs.ROAug 21, 2026

Rethinking Demonstration Unlearning in Imitation Learning for Robotics

Authors: Jiazhuo Li, Yu Zhang, Yiming Fei, Kangkang Dong, Xiaojun Zhu, Houde Liu, Jinze Tao

Organizations: Zhejiang University · Tsinghua Shenzhen International Graduate School · Wuhan University of Technology

Abstract

Imitation learning for robotics depends on human demonstrations, some of which people may later ask to remove. Retraining without them is the natural reference, but its cost grows with policy and dataset scale, motivating cheaper operators that edit a trained policy. Metrics inherited from machine unlearning, such as forgetting loss or a single membership attack, do not establish what an edit removed from a policy acting in closed loop. We therefore introduce a retrain-calibrated audit that reads demonstration unlearning along two axes: behavior, whether the edited policy acts like one retrained without the removed demonstrations, and evidence, whether an auditor can still detect it was trained on them. The behavior axis measures action divergence to that retrain at matched states, calibrated by a floor built from independent retrains, so a policy at the floor is as close to a retrain as retrains are to each other. The evidence axis applies a per-demonstration membership attack against a retrain null, reporting both its rank and its absolute member-loss level, since rank alone accepts operators that inflate member losses past the null. A conformal test then combines both axes into one hypothesis of joint retrain consistency, against a fleet of independent retrains large enough to reject at conventional significance. Across five preregistered conditions on three real-robot policy classes and two simulation suites, the axes dissociate in both directions on one checkpoint, as an edit may repair task behavior while leaving evidence unchanged, or reduce evidence while moving behavior away from retraining. On the ACT arm, a redirect edit restores blind-scored robot success to 18 of 20 trials.

Figures & tables

Figure 1: Demonstration unlearning in imitation learning, and why one audit is not enough. Left : real frames from the shared teleoperated dataset — a kept demonstration seats the cup; the 30 later-revoked ones teach release at a displaced point that topples it. Middle : the policy trained on all 130 absorbs the revoked mode; the request admits retraining without the 30 (the reference $\pi^{*}$ ) or editing the weights (shown: redirect ; the operator ladder and budget-matched FT control are in Sec. 4.2 ). Right : every number is ACT, and each row’s two verdicts score the same checkpoint : similar observed task success to the retrain’s (blind-scored; Table 11 ), membership-attack AUC identical to the un-edited policy’s. The grouping switches between columns — repaired, but not forgotten (Sec. 5.1 ).

	rank AUC [95% CI]	vs. null	mem/null
$\theta_{0}$ / FT200 / R200	$1.000$ [1.00, 1.00]	$+0.36$	$0.22$ / $0.19$ / $0.40$
R400	$0.990$ [0.96, 1.00]	$+0.35$	$0.44$
B50	$0.997$ [0.98, 1.00]	$+0.36$	$0.38$
B100	$0.940$ [0.86, 1.00]	$+0.30$	$0.57$
B200	$0.650$ [0.44, 0.84]	$+0.01$	$0.84$
B300	$0.557$ [0.34, 0.76]	$-0.08$	$1.59$

Table 1: The ACT c50 evidence ladder (primary

\theta_{0}

lineage) , one attack, one null pooled over 5 retrain seeds

\times

3 frame draws (grand mean

0.639

, margin

0.105

; Sec. 4.3 ). Rank is scale-free but overshoot-blind; the absolute mem/null level is not. No rung of this ladder lands on the null in absolute terms: every episode-bootstrap CI excludes

1.0

(B200

[0.79,0.89]

; B300

[1.48,1.67]

; B400

[1.72,1.94]

). The

1.000

boundary readings replicate on three independently trained

\theta_{0}

seeds; the ascent endpoint does not (Sec. 5.1 ).

	succ.	release ( ∘ )	grasp ( ∘ )	carry (s)
ceiling	19/20	$15.1\pm 1.6$	$-20.6\pm 0.8$	2.95
redirect (H)	17/20	$14.3\pm 1.5$	$-16.6\pm 5.8$	2.43
FT (D)	18/20	$13.8\pm 2.3$	$-15.0\pm 8.8$	2.87
ascent (F)	8/20	$13.3\pm 1.8$	$-15.5\pm 7.0$	2.33
$\theta_{0}$ (B)	2/20	$11.3\pm 1.5$	$-17.2\pm 5.9$	1.92

Table 2:

\pi_{0.5}

hardware, all five conditions ,

n{=}20

each under the paired protocol. Release angle orders with success;

\theta_{0}

’s tight spread is a consistent displaced aim, not noise. Blind-scored labels (Table 11 ); continuous channels from recorded joint trajectories.

Panel of Figure 2 — Figure 2: One mechanism, measured and averaged ( $\pi_{0.5}$ ). Top: median base-yaw trajectory, band $=$ IQR, $n{=}20$ ; $\bullet$ median grasp, $\blacklozenge$ median release, dashed after it. Release markers order with success — ceiling $15.1^{\circ}$ (19/20) down to $\theta_{0}$ $11.3^{\circ}$ (2/20), which also releases earliest; the grasp-phase dispersion is the inherited 130-lineage spread, unrepaired by every operator. Bottom: each condition’s medoid end-state — the real terminal frame nearest that condition’s median endpoint, one fixed rule, nothing hand-picked (pixelwise averages: appendix); endpoint spread: tight where a modal outcome exists (redirect 5 px, ceiling/FT 8 px), $3\times$ looser where none does ( $\theta_{0}$ , ascent 22 px). The failure modes differ: $\theta_{0}$ topples the assembly; ascent sets the cup beside the target. Blind-scored; Wilson CIs and taxonomy: Table 11 .

Figure 3: The PushT arm: the deletion request, what the operator moves, and what the audits see. (a) gym-pusht at real dataset states: a clean episode ends with the T on the green goal; its mirror ( $x\to 512{-}x$ ) pursues the reflected pose — a coherent wrong-goal mode. Blue: agent path. (b) Ascent moves what it optimizes (forget $+180.9\%$ ) at a climbing price (retain $+42.7\%$ , collateral $+70.1\%$ ; s1000); dashed: the frozen dose. (c) Top: success change per seed (r050: over-budget, exploratory); frozen-dose movement sits inside FT’s own seed range. Bottom: neither evidence statistic approaches the null at any dose (rank $\leq 3.8\%$ , absolute $\leq 18.1\%$ ).

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

arm	units	reference	regions	horizon	samples
ACT	raw joint $L_{1}$	3 retrains (3 pairs)	PRE/BRANCH/POST/TAIL	25 executed †	1200 banked obs
DP	raw action $L_{1}$	4 retrains (6 pairs)	PRE/BRANCH/POST/TAIL	8 (executed)	banked obs
$\pi_{0.5}$	action MAE / flow loss	3 LoRA re-fits	—	50 (executed)	96 samples/split
PushT	closed-loop success	5-seed fleet	—	episode	250 rollouts/cond.

Appendix

Table 3: Per-arm measurement conventions. † ACT’s primary horizon is the executed 25 steps of each 50-step chunk; the full-plan figures are the late-ramp diagnostic (Appendix B , Sec. 4.3 ). Cross-arm comparisons use a matched 8-step slice. Units name the axis: action-space units read counterfactual conduct (ACT, DP,

\pi_{0.5}

); PushT’s closed-loop success reads the manifestation channel only. Membership audits: per-demonstration loss attack, mean aggregation, 2000-resample bootstrap CIs; members/non-members 30/10 (ACT, DP,

\pi_{0.5}

; 50-set episodes 0–29 vs. 40–49) and 80/80 for PushT (synthesized mirror non-members, Sec. 5.3 ); nulls pool all retrain seeds of the arm — for ACT, 5 seeds

\times

3 frame draws (Sec. 4.3 ).

	executed slice (25 steps, primary)				full plan (50 steps, diagnostic)
	BRANCH	POST	PRE	TAIL	BRANCH	POST	PRE	TAIL
floor (3 pairs)	$1.81\pm 0.07$	$2.01\pm 0.08$	$2.83\pm 0.14$	$7.27\pm 0.97$	$1.83\pm 0.06$	$3.27\pm 0.03$	$3.44\pm 0.11$	$8.93\pm 0.64$
$\theta_{0}$	3.99	13.25	4.48	16.51	9.70	16.86	6.03	22.73
R200	3.45	7.21	4.10	15.90	3.79	9.02	5.09	22.12
R400	3.48	6.27	3.84	16.37	3.80	8.06	4.72	22.76
B50	3.96	14.09	4.54	20.58	10.49	20.14	6.02	25.82

Appendix

Table 4: ACT c50 divergence to ceiling , raw joint units, forget regions, both horizon conventions. The

\theta_{0}

BRANCH gap over the floor is

2.2

raw units on the executed slice vs.

7.9

full-plan; operator ordering is identical under both.

FT control vs. learning rate (no forget objective; must move nothing)
		retain	forget	collateral
lr $10^{-6}$	steps 200/200	+0.07%	+1.62%	+1.59%
lr $2.5\times 10^{-6}$	steps 200/200	+2.15%	+2.77%	+1.86%
lr $10^{-5}$	steps 200/200	+12.29%	+9.51%	+3.50%
the voided $10^{-5}$ ascent grid (guard-stopped at 100/200 steps; lr artifact, not an operator property)
ratio 0.20	steps 100/200	+40.3%	+325.3%	+126.3%

Appendix

Table 5: The lr cliff (s1000). Loss changes from

\theta_{0}

seed	operator	$\Delta$ success (pts)	retain	forget	steps
s1000	ascent @0.20 (frozen)	-0.4	+6.1%	+43.1%	200/200
s1000	ascent @0.50	+8.0	+42.7%	+180.9%	150/200
s1000	FT200	+1.2	+2.2%	+2.8%	200/200
s2000	ascent @0.20 (frozen)	+3.6	+11.3%	+40.4%	200/200
s2000	ascent @0.50	+11.6	+33.8%	+141.0%	150/200
s2000	FT200	-1.2	+3.6%	+3.7%	200/200

Appendix

Table 6: Three-seed replication at the stable rate. Success deltas are against each seed’s own

\theta_{0}

(250 rollouts); losses from the edit reports. Ratio 0.50 moves success toward the fleet on all three seeds and is guard-stopped at 150/200 steps on all three — not budget-matched, not deployable. The dose-resolved MIA ladder exists for s1000; the r050 endpoint is measured on all three seeds (

0.964/0.981/0.985

Panel of Figure 4 — Figure 4: The robomimic-BC environments , rendered by robomimic itself at the mid-episode state of each dataset’s first demonstration (fixed rule, nothing hand-picked): PickPlaceCan (left) and NutAssemblySquare (right).

	success per seed (%)	mean $\pm$ SD
ceiling (5 retrains)	75.6 / 76.0 / 70.4 / 64.4 / 76.8	$72.6\pm 5.25$
$\theta_{0}$ (3 seeds)	52.4 / 58.0 / 63.2	$57.9\pm 5.40$

Appendix

Table 7: PushT gate, as fleets : 250 rollouts per seed. The contaminated recipe sits 14.8 points below the retrain recipe (seed-level Welch

t{=}3.78

p\approx 0.018

); every

\theta_{0}

seed falls below every ceiling seed.

Figure 5: Why localized redirect has nothing to edit under whole-episode contamination. (a) Quantile summaries (q05–q95 whiskers, q25–q75 boxes, q50 bars) of the two gating channels: forget frames sit high in action discrepancy and in state-support distance at once — state and action leave the clean distribution together. (b) Detected branch-window fraction vs. the support-gated editable intersection at the three detector calibrations: 15.7 $\to$ 3.65%, 6.2 $\to$ 0.85%, 2.0 $\to$ 0.19% — under the pre-set 5% bar everywhere, and tightening the detector monotonically worsens it. From the archived diagnostic JSON; no thresholds recomputed.

method	recovery	AUC-dist	note
redirect (ours)	$+12$ to $+33\%$	0.210	only positive-recovery method
fine-tune $\mathcal{D}_{r}$	$-3.5\%$	0.249	control; seed range $-36$ to $+49$
EU- $k$ ( Goel et al., 2022 )	$-101\%$	0.008	best evidence score; policy destroyed
Blindspot ( Tarun et al., 2023 )	$-97\%$	0.097	prior regression-unlearning method
RMU	$-101\%$	0.109
NegGrad+	$-101\%$	0.109

Appendix

Table 8: The 14-method comparison (Can corr60, identical budgets, 500-rollout evals over gate-passing seeds).

best-tuned baseline	mean SR	recovery	AUC-dist
redirect (ref)	—	$+12$ to $+33\%$	0.21
NPO+tether(10)	0.168	$-31\%$	0.119
NPO+tether(3)	0.128	$-41\%$	0.108
SimNPO	0.118	$-53\%$	0.070
NegGrad++tether(10)	0.068	$-63\%$	0.024
SSD	0.070	$-63\%$	0.218

Appendix

Table 9: The 32-configuration fairness sweep (per-method hyperparameter search plus our proximity tether transplanted into every baseline; finalists at 500 rollouts, 2 seeds; saliency-masked unlearning ( Fan et al., 2024 ) was swept in the same grid). No fairly-tuned baseline reaches positive recovery; the tether helps monotonically and rescues none; the offline/closed-loop dissociation survives tuning (NegGrad++tether: AUC-dist 0.024, dead at

-63\%

checkpoint	audit A says	audit B says	mechanism
ACT c50 redirect	behavior: 24.9% of executed-slice gap closed	evidence: AUC 1.000 $=\theta_{0}$	masking, not deletion
ACT c50 ascent (primary lineage)	rank audit stops flagging by 200–400 steps	behavior never crosses; mem/null 1.6–1.8	rank acceptance through destructive overshoot
$\pi_{0.5}$ redirect (robot)	passes all three behavior gates; success 17/20	per-demo attack: rank AUC 1.000 $=\theta_{0}$ ; mem/null 0.34	masking recurs cross-class (3-seed null)
ACT-c100 ascent	rank MIA at null (0.650 vs 0.653)	$+58\%$ past deployed null	rank statistics blind to overshoot
DP ascent-200	32 $\times$ loss-gap closure vs FT	zero action movement	loss space $\neq$ behavior space
PushT ascent @0.50 (exploratory)	success $+8.0/{+}11.6/{+}2.4$ pts (3 seeds)	retain damaged $+34$ – $69\%$ ; rank still flags all seeds (0.96–0.99)	repair by asymmetric damage

Appendix

Table 10: The blindness ledger : standard audits disagreeing on one checkpoint. Every cell is measured; rows 1–6 are the cross-class exhibits of Sec. 5 , rows 7–13 extend them. The PushT @0.50 row is exploratory (Sec. 5.3 ).

arm	condition	success	Wilson 95% CI	McNemar vs. ceiling	dominant failure mode
ACT	ceiling	20/20	[0.84, 1.00]	—	—
ACT	redirect	18/20	[0.70, 0.97]	$b{=}2$ , $c{=}0$ , $p{=}0.500$	dropped_short (1)
ACT	FT	9/20	[0.26, 0.66]	$b{=}11$ , $c{=}0$ , $p{<}0.001$	sway_and_knock (8)
ACT	ascent	7/20	[0.18, 0.57]	$b{=}13$ , $c{=}0$ , $p{<}0.001$	sway_and_knock (10)
ACT	$\theta_{0}$	5/20	[0.11, 0.47]	$b{=}15$ , $c{=}0$ , $p{<}0.001$	sway_and_knock (14)
$\pi_{0.5}$	ceiling	19/20	[0.76, 0.99]	—	dropped_short (1)

Appendix

Table 11: Robot trials by condition, blind-scored (200 anonymized shuffled clips, sealed mapping opened only after scoring): success with Wilson 95% CIs, exact McNemar vs. each arm’s ceiling paired by initial position, and the dominant failure mode. Live keys agreed with the blind labels on 199/200 episodes (blind authoritative). DP hardware is excluded from this version.

Figure 6: $\pi_{0.5}$ hardware film-strip , position-matched by the paired protocol (trial 0): the retrain seats the cup, the redirect edit matches it, $\theta_{0}$ topples the assembly. Frames at fixed fractions of each episode’s duration; terminal frame recorded by the rig.

ACT (chunk-50) primary $\theta_{0}$ lineage (seed replication: Sec. 5.1); behavior = executed-slice BRANCH closure vs. 3-retrain floor (conduct axis; full-plan: App. B ); retrain null AUC $0.639$
	behavior	evidence
	% of gap closed	rank AUC	vs. null	mem/null
$\theta_{0}$	+0.0	1.000	+0.36	0.22
redirect R200	+24.9	1.000	+0.36	0.40
redirect R400	+23.4	0.990	+0.35	0.44
ascent B200	-57.8	0.650	+0.01	0.84

Appendix

Table 12: The two-axis matrix , one block per policy class, each block read by its own pre-declared instruments (Table 3 ): behavior — % of the

\theta_{0}\!\to

reference gap closed, on the counterfactual-conduct axis by offline action gap (ACT) and on the closed-loop manifestation channel by success (PushT;

\pi_{0.5}

blind-scored). The channels dissociate on the same checkpoint (

\pi_{0.5}

FT, Sec. 5.2 ), so a manifestation cell never stands in for a conduct claim; evidence — rank AUC vs. that arm’s pooled retrain null, difference from the null, and the absolute mem/null level. A qualitative cross-arm ledger, not a quantitatively uniform comparison: cells compare within a block, never across blocks. Auto-generated from the measured artifacts by table1_matrix.py ; a cell without a measurement would print a red placeholder rather than a number.

\pi_{0.5}

evidence cells were measured 2026-08-17 (per-episode artifacts in the release archive); DP has no closure instrument, so its block is absent (Appendix 6 ).

Figure 7: Where divergence lives in time — pooled magnitudes hide it. (a) ACT

\theta_{0}

-vs-ceiling divergence by step within the 50-step chunk (pooled forget states): the profile ramps

4\to 17

raw units across the plan, which is why the full-chunk mean and the executed 25-step mean support different closure fractions (Sec. 4.3 ) while the operator ordering is unchanged. (b) DP within its executed 8-step slice. (c) The same DP divergence re-binned over normalized episode time: it concentrates mid-episode at

1.6\times

the pooled mean. The FT control tracks

\theta_{0}

almost exactly in every panel — budget-matched fine-tuning moves essentially nothing. From the banked temporal profiles ( act_floor_c50_raw.json , dp_floor_report.json ).

	audit vector $z$
	BRANCH div.	$\|\log$ m/n $\|$	$\|\Delta$ AUC $\|$	$s$	conformal $p$
$\theta_{0}$	4.00	1.54	0.34	31	0.050
FT200	3.92	1.69	0.34	33	0.050
redirect R200	3.47	0.92	0.34	18	0.050
redirect R400	3.47	0.83	0.33	16	0.050
ascent B50	3.97	0.97	0.33	21	0.050

Appendix

Table 13: Joint retrain-consistency, one test per checkpoint,

K{=}19

retrain replicas. The replicas’ own leave-one-out nonconformity spans

0.38

–

3.32

; every checkpoint scores

16

–

57

and is rejected at the fleet’s attainable minimum

p{=}0.050

. Redirect is nearest; ascent’s high rungs are farthest — rank acceptance and joint rejection disagree on the same checkpoints.

region (frame share)	$\theta_{0}$	FT200	redirect R200	ascent B300
PRE (46.6%)	0.33	0.32	0.52	2.31
BRANCH (8.6%)	0.26	0.21	1.07	2.19
POST (7.8%)	0.20	0.17	0.68	1.27
TAIL (37.0%)	0.17	0.13	0.20	1.30

Appendix

Table 14: Region-resolved mem/null . Redirect reaches the retrain null on the

8.6\%

of frames it edits and nowhere else — the episode-mean statistic dilutes a real, local repair

12{:}1

. Ascent (the overshooting lineage) overshoots every region; FT deepens memorization slightly everywhere.

Explore similar work

Jun 4, 2026

Auditing Demonstration Curation Metrics: Action-Only Scorers Fail on the Structural Defects That Degrade Imitation Policies

Imitation-learning policies inherit the quality of the demonstrations they are trained on, and a growing set of curation metrics promise to score and filter low-quality demonstrations automatically. These metrics are each validated on different data with different protocols, so it is unclear which of them actually identify the demonstrations that harm a policy. We build a controlled testbed in which demonstration defects are injected with known type, and audit seven curation metrics along two axes: how well each separates defective from clean demonstrations, and whether training a behavior-cloning policy on each metric's curated subset improves task success. We study two defect regimes. Subtle perturbations (correlated action noise, tremor, truncation) are detectable by multivariate outlier scoring and, once removed, recover the full downstream gap. Structural errors, where the demonstration executes a wrong action at a key moment, are invisible to every action-only metric we test, and two of them are inverted: they score defective demonstrations as higher quality and, used for curation, tend to leave the policy at or below the uncurated baseline rather than above it. Only metrics that examine the state trajectory detect structural errors, and even the best of them recovers just a third of the downstream gap. High detection accuracy does not guarantee downstream improvement. We release the testbed and all curation implementations.

Aarav Bedi

University of California, Berkeley · Department of Mechanical Engineering University of California, Berkeley Berkeley, California, USA

Imitation Learning Learning from Demonstration Behavior Cloning

Ask PDF

Oct 13, 2026

OneDemo: Unified Policy Learning and Data Generation from a Single Demonstration

Learning robot policies from demonstrations is often limited by the cost of collecting diverse, high-quality expert data. While an appealing alternative is to expand a single successful demonstration into a diverse set of successful trajectories, existing augmentation methods typically rely on manually designed transformation rules or complex trajectory planning, requiring substantial task-specific engineering and human effort. We introduce OneDemo, a reinforcement learning framework that learns to edit a single successful demonstration for new task configurations. We formulate demonstration editing as a structured reinforcement learning problem, in which an editor learns, from sparse task-success rewards, both when to transition between object-centric trajectory segments and how to geometrically adapt each segment. The learned editor serves both as a task policy and as a reusable demonstration generator. Across 15 Meta-World tasks, OneDemo reaches a 92.7% success rate within 53.7k environment interactions on average. Across 9 DexMimicGen tasks, it generates successful demonstrations with a 98.4% average success rate, and BC-RNN policies trained on the generated data achieve 99.6% average success at matched dataset scale. These results show that learning to edit a single successful demonstration provides a structured and effective basis for both policy learning and scalable demonstration generation.

Ying Qian, Jianshu Hu, Han Fang +2

Stanford University · Shanghai Jiao Tong University · Tsinghua University

Robot Policy Learning Learning from Demonstration RL for Robotics Synthetic Data Generation for Robotics Robot Imitation Learning

Ask PDF

Aug 23, 2026

The Imitator Game: Benchmarking Robot Imitative Ability Beyond Action Prediction

Humans imitate at the level of intent: given a demonstration, we infer its goal and carry it out with whatever tools, objects, and layouts are at hand. Current robot policies instead learn observation-to-action mappings from visual inputs and language instructions, without explicitly inferring the demonstrated task. Learning from human video thus remains largely trajectory-level: models can replay motions in near-identical scenes, but still struggle to imitate what the demonstrator intends rather than merely what they do. We introduce The Imitator Game, a four-level benchmark (L0-L3) that progressively widens the gap between the human demonstration and the robot's own scene, isolating where trajectory replay ceases to suffice and task understanding becomes necessary. We pair it with IG-10K, the largest environment-aligned paired human-robot dataset to date and the only one instantiated across all four levels in both real and simulated settings (20,000+ paired episodes, 50+ tasks, 6 domains), and Imitator Arena, an open platform for blind A/B human evaluation. Across nine state-of-the-art models, performance is stable from L0 to L2 but collapses at L3, identifying functional substitution - achieving the same intent through a different object affordance - as the decisive barrier to intent-level imitation. Human-video-conditioned models outperform caption-conditioned ones, yet every model falls below 13% zero-shot success on unseen tasks; fine-tuning IG-10K-pretrained models with only

10

paired human-robot demonstrations yields large gains that grow with pretraining scale. The project website and access to Imitator Arena are available at https://imitator-game.github.io.

Xunzhe Zhou, Yiyang Cai, Fengyi Wang +9

The University of Hong Kong · TranscEngram · Fudan University +1

Imitation Learning Robot Imitation Learning Learning from Demonstration Cross-Embodiment Robot Learning

Ask PDF