REDIRECT: A 1% Fix for Bad Robot Habits

Yu Zhang; Jiazhuo Li; Yancong Wei; Kangkang Dong; Xiaojun Zhu; Houde Liu

cs.ROOct 2, 2026

REDIRECT: A 1% Fix for Bad Robot Habits

Authors: Yu Zhang, Jiazhuo Li, Yancong Wei, Kangkang Dong, Xiaojun Zhu, Houde Liu

Organizations: Tsinghua University. · University of Michigan.

Abstract

Robots can acquire bad habits from a few defective moments in otherwise useful teleoperation. In mixed-quality robot data, normal and problematic demonstrations share most task behavior and differ only at a local action continuation. Full retraining is costly, while fine-tuning on clean data alone offers limited recovery under a small update budget. We ask whether the local difference can instead be fixed with only 1% of the full-training sample-backward budget. Using only episode-level retained/problematic labels, REDIRECT localizes the branch, assigns a coherent retained continuation to the original observations around it, and anchors shared behavior, without frame-level annotations or additional interaction. Across three randomized ManiSkill tasks and three seeds, REDIRECT raises the mean success rate from 68.2% to 90.3%, recovering 88.8% of the clean-retraining gap versus 22.1% for matched-compute fine-tuning. On a PiPER arm, it recovers 86.7-92.9% of the clean-only gap across cup insertion and towel folding. Local robot habits can therefore be repaired by spending the update budget on the behavioral difference rather than relearning shared behavior.

Figures & tables

Fig. 1: Overview of Redirect at the 1% operating point. (a) Retained and problematic demonstrations share a prefix, then diverge at branch $b$ into retained and failed continuations. (b) Schematic: Redirect extracts motion-transition candidates, retrieves comparable retained futures using kinematics and RGB, localizes the largest rise in clean-calibrated future-action departure, selects one coherent retained continuation, and performs an edit/retain-anchored local update. (c) On ACT real-robot cup insertion, Redirect improves $\theta_{0}$ from 5/20 to 18/20 successes, close to clean-only at 20/20, while requiring 176.3 s of optimization versus 13,645 s for full training (1.29%).

Fig. 2: Randomized ManiSkill tasks with paired clean and problematic local behaviors. Correct (cyan) and erroneous (purple) operation points are marked in each scene; the lower-left and lower-right insets show matched clean and problematic intermediate states. PushCube arrows connect the current gripper center to the aligned or offset endpoint, while StackCube uses a side view to expose placement alignment.

Fig. 3: Automatic branch localization and repair return. At matched sample-backward budgets, Redirect converts a 1%-scale update into substantially more clean-gap recovery than ordinary fine-tuning across simulation and real-robot settings; open circles denote matched clean FT, filled squares denote Redirect , and the triangle denotes four-times-SB

\pi_{0.5}

fine-tuning. Panel (b) shows the empirical cumulative distribution of absolute localization error over 90 designated simulation episodes; the shaded band marks the

\pm 5

-frame tolerance, and synthetic branch timestamps are used only for audit.

	Full training		1% SB budget
Task	Clean	$\theta_{0}$	FT-CM	Supp.	Redirect
PickCube	$93.0{\pm}1.0$	$65.7{\pm}2.9$	$74.7{\pm}4.0$	$79.3{\pm}7.5$	$\mathbf{92.0{\pm}2.6}$
PushCube	$96.0{\pm}4.0$	$74.3{\pm}3.1$	$71.7{\pm}11.0$	$61.7{\pm}11.9$	$\mathbf{92.7{\pm}1.2}$
StackCube	$90.3{\pm}2.3$	$64.7{\pm}2.5$	$74.7{\pm}7.2$	$69.7{\pm}14.6$	$\mathbf{86.3{\pm}4.7}$
Mean	$93.1{\pm}1.3$	$68.2{\pm}1.4$	$73.7{\pm}4.1$	$70.2{\pm}5.0$	$\mathbf{90.3{\pm}2.6}$

TABLE I: One-percent repair on Independent100. Success (%), mean

\pm

SD over three training seeds. All post-training methods use the same 1%-SB budget; bold denotes the best.

Variant	Success $\uparrow$	RR $\uparrow$	RG $\downarrow$
Random Window	81.3	59.6	9.7
Local Clean FT	77.8	44.0	7.7
K-neighbor average	87.7	78.2	7.9
Per-chunk reselection	82.0	71.3	12.9
Per-step reselection	84.4	70.6	9.0
Shuffled relative time	86.3	73.6	7.4

TABLE II: Branch/target construction audit on Independent100 (macro %); budget and optimizer are fixed.

Variant	Ret.	Edit scale	Proj.	Succ. $\uparrow$	RR $\uparrow$	RG $\downarrow$
Starting $\theta_{0}$	–	–	–	68.2	–	–
Edit only	N	Raw	N	39.0	33.4	58.6
Edit $+$ retain	Y	Raw	N	86.0	75.5	8.5
Fixed $\lambda$	Y	0.32	N	85.3	78.8	11.4
Norm-balanced	Y	Norm	N	88.0	76.9	6.7
Projected, no norm	Y	Raw	Y	84.7	74.6	10.0

TABLE III: Why the 1% fix stays local: preservation-factor ablation on Independent100 (macro %). All variants use 250 updates and 4,000 SB.

Policy	Method	Upd.	SB	Succ.	Wilson CI	GapRec
$\pi_{0.5}$ /C	$\theta_{0}$	full	–	2/20	[2.8,30.1]	0.0
	Clean FT	250	8000	2/20	[2.8,30.1]	0.0
	Clean FT	1000	32000	15/20	[53.1,88.8]	76.5
	Redirect	250	8000	17/20	[64.0,94.8]	88.2
	Clean-only	30000	–	19/20	[76.4,99.1]	100.0
ACT/C	$\theta_{0}$	full	–	5/20	[11.2,46.9]	0.0

TABLE IV: Real-robot results. SB is sample-backward count; C/T denote cup/towel; intervals are Wilson 95% CI; GapRec uses Eq. 14 .

Method	Success $\uparrow$	GapRec $\uparrow$	RR $\uparrow$	RG $\downarrow$
Clean-only	$89.3{\pm}1.5$	100.0	–	–
$\theta_{0}$	$63.3{\pm}3.5$	0.0	–	–
FT-CM	$66.0{\pm}5.6$	$11.3{\pm}16.3$	$40.8{\pm}4.8$	$19.6{\pm}9.9$
Suppression	$64.0{\pm}15.6$	$5.7{\pm}46.8$	$53.2{\pm}7.8$	$30.1{\pm}20.2$
Redirect	$\mathbf{81.0{\pm}1.7}$	$\mathbf{68.6{\pm}7.7}$	$\mathbf{70.0{\pm}0.3}$	$\mathbf{12.7{\pm}2.0}$

TABLE V: Heterogeneous-Defect30 StackCube (%, mean

\pm

SD over three seeds).

Fig. 4: Branch-local policy diagnostics on problematic real trajectories. (a) ACT branch-relative original-label loss shift from

\theta_{0}

(0%) toward clean-only (100%); shading marks BRANCH observations. (b)

\pi_{0.5}

problematic-pattern assignment and clean-directed action progress. Error bars in (a) and brackets in (b) are 95% episode-bootstrap intervals.

Fig. 5: Representative real-robot rollouts aligned to the frozen manifest branch for the corresponding placement. Columns show pre-branch, branch, immediate post-branch, and terminal observations; rows compare $\theta_{0}$ and Redirect on cup insertion and towel folding. Blue, red, and green borders mark the aligned branch, labeled failure, and labeled success, respectively. Table IV reports all trials.

Policy	Method	Upd.	SB	Opt. (s)	%Full
$\pi_{0.5}$	Full train	30000	480000	55807–57276	100.0
	Clean FT	250	8000	832.8–833.6	1.45–1.49
	Clean FT ∗	1000	32000	3318.2	5.95
	Redirect	250	8000	878.0–890.6	1.55–1.57
ACT	Full train	50000	400000	13645.0	100.0
	Clean FT †	200	3200	192.5	1.41

TABLE VI: Optimization wall-clock and percentage of clean-only full retraining; ranges span tasks.

Explore similar work

RoboCoach: World Models as Active Coaches for Compositional Robot Skills
Sep 30, 2026Jiajun Liu, Yifan Chen, Yichao Liu +7Robot Policy Learning Robot Policy Adaptation
Renmin University of China · Tsinghua University · Shanghai Qizhi Institute +2
Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering
Jun 28, 2026Hao Wang, Jiuzhou Lei, Dayou Li +5Robot Policy Learning Robot Policy Adaptation
Texas A&M University · University of Wisconsin · Northwestern University +1
Position: Good Embodied Reward Models Need Bad Behavior Data
May 31, 2026Ran Tian, Yilin Wu, Andrea BajcsyReward Modeling Preference Alignment
UC Berkeley · NVIDIA Research · Carnegie Mellon University

Sep 30, 2026J/K move · Enter open · S save

RoboCoach: World Models as Active Coaches for Compositional Robot Skills

Jiajun Liu, Yifan Chen, Yichao Liu, Jiayi Zhang+6

Renmin University of China · Tsinghua University · Shanghai Qizhi Institute+2

Robot Policy Learning Robot Policy Adaptation Long-Horizon Robotic Manipulation Robot Skill Learning World Models for Robotics Imitation Learning

Open paper AskPDF arXiv

Long-horizon robot manipulation reuses skills across many task compositions, but improving these compositions with additional end-to-end demonstrations is costly. A practical self-improving system must decide both what to teach next and where to apply that supervision. We present ROBOCOACH, a world-model-guided coaching framework that uses imagined failures to guide demonstration requests and expert updates. Its Route-Imagine-Diagnose-Improve (RIDI) loop executes reusable skill experts inside COACHWORLD, our shared action-conditioned world model, and uses a progress judge to record the first subtask that fails to complete. Aggregated records select which subtask demonstrations to acquire and which expert adapters to update. Across two simulation suites and two real-robot platforms, imagined and deployed success correlate over 22 task-policy pairs (rho = 0.840). Controlled comparisons show that our coaching method outperforms matched baselines under matched data budgets and update schedules. With only 150 additional subtask demonstrations, success rises from 13.3% to 75.0% on Franka and from 40.0% to 83.8% on AgileX. The coached experts also transfer to four held-out compositions, achieving an average success of 35.0%, compared with 0% for a shared-policy baseline updated with uniformly acquired demonstrations. Together, these results show that world models can serve as active coaches, turning imagined failures into targeted supervision for modular policy improvement. Project Page: https://robocoach-ai.github.io/

Abstract

Explore similar work

RoboCoach: World Models as Active Coaches for Compositional Robot Skills

Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering

Position: Good Embodied Reward Models Need Bad Behavior Data