Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning

Lars Ankile; Perry Dong; Rohan Bhowmik; Aneesh Muppidi; David D. Yuan; Shuran Song; Chelsea Finn

cs.ROOct 5, 2026

Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning

Authors: Lars Ankile, Perry Dong, Rohan Bhowmik, Aneesh Muppidi, David D. Yuan, Shuran Song, Chelsea Finn

Organizations: Stanford University

Abstract

Learning from human demonstrations is a reliable way to teach robots new tasks, but the gains from each additional demonstration shrink as the policy improves. Continued improvement can instead come from supervised deployment, where an operator places the objects and intervenes when the policy fails. We ask how to maximize improvement from a fixed budget of supervised episodes on high-precision manipulation tasks with wide ranges of object placements. We observe that failures can concentrate in a small subset of initial states, so uniform collection spends much of the operator's time on states the policy already handles. Mulligan makes the initial-state distribution a decision, starting each round's episodes at observed failures and untried states. To further improve data efficiency, we augment interactive imitation learning with a value function trained on all data, including failures that imitation discards. Across three real-world tasks evaluated on 2,550 held-out, blinded episodes and two simulated tasks, Mulligan outperforms uniform initial-state sampling at matched collection budgets, and combined with value-based action selection, HiL-IDQL+Mulligan, improves final real-task success by 10-34 percentage points. With operator interventions, the human-robot team completes 98% of collection episodes, remaining productive while the policy learns. Videos, code, and data are available at https://mulligan.page/.

Figures & tables

Figure 1: Diverse initial states: We study learning in deployment with wide initial-state distributions. Concentrated failures: Failures are not spread evenly across the initial-state space. Targeted collection: Retry observed failures (red) and fill the rest with the least-explored states (blue), to spend operator time where needed.

Figure 2: RLPD [ 18 , 19 ] reaches 84% on Square-Narrow but 0% on Square-Broad ; HiL-SERL [ 20 ] , with interventions, ends at 94% and 2%.

Panel of Figure 3 — Figure 3: The three real-world tasks. Each row is the median-length successful held-out episode of the final-round HiL-IDQL+ Mulligan policy from the side camera, with frames at the rungs of each task’s stage ladder ( Sec. C.3 ), not at even intervals. The median round-0 demonstration takes $12.7$ , $10.1$ , and $18.6$ s. In (b), the policy carries the nut above the camera’s field of view between frames.

Panel of Figure 4 — Figure 4: Left: the Thread Nut reset-helper card from which the operator places each episode’s objects. The nut center is sampled uniformly over the blue range at any yaw, and the peg is snapped to the orange grid (all three tasks in Sec. E.4 ). Middle and right: side-camera overlays of boundary and interior reset poses for Thread Nut and Insert Marker , with each object’s envelope, its center range grown by its reach, outlined and labeled. The Route Cable overlay is in Mulligan : Performance-Guided Data Collection for Efficient On-Robot Learning , left.

Figure 5: Held-out performance over DAgger rounds, real world (top) and simulation (bottom). Real-world panels show full-task success on Insert Marker and Thread Nut and clip success rate on Route Cable , the fraction of two clips seated (per-method full-task success in Sec. B.1 ). Real-world whiskers are

\pm 1

SE over

50

episodes, or

100

for the pooled Insert Marker R2 HiL-IDQL+ Mulligan point and both Thread Nut R3 points ( App. D ). Square-Narrow failures (

100-\text{SR}

) use a log scale. Dashed lines mark each campaign’s static round-0 policy. For real-world HiL-IDQL+ Mulligan , teal points use the actor alone and purple points add critic reranking.

Figure 6: Completed tasks per robot minute, counting time on failed episodes. A completed Route Cable task requires both clips seated. Colors follow Fig. 5 , with

\pm 1

bootstrap SE whiskers.

Figure 7: Component ablations in simulation (left two) and the real world (right two). State sampling and actor data on Square-Narrow round 1, with five-seed means, seed dots, and

95\%

t

-intervals ( Sec. B.5 ). Counterfactual demos on Thread Nut and Insert Marker through the last measured no-CF round, with Wilson

\pm 1

SE whiskers. R0 is the shared source actor.

Appendix figures & tables54 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Per-arm full-success results for the policies in Fig. 5 , with HG-DAgger (grey), HG-DAgger+ Mulligan (teal), and HiL-IDQL+ Mulligan with critic reranking (purple) as separate series. The top row gives the per-round delta of the policy deployed in the Mulligan campaign over the uniform-sampling HG-DAgger baseline in percentage points. Intervals are Wilson-score one-standard-error (

z{=}1

). The Route Cable panel shows full-episode success for rounds R0–R5, scored on one shared block of

50

initial states.

Task	Arm	Success	Contrast	$\Delta$ [CI], $p$
Insert Marker R5	HG-DAgger	$21/50$ ( $42\%$ )	—	—
	HG-DAgger+ Mulligan	$32/50$ ( $64\%$ )	vs. base	$+22$ pp $[+4,+40]$ , $\mathbf{p=0.0347}$
	HiL-IDQL+ Mulligan	$38/50$ ( $76\%$ )	vs. base	$+34$ pp $[+16,+52]$ , $\mathbf{p=0.0015}$
			vs. actor	$+12$ pp $[0,+26]$ , $p=0.146$
Thread Nut R5	HG-DAgger	$31/50$ ( $62\%$ )	—	—
	HG-DAgger+ Mulligan	$35/50$ ( $70\%$ )	vs. base	$+8$ pp $[-8,+24]$ , $p=0.454$

Appendix

Table 1: Final-round paired comparisons. We evaluated Insert Marker R5 in one contemporaneous interleaved session over a single Sobol-

50

block, and Thread Nut R5 in the final three-arm session on 50 fresh Sobol initial states (

n{=}50

per arm) with the selected FiLM critic. On Route Cable , R5 is the

600

-episode round, and we score both-clip success on all

50

initial states of the shared block. We report paired-bootstrap intervals and exact McNemar

p

-values. “vs. base” contrasts an arm with that task’s uniform HG-DAgger arm, and “vs. actor” with HG-DAgger+ Mulligan . Dashes mark contrasts we do not report for that round.

Task	Paired $n$	HG-DAgger	HG-DAgger+ Mulligan	Paired $\Delta$ [ $95\%$ CI]	Exact McNemar $p$
Insert Marker	$300$	$103$ ( $34.3\%$ )	$138$ ( $46.0\%$ )	$+11.7$ pp $[+5,+19]$	$\mathbf{0.0019}$
Thread Nut	$350$	$142$ ( $40.6\%$ )	$180$ ( $51.4\%$ )	$+10.9$ pp $[+5,+17]$	$\mathbf{0.0007}$
Route Cable	$300$	$33$ ( $11.0\%$ )	$51$ ( $17.0\%$ )	$+6.0$ pp $[+1,+11]$	$\mathbf{0.0356}$

Appendix

Table 2: Round-pooled paired comparison of uniform HG-DAgger against HG-DAgger+ Mulligan . We compute exact McNemar

p

-values on the summed discordant pairs and bootstrap the intervals stratified within block. All six Route Cable blocks use the same

50

initial states.

Round	Training episodes / arm	Full successes	Mean clip score
R0	$100$	$1/3/\text{---}$	$0.22/0.40/\text{---}$
R1	$200$	$4/3/\text{---}$	$0.58/0.44/\text{---}$
R2	$300$	$7/5/\text{---}$	$0.56/0.62/\text{---}$
R3	$400$	$8/11/10$	$0.70/0.82/0.88$
R4	$500$	$4/15/17$	$0.40/1.12/1.10$
R5	$600$	$9/14/17$	$0.74/0.98/1.00$

Appendix

Table 3: Route Cable results per round on the shared block of

50

initial states. A success count requires both clips seated, and progress is the mean

0

–

2

clip score. Triplets list HG-DAgger / HG-DAgger+ Mulligan / HiL-IDQL+ Mulligan , with a dash where we did not evaluate the reranked arm.

Square-Narrow
Contrast	R0	R1	R2	R3
HG-DAgger+ Mulligan vs. HG-DAgger	$+8.4$ $[+5.5,+11.3]$ $\mathbf{p<0.001}$	$+8.5$ $[+5.3,+11.8]$ $\mathbf{p<0.001}$	$+3.6$ $[+2.5,+4.7]$ $\mathbf{p<0.001}$	$+1.8$ $[+0.3,+3.3]$ $\mathbf{p=0.028}$
HiL-IDQL+ Mulligan vs. HiL-IDQL	$+8.2$ $[+5.2,+11.2]$ $\mathbf{p<0.001}$	$+7.8$ $[+4.6,+11.0]$ $\mathbf{p=0.001}$	$+3.0$ $[+2.0,+4.1]$ $\mathbf{p<0.001}$	$+1.5$ $[+0.5,+2.5]$ $\mathbf{p=0.007}$
HiL-IDQL+ Mulligan vs. HG-DAgger	$+5.9$ $[+3.1,+8.7]$ $\mathbf{p=0.001}$	$+8.7$ $[+5.4,+11.9]$ $\mathbf{p<0.001}$	$+3.5$ $[+2.3,+4.7]$ $\mathbf{p<0.001}$	$+2.5$ $[+1.0,+4.1]$ $\mathbf{p=0.006}$
Square-Broad
HG-DAgger+ Mulligan vs. HG-DAgger	$+3.3$ $[+1.0,+5.6]$ $\mathbf{p=0.011}$	$+8.5$ $[+6.4,+10.5]$ $\mathbf{p<0.001}$	$+11.1$ $[+7.9,+14.3]$ $\mathbf{p<0.001}$	$+7.6$ $[+2.7,+12.4]$ $\mathbf{p=0.009}$

Appendix

Table 4: Simulation per-round contrasts in percentage points of held-out success: Welch-

t

difference of five-seed means with

95\%

intervals and two-sided

p

-values. Actor-only arms use the headline evaluation of each seed’s final checkpoint. Critic-reranked arms (

N{=}32

) use the DIVL critics of Sec. J.4 , which are the

N{=}32

numbers plotted in the main paper.

Figure 11: Execution speed on the held-out evaluations: mean robot time per completed task unit (one standard error over completed units), where a unit is a full task completion on every task, requiring both clips on Route Cable . Route Cable uses all six rounds on its shared block of

50

initial states. Failed episodes are excluded from this duration metric.

Figure 12: Throughput on the held-out evaluations: completed task units per robot minute over all evaluation episode time (one bootstrap standard error over evaluation episodes). A task unit is a full episode on all three tasks, and Route Cable uses all six rounds on its shared block of

50

initial states. Unlike the main-text summary in Fig. 6 , this view shows HG-DAgger+ Mulligan and HiL-IDQL+ Mulligan together wherever both were evaluated. Failed and timed-out episodes spend robot time but produce no units.

Figure 13: Simulation execution speed on the fixed-grid held-out evaluations for Square-Narrow (left) and Square-Broad (right): mean successful-rollout duration in seconds (env control steps at 20 Hz), averaged over the five training seeds (two-sided Student-t 95% CIs, dots are per-seed means, y-axes are zoomed at the axis break). Series colors and markers follow Fig. 5 .

Figure 14: Throughput on the simulation fixed-grid held-out evaluations for Square-Narrow (left) and Square-Broad (right): successful rollouts per robot-minute over all evaluation episode time (two-sided Student-t 95% CIs over five seeds, dots are per-seed values, series and marker conventions as in Fig. 13 ). Failed episodes spend robot time but produce no completions, so this metric combines reliability and speed, as in Fig. 6 .

Collection strategy / actor data	Grid SR (%)	$\Delta$ [ $95\%$ CI]
Uniform, no-CF, HO (reference)	$85.73\pm 1.16$	—
Uniform, no-CF, $+$ auto successes	$82.37\pm 0.52$	$-3.36$ $[-6.53,-0.18]$
Sobol coverage, no-CF, HO	$89.59\pm 0.45$	$+3.86$ $[+0.69,+7.03]$
Sobol coverage, with-CF, HO	$91.49\pm 0.61$	$+5.76$ $[+2.55,+8.96]$
Performance-guided ( $\beta{=}0.5$ ), no-CF, HO	$92.69\pm 0.36$	$+6.96$ $[+3.78,+10.13]$
Performance-guided ( $\beta{=}0.5$ ), with-CF, HO	$93.52\pm 0.52$	$+7.80$ $[+4.62,+10.97]$

Appendix

Table 5: Sampling-strategy and actor-data ablation on Square-Narrow round 1. Entries are fixed-grid held-out success rates with

N{=}32

DIVL reranking, mean

\pm

standard error over five seeds, and

\Delta

compares each row with the uniform reference (Welch-t

95\%

intervals). “HO” is a human-only actor, and “

+

auto successes” adds successful autonomous rollouts.

Figure 15: Counterfactual-demonstration ablation on Insert Marker (left) and Thread Nut (right): held-out success rate over DAgger rounds for HG-DAgger+ Mulligan with and without CF demos, with Wilson-score one-standard-error intervals (

z{=}1

). Each panel ends at the last round with a no-CF evaluation ( Insert Marker R2, Thread Nut R4). R0 is the shared source actor, shown once. Most points use 50 episodes, and the with-CF Insert Marker R2 and Thread Nut R3 points pool 100 and 150 episodes. Removing counterfactuals lowers early-round success more on Insert Marker than on Thread Nut , and on both tasks the arms converge as corrections accumulate.

Figure 16: Per-stage decomposition on Thread Nut across rounds: grasp+transport (left), insertion/seat conditioned on grasp (middle), and overall success (right), with Wilson-score one-standard-error intervals (

z{=}1

). The grasp-stage gap narrows in later rounds, and the critic improves conditional insertion success at R3–R4. At R5, HG-DAgger+ Mulligan and HiL-IDQL+ Mulligan each have

36/50

stage successes. The R5 HG-DAgger+ Mulligan labels come from an earlier three-arm session and the HiL-IDQL+ Mulligan labels from a candidate-critic evaluation, as the final R5 evaluation has no stage labels.

Figure 17: Per-stage decomposition on Insert Marker across rounds: grasp+transport (left), insertion/seat conditioned on grasp (middle), and overall success (right), with Wilson-score one-standard-error intervals (

z{=}1

). The collection arm retains a grasp-stage advantage at R5, while reranking improves conditional insertion success. R5 uses the stage labels of the headline three-arm session.

Figure 18: Collection success on Insert Marker , Thread Nut , and Route Cable . The human-robot team completes

98\%

of fresh collection episodes, with no round below

90\%

(solid), while the collector policy’s preceding-round held-out success is lower (dashed). Whiskers are Wilson-score one-standard-error intervals (

z{=}1

). CF replay episodes are excluded, and Route Cable rounds are the

100

-episode increments of Tab. 6 .

Figure 19: Human-robot team success rate during DAgger collection, per round (Wilson-score one-standard-error intervals with

z{=}1

, y-axis truncated at 50%), on Insert Marker (left), Thread Nut (middle), and Route Cable (right,

100

-episode increments), for the uniform-sampling (grey, HG-DAgger) and Mulligan (teal, HiL-IDQL+ Mulligan ) collection campaigns. Over all rounds, the Mulligan campaign completes

98\%

of collection episodes with no round below

90\%

, and the uniform-sampling campaign completes

98\%

with no round below

91\%

Round	Arm	Fresh / CF	Collection success	Human share
R1	HG-DAgger	$100$ / $0$	$100/101$	$43.6\%$
	HG-DAgger+ Mulligan	$87$ / $13$	$100/101$	$44.2\%$
R2	HG-DAgger	$100$ / $0$	$100/104$	$44.0\%$
	HG-DAgger+ Mulligan	$76$ / $24$	$100/102$	$41.6\%$
R3	HG-DAgger	$100$ / $0$	$100/101$	$38.9\%$
	HG-DAgger+ Mulligan	$89$ / $11$	$100/102$	$38.0\%$

Appendix

Table 6: Route Cable collection on the paper’s

100

-episode increments, both HG-DAgger arms. “Fresh / CF” splits each increment’s credited

100

episodes into episodes that start under policy control and counterfactual replays that the operator demonstrates from the first frame. Collection success and human share count only the policy-initiated episodes. Success is successful fresh episodes over fresh attempts, and human share is the fraction of their valid frames under human control. The comparison is descriptive, since collector policies and sampled initial states change across increments. At R5, the HG-DAgger+ Mulligan operator stopped fresh collection at the

88

episodes the increment credits, rather than the

100

of earlier rounds, because the R5 dataset without counterfactuals would not be used. R5 collection success is out of

88

attempts.

Figure 20: Human collection burden over DAgger rounds on Insert Marker (top), Thread Nut (middle), and Route Cable (bottom,

100

-episode increments, each normalized by the fresh-episode-weighted held-out success of the policies that collected its source rounds), for the uniform-sampling (grey, HG-DAgger) and Mulligan (teal, HiL-IDQL+ Mulligan ) collection campaigns. Left: raw burden (share of frames under human control in fresh collection episodes). Right: the same share normalized by the collection policy’s failure rate

(1-\text{SR})

. On Route Cable , where complete success is still near the floor, the normalization is close to the identity.

Figure 21: Composition of each round’s training data for the uniform-sampling (top) and Mulligan (bottom) collection campaigns, as a share of valid frames. The number above each bar is the round’s count of human-controlled frames. Both campaigns collect the same number of episodes per round. The uniform-sampling campaign’s data train HG-DAgger, and the Mulligan campaign’s data train both HG-DAgger+ Mulligan and HiL-IDQL+ Mulligan .

		HG-DAgger			HiL-IDQL+ Mulligan
Task	Round	Demos	Corrections	Autonomous	Demos	Corrections	Autonomous
Insert Marker	R0	$19{,}382$	—	—	$19{,}096$	—	—
	R1	—	$2{,}496$	$9{,}168$	$2{,}380$	$2{,}157$	$6{,}966$
	R2	—	$3{,}057$	$10{,}871$	$2{,}920$	$1{,}883$	$7{,}507$
	R3	—	$3{,}208$	$9{,}726$	$2{,}759$	$1{,}873$	$7{,}898$
	R4	—	$2{,}017$	$11{,}580$	$1{,}010$	$1{,}667$	$10{,}741$

Appendix

Table 7: Training frames per round and source for the uniform-sampling and Mulligan collection campaigns ( Fig. 21 ). A dash marks a source that the round does not contain. Demonstrations plus corrections give the human frames, and the Mulligan totals equal

\mathcal{D}_{h}

and

\mathcal{D}_{\text{all}}

in Tab. 13 .

Task	Objective	Median demo
Real world
Insert Marker	Grasp a pen from the table and insert it into a holder whose mouth leaves $\sim 1$ mm of clearance.	$12.7$ s
Thread Nut	Grasp a square nut and thread it onto a fixed-diameter peg.	$10.1$ s
Route Cable	Seat a rope into two clips in succession.	$18.6$ s
Simulation
Square-Narrow	Insert a square nut onto a fixed peg. This is the Robomimic Square task (MimicGen Square_D0 ).	$8.3$ s

Appendix

Table 8: The five tasks. The last column is the median length of the round-0 teleoperated demonstrations over all arms (

200

on Square-Narrow and

400

on Square-Broad ), and in the real world also the held-out round-0 block (

250

per task).

Task	DoF	Position range ( $x{\times}y$ , cm)	Yaw
Insert Marker	$5$	Pen $15.2{\times}30.5$ , holder $5.1{\times}10.2$	$360^{\circ}$
Thread Nut	$5$	Nut $25.4{\times}45.7$ , peg $5.1{\times}10.2$	$360^{\circ}$
Route Cable	$7$	Rope free end $40.6$ in $x$ only, each clip $40.6{\times}22.9$	clips $\pm 45^{\circ}$
Square-Narrow	$3$	Nut $0.5{\times}11.5$ , peg fixed	$360^{\circ}$
Square-Broad	$5$	Nut $23.0{\times}51.0$ , peg $40.0{\times}40.0$	$360^{\circ}$

Appendix

Table 9: Initial-state ranges. Position ranges are the

x{\times}y

extents, in centimeters, over which each object or target center is placed. Yaw is the in-plane rotation range of the movable object. Each Route Cable clip takes one of three discrete yaws,

0^{\circ}

\pm 45^{\circ}

from its nominal orientation.

Panel of Figure 22 — Figure 22: Typical and hardest initial states of Insert Marker and Thread Nut in round-0 teleoperation. (a)–(d) are the first frames from the side camera next to the robot base, which is on the right (operator blurred). Each hardest initial state is the first state of the task’s longest round-0 demonstration, with the object near the minimum of its $x$ range and pointing toward the robot. Each typical initial state comes from a median-length demonstration whose object points away from the robot. The typical Insert Marker initial state lies at nearly the same $x$ as the hardest one. (e) From the hardest Thread Nut initial state, the robot places a finger next to the handle with the gripper yawed $180^{\circ}$ from the grasp orientation, rotates the nut, and then grasps it, seen from the opposite side camera. The three longest round-0 demonstrations of each task all start with the object within $21^{\circ}$ of pointing toward the robot.

Task	Success	Headline metric	Cap
Insert Marker	Marker released and fully seated (S7)	success rate	$800$ ( $53$ s)
Thread Nut	Nut released and fully seated (S7)	success rate	$600$ ( $40$ s)
Route Cable	Both clips seated (S10)	clip success rate	$850$ ( $57$ s)
Square-Narrow	Nut on the peg (Robosuite success check)	success rate	$400$ ( $20$ s)
Square-Broad	Nut on the peg (Robosuite success check)	success rate	$400$ ( $20$ s)

Appendix

Table 10: Scoring and evaluation caps. Clip success rate is the mean Route Cable clip score (

0

–

2

) divided by two. Caps are in environment steps, at

15

Hz in the real world and

20

Hz in simulation.

Rung	Insert Marker	Thread Nut
S0	No useful approach toward the marker.	No useful approach toward the nut.
S1	Approach or grasp attempt without a usable grasp (missed close, or an end pinch that cannot be inserted).	Reach or grasp attempt without a grasp that can lift and carry the nut.
S2	Marker acquired with a usable grasp and lifted clear of the table.	Nut grasped and lifted clear of the table, even if later lost.
S3	Marker brought to the holder area; pressing the holder body without the tip engaging the hole stays S3.	Held nut reaches the peg region in an insertion-directed pose; hole not verified on the peg.
S4	Marker tip engages the hole opening but is not fully seated.	Nut’s hole visibly engages or aligns over the peg top; not seated at the base.
S5	Gripper released a partial insertion that the holder retains.	Gripper released a partial insertion; the nut hangs on the peg above the base.

Appendix

Table 11: Stage ladders of the two binary real-world tasks. The labeler reports the furthest rung reached. S7 is the success predicate. An episode whose jaws never close is capped at S1, and one whose jaws never reopen at S3. On Insert Marker , an episode that ends with the marker back on the table is capped at S2.

Rung	Route Cable
S0	No useful approach.
S1	Approach or contact without a usable grasp.
S2	A grasp sufficient to route.
S3–S5	Reach, align, and engage the first clip.
S6	One clip seated.
S7–S9	Reach, align, and engage the remaining clip.

Appendix

Table 12: Stage ladder of Route Cable . S10 is the both-clips success predicate.

Figure 23: Blinded, paired evaluation (schematic, shuffles and outcomes are illustrative). The operator places the objects at each initial state of the held-out Sobol block, and every compared policy runs from that placement under an anonymous label, with the labels reshuffled for every initial state. Outcomes are recorded under the labels only. After the session, the sealed assignment is opened, and the outcomes are regrouped by policy, one row per shared initial state. Success rates come from the column totals, and the paired tests compare two policies on the initial states where their outcomes differ (shaded rows).

Figure 24: One supervised deployment round as a trajectory funnel. Each rollout starts from a Sobol-sampled initial state and succeeds only if it passes a precision bottleneck such as the insertion. The operator can correct early , steering the policy back before it fails, or rescue it after it has left the funnel. When a rollout fails without intervention, we keep it as negative data for the critic, and the operator may record a counterfactual demonstration from its initial state. Most action chunks sampled at the bottleneck miss. Best-of-

N

reranking with the IQL critic executes one that passes.

Task	Rounds	$M$	$\|\mathcal{D}_{h}\|$ episodes	$\mathcal{D}_{h}$ frames	$\mathcal{D}_{\text{all}}$ frames	Critic set (episodes / frames)	Collected episodes
Insert Marker	R0–R5	$50$	$294$	$38{,}017$	$81{,}526$	$825$ / $262{,}164$	$824$
Thread Nut	R0–R5	$50$	$301$	$34{,}536$	$66{,}354$	$900$ / $209{,}953$	$824$
Route Cable	R0–R5	$100$	$585$	$112{,}292$	$215{,}475$	$1{,}200$ / $548{,}588$	$1{,}334$
Square-Narrow	R0–R3	$100$	$229$	$25{,}216$	$70{,}563$	$700$ / $130{,}983$	$800$
Square-Broad	R0–R3	$100$	$443$	$57{,}983$	$100{,}781$	$900$ / $208{,}219$	$1{,}000$

Appendix

Table 13: Collection budgets and final-round training-set sizes for the HG-DAgger+ Mulligan arm.

M

counts episodes per round per arm after round 0, which records

100

demonstrations per arm (

200

on Square-Broad ). “

\mathcal{D}_{h}

” is the actor’s training set (episodes with human frames, with the human-controlled frame count), “

\mathcal{D}_{\text{all}}

frames” counts all valid frames of the collected episodes, “critic set” is the final-round critic’s training set including rollouts of earlier policies, and “collected” counts every recorded round-0 and collection episode across both the uniform-sampling and Mulligan arms.

Task	R1	R2	R3	R4	R5
Insert Marker	$12$	$15$	$14$	$9$	$9$
Thread Nut	$17$	$16$	$15$	$15$	$9$
Route Cable	$13$	$24$	$11$	$10$	$12$

Appendix

Table 14: Counterfactual demonstrations per collection round in the HG-DAgger+ Mulligan arm, out of

50

episodes on Insert Marker and Thread Nut and

100

on Route Cable .

Panel of Figure 25 — Figure 25: Per-episode reset-helper schematics for placing objects at each sampled initial state. Each card shows the allowed object-center grid, the chosen locations, and the sampled object poses (position and yaw) in table coordinates. Left: Insert Marker (pen + holder). Middle: Thread Nut (nut + peg). Right: Route Cable (cable + two clips).

Figure 26: Success rate by initial state on Square-Narrow (

80

lattice initial states over the full reset range,

50

rollouts each, sorted) for a round-2 policy trained with coverage-only (Sobol) sampling, seed 1 of three. Overall success is

85.95\%

(dashed line), and the hardest

25\%

of the states account for

94.8\%

of failures. We recovered the per-state success counts exactly from the original evaluation plot, whose bars fall on the

2

-point grid that

50

rollouts allow.

Figure 27: Square-Narrow R0 initial-state coverage of uniform HG-DAgger (left) and Sobol (right),

n=100

each, over nut

y

and yaw. Bottom: centered discrepancy (distinct from the star discrepancy defined above), minimum Euclidean nearest-neighbor distance on the normalized unit square, and the mean of the two marginal count standard deviations over ten equal-width bins.

Figure 28: Performance-guided initial-state design across rounds (schematic, points are scrambled-Sobol draws). A collection round covers the joint initial-state space with a Sobol set. Deploying the trained policy reveals a hard region (red). The next round’s collection set promotes the observed failures verbatim and spends the remaining budget on the coverage fill, optionally hardness-weighted. During collection, initial states where the policy fails again can receive a counterfactual demonstration. Over rounds, the hard set shrinks.

Task	Fit	Runs	Median h	Range h	Sum GPU-h
Square-Narrow	Actor + scalar critic	80	1.90	1.33–5.17	166.9
Square-Narrow	DIVL critic	80	0.65	0.53–1.17	56.9
Square-Broad	Actor + scalar critic	90	3.01	1.74–6.53	298.8
Square-Broad	DIVL critic	90	0.97	0.83–1.90	91.3
Insert Marker	Actor	12	3.16	1.82–7.85	46.3
Insert Marker	Critic	4	5.29	1.61–10.30	22.5

Appendix

Table 15: Logged runtime of the training runs behind the reported results, in hours on one GPU. Simulation actors and their scalar critics were trained jointly.

Hyperparameter	Simulation	Real world
Architecture
Backbone	1-D conditional U-Net over the action chunk
U-Net channel widths	$[512,1024]$
Kernel size / group-norm groups	$5$ / $8$
Diffusion-step conditioning	$128$ -D embedding, FiLM scale modulation
Observation encoder	none (state vector)	ResNet18, one per camera

Appendix

Table 16: Diffusion-policy actor. Simulation uses five replicate runs per arm, and the real world one run per arm per round. We fixed the training step count before seeing any result and deployed the final checkpoint in both domains.

Hyperparameter	Simulation	Real world
Objective
Value head	categorical (DIVL)	categorical (DIVL)
Expectile / base quantile	$\tau_{\text{base}}=0.7$
Discount $\gamma$	$0.99$	$0.99$ ( Thread Nut ), $0.997$ ( Insert Marker , Route Cable )
Target-network rate $\rho$	$0.005$ (Polyak)
$Q$ -ensemble	$2$ networks, min aggregation for both the TD label and reranking

Appendix

Table 17: Critic objective and reward. Both domains fit a categorical value distribution, with a quantile level that adapts to the predicted entropy. Simulation entries are for the reported human-in-the-loop cells and real-world entries for the deployed round-5 critics, with the task named wherever the three tasks differ.

Hyperparameter	Simulation	Real world
Network and optimization
$Q$ and $V$ trunk	MLP $[1024,1024,1024]$ with layer norm
$Q$ input	observation features concatenated with the flattened $6$ -step chunk
Visual features	—	the deployed actor’s encoder, frozen; features precomputed over $10$ augmented views per frame
Proprioception dropout	$0$	$0.5$
Extra critic head	none	state-conditioned FiLM rescaling of the critic’s action input ( Sec. G.3 ), used in the round-5 Insert Marker and Thread Nut critics and in all Route Cable critics

Appendix

Table 18: Critic network, optimization, and distributional value head. Real-world values are those of the deployed round-5 critics. Where the three tasks differ, the task is named.

Setting	Value
Candidates $N$ , simulation	$32$
Candidates $N$ , Insert Marker	$16$ at R2–R4, $32$ at R5
Candidates $N$ , Thread Nut	$16$ at R3–R4, $32$ at R5
Candidates $N$ , Route Cable	$32$ at R3–R5
Scored quantity	$Q(s,a_{t:t+6})$ , min over the $2$ -network ensemble
Selection rule	argmax

Appendix

Table 19: Reranking at deployment. Real-world rows list the reranked rounds, and actor-only evaluations use

N=1

. Sec. J.6 discusses the choice of

N

Setting	Simulation	Real world
Sobol draw
Sampled dimensions	$2$ ( Square-Narrow : nut $y$ , yaw), $5$ ( Square-Broad )	$5$ ( Insert Marker , Thread Nut ), $7$ ( Route Cable )
Remaining dimensions	Square-Narrow nut $x$ : drawn independently uniform	—
Scrambling	linear matrix scrambling with a digital shift (SciPy), always on
Pool size	$1024$ ( Square-Narrow ), $2048$ before rejection ( Square-Broad )	$32768$ ( Insert Marker , Thread Nut ), $4096$ ( Route Cable )
Rejection	Square-Broad : nut–peg center distance above $13.3$ cm	Route Cable : clip separation at least $5.1$ cm

Appendix

Table 20: Initial-state sampler. All budgets are per arm per round. In simulation, “uncapped” promotion promotes every failure among the round’s

100

diagnostic rollouts (the first

100

200

at Square-Broad R1), which came to

1

–

21

promotions on Square-Narrow and

14

–

51

on Square-Broad .

Setting	Simulation	Real world
Platform	Robosuite with MimicGen assets	Franka Panda on DROID
Control rate	$20$ Hz	$15$ Hz
Controller	operational-space pose, delta, base frame	Cartesian velocity
Per-step action limits	$5$ cm, $0.5$ rad	—
Cameras	none; state observations	ZED stereo, HD720 captured at $60$ fps
Teleoperation	SpaceMouse, with operator-triggered interventions

Appendix

Table 21: Environment, control loop, and improvement loop.

Cohort	DP $+$ IQL	RECAP	$\Delta$ (pp)	Paired $95\%$ CI
Square-Narrow R2	$95.39\pm 0.30$	$87.02\pm 0.45$	$-8.37$	$[-9.43,-7.31]$
Square-Narrow R3	$95.68\pm 0.31$	$90.03\pm 0.42$	$-5.65$	$[-6.64,-4.66]$
Square-Broad R3	$86.03\pm 0.81$	$65.36\pm 0.67$	$-20.67$	$[-23.75,-17.59]$

Appendix

Table 22: RECAP vs. the matched HiL-IDQL learner on identical data. Mean

\pm

standard error over five seeds. Differences are computed per seed, with two-sided Student-t

95\%

confidence intervals on four degrees of freedom. Each seed is scored at its final checkpoint on the fixed held-out grid (

8{,}000

episodes on Square-Narrow ,

30{,}000

on Square-Broad ).

Figure 29: HiL-SERL eval success per session (one seed each) against RLPD (five-seed mean, shaded

\pm

standard error) on Square-Narrow (left) and Square-Broad (right). Operator forks branch from the operator-free run at

150

k and

200

k. The operator-buffer arm continues the sparse-burst session from

130

k with no operator.

Square-Narrow
Operator protocol	Env steps	Epi- sodes	Operator hours	Human frames	Takeover episodes	Training success	Policy-only successes	Final eval	Peak eval
Step 0, dense	$0$ – $130$ k	$558$	$6.0$	$47.5\%$	$80\%$	$0.37$	$0/112$	$0.00$	$0.02$
Step 0, sparse bursts	$0$ – $130$ k	$781$	$3.3$	$22.9\%$	$54\%$	$0.13$	$0/363$	$0.00$	$0.08$
Step 0, long bouts	$0$ – $90$ k	$471$	$2.7$	$51.3\%$	$66\%$	$0.24$	$0/162$	$0.00$	$0.00$
Buffer, then none	$130$ – $300$ k	$674$	–	$0.0\%$	$0\%$	$0.55$	$374/674$	$0.78$	$0.86$
After takeoff	$150$ – $300$ k	$1{,}310$	$3.1$	$15.6\%$	$35\%$	$0.93$	$819/856$	$0.94$	$0.94$

Appendix

Table 23: HiL-SERL operator cost per session, one seed each. A fork counts only its episodes after the fork. Steps: the environment-step range run under the protocol. Operator hours: summed wall time of the episodes. Human frames: share of environment steps under operator control. Training success: share of training episodes that succeeded, with operator help where given. Policy-only: successes among training episodes without a takeover. Final and peak eval: at the last checkpoint and the maximum after the fork. Total rows sum the sessions with an operator. Their human frames and takeover episodes are pooled shares.

Figure 30: Offline diagnostics across training budgets for the round-5 Insert Marker critic recipe, using the same

50

excluded rollouts (

23

successes,

27

failures). (a) How well the maximum

Q

over

32

sampled actions predicts the recorded episode outcome, as AUROC on an axis zoomed to

[0.25,0.85]

. Shading gives pointwise

95\%

stratified episode-bootstrap intervals (

5{,}000

resamples), and the dashed line marks chance. (b) Mean initial value on the

23

successful episodes

\pm

one standard error across episodes. We took the

125

k and

150

k checkpoints from the original run, and the

200

k–

475

k checkpoints from a longer run with the same recipe and seed.

Figure 31: FQE backtest on

16

historical Insert Marker policies, shown with (a) a pooled fit and (b) separate fits by policy family. Dashed lines are descriptive least-squares fits, and

\rho

denotes Spearman correlation. The value axes are zoomed to

[0.14,0.425]

. Markers average three evaluator seeds, small dots show individual seeds, and vertical whiskers give one standard error across seeds. Horizontal whiskers are Wilson

95\%

intervals from recorded robot counts (

50

attempts per policy, except

100

for round-2 DP), without adjustment for repeated initial states. Evaluators use

\gamma=0.995

, timeout bootstrapping, and their final

40

k checkpoint. FQE scores every policy on the same

150

round-5 reset images (

50

initial states), but each robot success rate comes from that policy’s own historical round. The round-5 BoN point uses the original

31/50

critic, not the critic selected later. Because the policies are related and the two axes use different initial-state distributions, the pooled fit is hard to interpret.

Task	All-data $N{=}32$	Human-only $N{=}32$	$\Delta$ at $N{=}32$	$\Delta$ at $N{=}1$ (same actor)	Rerank gain all-data	Rerank gain human-only
Square-Narrow	$96.25\%$	$95.15\%$	$-1.10$ $[-1.91,-0.29]$	$+0.05$ $[-1.88,+1.98]$	$+0.40$ $[-0.89,+1.69]$	$-0.75$ $[-1.83,+0.33]$
Square-Broad	$85.80\%$	$85.00\%$	$-0.80$ $[-2.96,+1.36]$	$+0.05$ $[-1.38,+1.48]$	$-0.40$ $[-1.93,+1.13]$	$-1.25$ $[-2.80,+0.30]$

Appendix

Table 24: Human-only versus all-data critic behind the same frozen R3 actor, five paired seeds per task, final-step success on

400

fixed Sobol initial states.

\Delta

subtracts all-data from human-only success in percentage points, and the rerank gain subtracts

N{=}1

from

N{=}32

success within an arm. We report means over seeds with two-sided Student-

t

95\%

intervals over the five seed differences. The critic plays no role at

N{=}1

Condition	Seeds	Final SR (%)
Baseline, reward $[0,1]$ , clipped targets	5	$91.60\pm 0.75$
Shifted reward $[-1,0]$ , no target clipping	5	$91.40\pm 1.91$
Intervention frames treated as terminal	5	$92.00\pm 1.70$
Actor update cadence $4{:}1{:}1$	5	$91.60\pm 0.40$
Actor update cadence $2{:}1{:}1$	5	$89.60\pm 1.57$
Flat (uniform) critic sampling	3	$90.00\pm 0.58$

Appendix

Table 25: Critic-objective variations on the Square-Narrow task, final-step (

300

k) success rate, mean

\pm

standard error over training seeds, on a superseded ten-round DAgger campaign rather than the R0–R3 campaign of Tab. 13 . The reported critics use the unshifted reward and clipped targets of the baseline row ( Tab. 17 ). “QC” denotes the no-

V

-network variant that takes a best-of-

N

max over

Q

as the TD target.

Figure 32: Final-step (

300

k) success rates for all nine critic-objective conditions in Tab. 25 . Large dots and error bars show the mean and one standard error, and small dots show the completed seeds. The two QC conditions with four seeds exclude failed runs, and flat sampling has three seeds.

Task	Cells	Mean $\Delta$ (pp)	Cell range (pp)	Cells favoring DIVL / scalar
Square-Narrow	$16$	$+0.09$	$[-0.47,+0.55]$	$9$ / $7$
Square-Broad	$18$	$+0.00$	$[-0.32,+0.18]$	$11$ / $7$
Both tasks	$34$	$+0.04$	$[-0.47,+0.55]$	$20$ / $14$

Appendix

Table 26: Adaptive DIVL versus scalar IQL on the

34

human-in-the-loop simulation cells, five paired seeds per cell.

\Delta

subtracts scalar from DIVL success in percentage points and is averaged over seeds within a cell. The mean weights every cell equally, and the range and sign counts are over cell means.

Regime (scalar-IQL base SR)	scalar IQL	divl-fixed	divl-adaptive
Low SR ( $\sim 70\%$ )	$69.95\pm 0.74$	$+1.45$	$+4.05$
High SR ( $\sim 90\%$ )	$90.17\pm 0.73$	$-1.17$	$-0.83$
High SR ( $\sim 94\%$ )	$93.80\pm 0.65$	$+0.35$	$-0.35$

Appendix

Table 27: DIVL vs. scalar IQL,

\Delta

in percentage points against the scalar control within each regime, matched within a row but not across rows. The low-SR and

\sim

94\%

rows are round-

1

Square-Broad and Square-Narrow cells,

5

seeds, final-step

N{=}32

SR on

400

fixed-Sobol initial states. The

\sim

90\%

row is round

15

of a superseded Square-Broad campaign,

3

seeds,

N{=}128

SR on

200

episodes. Scalar controls show mean

\pm

SE. The distributional form alone ( divl-fixed ) gives no consistent gain, while the adaptive variant has a positive mean difference at low success and negative differences in both high-success regimes.

Critic sampler	scalar IQL (control)	divl-fixed	divl-adaptive
straddled	$93.80\pm 0.65$	$94.15$ ( $+0.35$ )	$93.45$ ( $-0.35$ )
balanced_outcome	$93.95\pm 0.70$	$92.60$ ( $-1.35$ )	$93.25$ ( $-0.70$ )
flat	$93.80\pm 0.50$	$93.55$ ( $-0.25$ )	$92.70$ ( $-1.10$ )

Appendix

Table 28: DIVL under three critic samplers in the

\sim

94\%

regime (

5

seeds each, final-step

N{=}32

SR on

400

fixed-Sobol initial states,

\Delta

against the scalar control within each sampler). straddled is success-biased (

50\%

human success /

50\%

DAgger), balanced_outcome is failure-biased (

50\%

success /

50\%

fail-or-intervened), and flat is uniform over the natural distribution.

Critic ensemble	$\|$ ensemble $\|$	Final $N{=}32$ SR (%)	$\Delta$
$2$ - $Q$ min	$2$	$95.70\pm 0.86$	—
REDQ-2	$10$	$93.50\pm 0.57$	$-2.20$
Single random critic	$10$	$95.60\pm 0.62$	$-0.10$

Appendix

Table 29: Critic ensemble size on Square-Broad at round

15

of a superseded DAgger campaign.

5

seeds each,

300

k steps, final-step

N{=}32

success rate (mean

\pm

standard error). “REDQ-2” takes a min over a random pair of critics resampled at every

Q

evaluation. “Single” draws one random critic per evaluation.

Method	Task	Varies	DoF	Position range	Yaw	Source
HiL-SERL [ 20 ]	IKEA side panel	arm	3	$8$ cm in $x$ , $y$	$1^{\circ}$	Tab. 6
HiL-SERL [ 20 ]	Cable clipping	object	3	$4$ cm in $x$ , $y$	$10^{\circ}$	Tab. 5
ConRFT [ 22 ]	Pick banana	object	2	$3$ cm in $x$ , $y$	none	Tab. V
ConRFT [ 22 ]	Insert wheel	object	2	$2$ cm in $x$ , $y$	none	Tab. XI
RL-100 [ 25 ]	Box folding	object	3	$75{\times}55$ cm	unstated ‡	Suppl.
RL-100 [ 25 ]	Orange juicing	object	3	$29{\times}20$ cm tray	unstated ‡	Suppl.

Appendix

Table 30: Initial-state randomization as reported by prior real-world RL systems, and ours. “Varies” distinguishes randomization of the object pose from randomization of the arm’s start pose with the object fixed, and marks the tasks in which the target the object must reach is also randomized. DoF counts the independent dimensions of the initial state that vary (three for an object’s

x

y

, and yaw). Where a paper reports a range for one element only, we take the remaining elements to be fixed. We quote position ranges as published. Orientation is in-plane (yaw). Table numbers under “Source” refer to each paper’s own numbering (appendix tables for HiL-SERL and ConRFT). ‡ RL-100 states that the blank’s and the half-orange’s rotations are randomized but gives no range.

Method	Task	Demos	Human corrections	Autonomous experience	Robot time	Source
HiL-SERL [ 20 ]	IKEA side panel	$20$	rate curve only †	$31{,}000$ / $36{,}000$ transitions ‡ (p1/p2)	$2$ / $1.75$ h	Tab. 1, 6
HiL-SERL [ 20 ]	Cable clipping	$20$	rate curve only †	$28{,}000$ transitions ‡	$1.25$ h	Tab. 1, 5
ConRFT [ 22 ]	Pick banana	$20$	rate curve only †	$80$ – $120$ episodes ∗	$45$ min	Tab. I, V
ConRFT [ 22 ]	Insert wheel	$30$	rate curve only †	$80$ – $120$ episodes ∗	$60$ min	Tab. I, XI
RL-100 [ 25 ]	Box folding	$200$ ( $3$ h)	none §	$608$ offline + $186$ online epi.	$15.3$ h	Tab. S3
RL-100 [ 25 ]	Orange juicing	$109$ ( $2$ h) ¶	none §	$791$ offline + $990$ online epi. ¶	$31.5$ h ¶	Tab. S3

Appendix

Table 31: Data and interaction budgets as reported by prior real-world RL systems, and ours. Rows match Tab. 30 task for task wherever the source paper reports per-task numbers. “Demos” are the offline human demonstrations that initialize learning, “Human corrections” are in-the-loop interventions, and “Autonomous experience” is the data the policy collected on the robot, in the paper’s own unit. “Robot time” is the wall-clock number each paper headlines. Its definition varies, with scripted motion, intended stops and onboard computation included in HiL-SERL and ConRFT, and computation, resets and intended stops in SERL. RL-100 reports data-collection time per stage, and the entry here sums its three stages (demonstration, iterative offline RL, online RL). Table numbers under “Source” refer to each paper’s own numbering. † Both human-in-the-loop systems report the intervention rate , the ratio of intervened to total timesteps, only as a training curve, for every task in HiL-SERL (Figs. 5 and 26) and for representative tasks in ConRFT (Figs. 3–4). Neither reports a count of interventions or a total intervened time for any task. ‡ HiL-SERL’s “total number of RL transitions” is the replay-buffer size at convergence and includes human-intervention timesteps, so it is not purely autonomous experience. The p1/p2 entries are the two side panels, which the paper trains and times separately. ∗ ConRFT states only that the policy collects “approximately 80 to 120 successful and failed trajectories” during the online stage, for all tasks together. § These systems have no in-the-loop corrections and train from demonstrations plus autonomous rollouts. ¶ Sum over the two Orange Juicing subtasks (Placing and Removal), which Tab. S3 reports separately. ∥ Our rows count the HG-DAgger+ Mulligan arm’s own training data at the final round ( Tab. 13 ), split into human-controlled frames (demonstrations, corrections, and counterfactual replays) and autonomous frames of the same episodes, at

15

Hz. For our rows, “robot time” adds up the recorded length of that arm’s collection episodes and of the held-out evaluations of the three headline arms. It counts full episodes, including any tail that a later outcome review trimmed, and leaves out resets and operator setup. On Route Cable , we measured session wall-clock time from save timestamps at about three times the recorded trajectory time.

Explore similar work

Sep 21, 2026cs.RO

Learning Beyond What Humans Can Demonstrate

Behavior cloning for robot manipulation relies on expert demonstrations. However, for tasks that require dynamic stability, precise contact timing, or dexterous coordination, human operators may find it hard or even impossible to collect data. We study this infeasible-demonstration regime and propose GLIDE: Guardrails for Learning from Infeasible Demonstrations Efficiently, a framework that infers task-specific failure modes and converts them into executable guardrails for data collection and policy deployment. Given a task description and the conditioning teleoperation code, GLIDE writes guardrails that use system states to filter teleoperation and policy commands, constrain failure-prone actions, and iteratively improve from trajectory feedback. Across three tasks, GLIDE discovers emergent guardrails that go beyond domain-expert hardcoded ones, improving data collection over naive VR teleoperation and domain-expert hardcoded guardrails. After refinement, GLIDE raises data-collection success from 0-10 percent to 70-90 percent across the three tasks. During policy execution, mixed-data guarded policies reach 70 percent, 60 percent, and 60 percent success on Tomato plate transfer, Marker handover and stand, and Wine serving tasks. These results show that GLIDE can support policy learning when direct demonstrations are infeasible. Project website: http://guardrail-policy.github.io/

Yuchen Song, Aditya Mittal, Unnat Jain

University of California, Irvine

Teleoperation Behavior Cloning Guardrail Latent Failure Patterns

Ask PDF

Jun 8, 2026cs.RO

ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration

Learning robot manipulation policies with deep neural networks from a single demonstration remains highly challenging, as even small deviations from the demonstrated trajectory can quickly compound into failure, while collecting substantial online interaction data is costly. We propose ReGIL, a retrieval-guided imitation learning framework that treats a single demonstration as an external memory. ReGIL repeatedly queries this static memory throughout training to simultaneously guide exploration, generate the regularization buffer, and construct rewards. Specifically, it computes rewards through local temporal alignment between the current trajectory and the retrieved segment, providing step-wise and informative feedback for policy improvement. We evaluate ReGIL on robotic manipulation tasks from the LIBERO and Meta-World benchmarks under the single demonstration setting. ReGIL outperforms prior baselines in both success rate and training efficiency. In real-robot experiments, using only one demonstration and less than one hour of online training, ReGIL achieves over 75% success rate across three manipulation tasks with randomness in both initial robot pose and target position. These results demonstrate that leveraging the single demonstration as reusable memory can provide more than static supervision for efficient robot learning. More details can be found on our website: https://regil2026.github.io/

Yuying Zhang, Francesco Verdoja, Wenyan Yang +1

School of Electrical Engineering Aalto University, Finland

Imitation Learning Scalable Robot Learning

Ask PDF

May 8, 2026cs.RO

How to Utilize Failure Demo Data?: Effective Data Selection for Imitation Learning Using Distribution Differences in Attention Mechanism

Imitation learning for robotic tasks has relied primarily on policies trained only on successful demonstrations, although failures are unavoidable during human data collection. Many existing approaches for exploiting failure data require additional data processing or iterative policy updates through autonomous rollouts, making it difficult to directly and stably utilize failure data accumulated during data collection. In this work, we propose a method that learns latent representations of success-failure discrepancies and incorporates them into the attention mechanism. During inference, an appropriate latent mode is selected from the initial observation to improve action stability. Furthermore, we introduce a post-training metric that quantifies the attention discrepancy between each failure sample and successful demonstrations to select failure data. Simulation results show that the proposed method improves task success rates when trained with failure data and that the proposed metric identifies failure samples that are beneficial for learning when combined with successful demonstrations. These results suggest that the proposed method can support more efficient use of collected demonstrations in robotic data collection pipelines.

Kana Miyamoto, Kanata Suzuki, Tetsuya Ogata

Faculty of Science and Engineering, Waseda University, Tokyo, Japan · Physical AI Laboratory, Fujitsu Limited, Kanagawa, Japan

Imitation Learning Robotic Data Acquisition Alignment-Aware Selection Lateral String Stability

Ask PDF