Modern robot learning systems feed on high-quality demonstration data, and their performance hinges on coverage. Without good coverage, small errors compound and the policy can quickly go into out-of-distribution states from which it cannot recover. Yet there is no principled way of collecting high-coverage data, and data collection often relies on heuristic instructions to human operators. Purely online reinforcement learning (RL) explores at the cost of safety and sample efficiency, while DAgger-style interventions require the policy to fail by chance before a human can correct it. In this paper, we ask instead whether we can modify how the robot behaves during data collection to collect more useful data. We propose Data Inoculation (DI), a shared-control framework in which a human provide closed-loop teleoperation commands while a residual policy perturbs these commands. The residual policy is trained online, on the growing dataset, with the same reward-maximizing objective later used for policy extraction, so its perturbations steer data collection toward the deviations a policy trained on the data would make. These perturbations expose the robot to a wider range of states, while the human's continuous corrections provide recovery demonstrations, "inoculating" the dataset against the learner's future mistakes while the human remains in control. On three challenging real-world robotic tasks, DI consistently improves or matches the success rate of policies trained with both behavioral cloning and offline RL, showing DI as an effective data collection strategy for improving downstream policy performance.
Figures & tables
Figure 1: Data inoculation (DI). Left : During data collection, we train a residual inoculative policy with online RL to perturb the human’s closed-loop teleoperation commands. The perturbations steer data collection towards the mistakes the policy trained on the data would make, and elicit human’s corrective feedback, “inoculating” the dataset against such mistakes. Right : We test DI on three challenging real-world robotic tasks that require precise manipulation, and DI consistently improve success rate of policies trained with behavioral cloning and offline RL.
L(ϕ)=(Qϕ(o,a~)−r−γQϕˉ(o′,a′+αΔ′))2,
Algorithm 1 Data Inoculation
Figure 2: Task success rates. The results for {t-cube/cable/keycap} are averaged over {9/5/5} seeds with 20 evaluation trials each. Error bars indicate the standard errors. Full table in Tables 1 , 3 , 2 and 4 .
Figure 3: Success rate over the course of data collection. DI starts at a lower success rate when the residual policy is untrained, but eventually reaches a higher success rate than clean and random collection.
Figure 4: Amount of human corrective demonstrations over different phases . DI elicits the most corrections at the most critical phase of the t-cube task.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Three real-robot tasks. Top: t-cube . The robot needs to move two colored cubes and carefully balance them on top of the yellow cube. Middle: cable . The robot needs to feed a USB-C cable through a desktop machine case port hole that is barely bigger than the cable itself. Bottom: keycap . The robot needs to use a plastic tool to extract a transparent keycap from a keyboard.
cable
t-cube
keycap
BC
RL
BC
RL
BC
RL
Clean
0.37±0.06
0.35±0.07
0.38±0.04
0.42±0.06
0.63±0.09
0.81±0.06
Random
0.48±0.09
0.56±0.04
0.40±0.05
0.42±0.03
0.69±0.06
0.67±0.09
Ours
0.66±0.04
0.67±0.06
0.50±0.03
0.59±0.02
0.72±0.05
0.84±0.06
Appendix
Table 1: Task success rates (mean ± standard error). The results for {t-cube/cable/keycap} are averaged over {9/5/5} seeds with 20 evaluation trials each. Best result per column in bold.
Clean
Random
Ours
Seed
BC
RL
BC
RL
BC
RL
0
04/20
02/20
06/20
09/20
14/20
14/20
1
11/20
09/20
13/20
13/20
12/20
17/20
2
07/20
07/20
07/20
12/20
12/20
10/20
3
09/20
09/20
07/20
10/20
12/20
13/20
4
06/20
08/20
15/20
12/20
16/20
13/20
Appendix
Table 2: Per-seed successes on cable (successes out of 20 trials per entry). The last row reports the mean success rate ± standard error over the five seeds.
Clean
Random
Ours
Seed
BC
RL
BC
RL
BC
RL
0
09/20
10/20
04/20
09/20
09/20
12/20
1
03/20
00/20
08/20
07/20
08/20
09/20
2
10/20
12/20
11/20
12/20
11/20
12/20
3
11/20
09/20
06/20
07/20
11/20
12/20
4
08/20
09/20
07/20
07/20
10/20
11/20
Appendix
Table 3: Per-seed successes on t-cube (successes out of 20 trials per entry). The last row reports the mean success rate ± standard error over the nine seeds.
Clean
Random
Ours
Seed
BC
RL
BC
RL
BC
RL
0
14/20
19/20
17/20
19/20
17/20
20/20
1
14/20
15/20
15/20
13/20
17/20
18/20
2
06/20
13/20
15/20
15/20
13/20
17/20
3
13/20
15/20
11/20
09/20
12/20
16/20
4
16/20
19/20
11/20
11/20
13/20
13/20
Appendix
Table 4: Per-seed successes on keycap (successes out of 20 trials per entry). The last row reports the mean success rate ± standard error over the five seeds.
cable
t-cube
keycap
Clean
252/260 (96.9%)
432/749 (57.7%)
113/118 (95.8%)
Random
233/244 (95.5%)
311/686 (45.3%)
122/123 (99.2%)
Ours
178/189 (94.2%)
361/663 (54.4%)
110/117 (94.0%)
Appendix
Table 5: Success rates during data collection, computed from the recorded episodes. Each entry reports successful episodes over all collected episodes and the corresponding percentage.
Learning from demonstrations is effective for robotic manipulation, but collecting sufficient task-specific data remains a major bottleneck. Under distribution shift, small errors compound, performance degrades, and expert time is often spent on redundant, low-value corrections instead of the few critical failure cases. We present VR-DAgger, a human-in-the-loop framework centered on an immersive VR application for dexterous teleoperation, demonstration collection, and selective policy correction. The VR client provides intuitive hand control with synchronized scene visualization, while a backend workstation runs simulation and learning, enabling autonomous rollouts without continuous operator oversight. We use Monte Carlo (MC) dropout to score uncertainty during Isaac Lab rollouts of a diffusion policy and select informative failure segments for correction. These segments are replayed in VR as clips, where the operator selectively labels and corrects the policy's behavior, concentrating supervision where uncertainty is highest without full-rollout monitoring or a separate intervention classifier. We evaluate on three dexterous manipulation tasks (Pan pick-and-place, Drawer opening, Valve turning) with a 10-DoF XHand under standard and challenging initial configurations. Active labeling consistently improves over behavioral cloning across all tasks, with gains of up to 23 percentage points. Compared to unguided human-in-the-loop inspection, VR-DAgger reduces per-sample collection time by approximately 40% by focusing review on selected segments rather than full rollouts.
René Zurbrügg, Tifanny Portela, Arjun Bhardwaj +3
The Authors are with the Robotics Systems Lab ETH Zürich.
Scaling robot learning is bottlenecked not only by data volume, but also by whether the data targets the policy's failures. Existing robot-free handheld interfaces enable portable in-the-wild data collection, yet remain open-loop: users record demonstrations without feedback from the current policy. Interactive methods such as DAgger provide such feedback by collecting corrective on-policy data, but require deploying imperfect policies on physical robots, making them costly, unsafe, and difficult to scale. We present RoboPocket, a smartphone-based system for robot-free policy iteration. By combining on-device sensing, display, and communication with cloud-side policy inference, RoboPocket provides real-time collection guidance and visualizes the policy's predicted trajectory in augmented reality as AR Visual Foresight. This allows users to identify policy weaknesses and collect targeted corrective data without deploying a physical robot. An asynchronous finetuning pipeline continuously incorporates new corrections and updates the policy within minutes, realizing a DAgger-inspired robot-free correction loop. Across diverse real-world manipulation tasks, we validate RoboPocket as a scalable data engine and show that online corrective data collection improves sample efficiency by up to 2× over scaling offline demonstrations. Project page and videos: https://robo-pocket.github.io.
Junjie Fang, Wendi Chen, Han Xue +7
Shanghai Jiao Tong University · Noematrix Ltd. · Shanghai Innovation Institute
Learning from human demonstrations is a reliable way to teach robots new tasks, but the gains from each additional demonstration shrink as the policy improves. Continued improvement can instead come from supervised deployment, where an operator places the objects and intervenes when the policy fails. We ask how to maximize improvement from a fixed budget of supervised episodes on high-precision manipulation tasks with wide ranges of object placements. We observe that failures can concentrate in a small subset of initial states, so uniform collection spends much of the operator's time on states the policy already handles. Mulligan makes the initial-state distribution a decision, starting each round's episodes at observed failures and untried states. To further improve data efficiency, we augment interactive imitation learning with a value function trained on all data, including failures that imitation discards. Across three real-world tasks evaluated on 2,550 held-out, blinded episodes and two simulated tasks, Mulligan outperforms uniform initial-state sampling at matched collection budgets, and combined with value-based action selection, HiL-IDQL+Mulligan, improves final real-task success by 10-34 percentage points. With operator interventions, the human-robot team completes 98% of collection episodes, remaining productive while the policy learns. Videos, code, and data are available at https://mulligan.page/.