Robotic Data Inoculation

Qiyang Li; Zhiyuan Zhou; Charles Xu; Sergey Levine

cs.ROOct 28, 2026

Robotic Data Inoculation

Authors: Qiyang Li, Zhiyuan Zhou, Charles Xu, Sergey Levine

Organizations: UC Berkeley

Abstract

Modern robot learning systems feed on high-quality demonstration data, and their performance hinges on coverage. Without good coverage, small errors compound and the policy can quickly go into out-of-distribution states from which it cannot recover. Yet there is no principled way of collecting high-coverage data, and data collection often relies on heuristic instructions to human operators. Purely online reinforcement learning (RL) explores at the cost of safety and sample efficiency, while DAgger-style interventions require the policy to fail by chance before a human can correct it. In this paper, we ask instead whether we can modify how the robot behaves during data collection to collect more useful data. We propose Data Inoculation (DI), a shared-control framework in which a human provide closed-loop teleoperation commands while a residual policy perturbs these commands. The residual policy is trained online, on the growing dataset, with the same reward-maximizing objective later used for policy extraction, so its perturbations steer data collection toward the deviations a policy trained on the data would make. These perturbations expose the robot to a wider range of states, while the human's continuous corrections provide recovery demonstrations, "inoculating" the dataset against the learner's future mistakes while the human remains in control. On three challenging real-world robotic tasks, DI consistently improves or matches the success rate of policies trained with both behavioral cloning and offline RL, showing DI as an effective data collection strategy for improving downstream policy performance.

Figures & tables

Figure 1: Data inoculation (DI). Left : During data collection, we train a residual inoculative policy with online RL to perturb the human’s closed-loop teleoperation commands. The perturbations steer data collection towards the mistakes the policy trained on the data would make, and elicit human’s corrective feedback, “inoculating” the dataset against such mistakes. Right : We test DI on three challenging real-world robotic tasks that require precise manipulation, and DI consistently improve success rate of policies trained with behavioral cloning and offline RL.

\displaystyle L(\phi)=\big(Q_{\phi}(\mathbf{o},\tilde{\mathbf{a}})-r-\gamma Q_{\bar{\phi}}(\mathbf{o}^{\prime},\mathbf{a}^{\prime}+\alpha\Delta^{\prime})\big)^{2},

Algorithm 1 Data Inoculation

Figure 2: Task success rates. The results for {t-cube/cable/keycap} are averaged over {9/5/5} seeds with 20 evaluation trials each. Error bars indicate the standard errors. Full table in Tables 1 , 3 , 2 and 4 .

Figure 3: Success rate over the course of data collection. DI starts at a lower success rate when the residual policy is untrained, but eventually reaches a higher success rate than clean and random collection.

Figure 4: Amount of human corrective demonstrations over different phases . DI elicits the most corrections at the most critical phase of the t-cube task.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Three real-robot tasks. Top: t-cube . The robot needs to move two colored cubes and carefully balance them on top of the yellow cube. Middle: cable . The robot needs to feed a USB-C cable through a desktop machine case port hole that is barely bigger than the cable itself. Bottom: keycap . The robot needs to use a plastic tool to extract a transparent keycap from a keyboard.

	cable		t-cube		keycap
	BC	RL	BC	RL	BC	RL
Clean	$0.37\pm 0.06$	$0.35\pm 0.07$	$0.38\pm 0.04$	$0.42\pm 0.06$	$\mathbf{0.63\pm 0.09}$	$\mathbf{0.81\pm 0.06}$
Random	$0.48\pm 0.09$	$0.56\pm 0.04$	$0.40\pm 0.05$	$0.42\pm 0.03$	$\mathbf{0.69\pm 0.06}$	$0.67\pm 0.09$
Ours	$\mathbf{0.66\pm 0.04}$	$\mathbf{0.67\pm 0.06}$	$\mathbf{0.50\pm 0.03}$	$\mathbf{0.59\pm 0.02}$	$\mathbf{0.72\pm 0.05}$	$\mathbf{0.84\pm 0.06}$

Appendix

Table 1: Task success rates (mean

\pm

standard error). The results for {t-cube/cable/keycap} are averaged over {9/5/5} seeds with 20 evaluation trials each. Best result per column in bold.

	Clean		Random		Ours
Seed	BC	RL	BC	RL	BC	RL
0	$\hphantom{0}4/20$	$\hphantom{0}2/20$	$\hphantom{0}6/20$	$\hphantom{0}9/20$	$14/20$	$14/20$
1	$11/20$	$\hphantom{0}9/20$	$13/20$	$13/20$	$12/20$	$17/20$
2	$\hphantom{0}7/20$	$\hphantom{0}7/20$	$\hphantom{0}7/20$	$12/20$	$12/20$	$10/20$
3	$\hphantom{0}9/20$	$\hphantom{0}9/20$	$\hphantom{0}7/20$	$10/20$	$12/20$	$13/20$
4	$\hphantom{0}6/20$	$\hphantom{0}8/20$	$15/20$	$12/20$	$16/20$	$13/20$

Appendix

Table 2: Per-seed successes on cable (successes out of 20 trials per entry). The last row reports the mean success rate

\pm

standard error over the five seeds.

	Clean		Random		Ours
Seed	BC	RL	BC	RL	BC	RL
0	$\hphantom{0}9/20$	$10/20$	$\hphantom{0}4/20$	$\hphantom{0}9/20$	$\hphantom{0}9/20$	$12/20$
1	$\hphantom{0}3/20$	$\hphantom{0}0/20$	$\hphantom{0}8/20$	$\hphantom{0}7/20$	$\hphantom{0}8/20$	$\hphantom{0}9/20$
2	$10/20$	$12/20$	$11/20$	$12/20$	$11/20$	$12/20$
3	$11/20$	$\hphantom{0}9/20$	$\hphantom{0}6/20$	$\hphantom{0}7/20$	$11/20$	$12/20$
4	$\hphantom{0}8/20$	$\hphantom{0}9/20$	$\hphantom{0}7/20$	$\hphantom{0}7/20$	$10/20$	$11/20$

Appendix

Table 3: Per-seed successes on t-cube (successes out of 20 trials per entry). The last row reports the mean success rate

\pm

standard error over the nine seeds.

	Clean		Random		Ours
Seed	BC	RL	BC	RL	BC	RL
0	$14/20$	$19/20$	$17/20$	$19/20$	$17/20$	$20/20$
1	$14/20$	$15/20$	$15/20$	$13/20$	$17/20$	$18/20$
2	$\hphantom{0}6/20$	$13/20$	$15/20$	$15/20$	$13/20$	$17/20$
3	$13/20$	$15/20$	$11/20$	$\hphantom{0}9/20$	$12/20$	$16/20$
4	$16/20$	$19/20$	$11/20$	$11/20$	$13/20$	$13/20$

Appendix

Table 4: Per-seed successes on keycap (successes out of 20 trials per entry). The last row reports the mean success rate

\pm

standard error over the five seeds.

	cable	t-cube	keycap
Clean	$252/260$ (96.9%)	$432/749$ (57.7%)	$113/118$ (95.8%)
Random	$233/244$ (95.5%)	$311/686$ (45.3%)	$122/123$ (99.2%)
Ours	$178/189$ (94.2%)	$361/663$ (54.4%)	$110/117$ (94.0%)

Appendix

Table 5: Success rates during data collection, computed from the recorded episodes. Each entry reports successful episodes over all collected episodes and the corresponding percentage.

Parameter	Value
Batch size	$256$
Discount factor ( $\gamma$ )	$0.97$
Action chunk size ( $h$ )	$5$
Best-of-N sampling ( $N$ )	$4$
Optimizer	Adam
Learning rate	$3\times 10^{-4}$

Appendix

Table 6: Shared hyperparameters.

Tasks	Random Noise	Ours	Training Steps	Data Size
	$c$	$\beta$
t-cube	$0.05$	$10$	$1\times 10^{5}$	$10000$
cable	$0.05$	$0.03$	$0.5\times 10^{5}$	$4000$
keycap	$0.1$	$0.01$	$0.5\times 10^{5}$	$3000$

Appendix

Table 7: Task-specific hyperparameters.

Robotic Data Collection

Robotic data collection develops ways to gather, label, and validate robot and human-generated experience for learning, evaluation, and sensing.

See all 181 papers

VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction
May 26, 2026René Zurbrügg, Tifanny Portela, Arjun Bhardwaj +3Robotic Data Collection Robot Teleoperation
The Authors are with the Robotics Systems Lab ETH Zürich.
RoboPocket: Improve Robot Policies Instantly with Your Phone
Mar 5, 2026Junjie Fang, Wendi Chen, Han Xue +7Data-Efficient Robot Learning Robot Learning
Shanghai Jiao Tong University · Noematrix Ltd. · Shanghai Innovation Institute
Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning
Oct 5, 2026Lars Ankile, Perry Dong, Rohan Bhowmik +4Robotic Data Collection Robotic Manipulation
Stanford University

May 26, 2026J/K move · Enter open · S save

VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction

René Zurbrügg, Tifanny Portela, Arjun Bhardwaj, Aravind Elanjimattathil Vijayan+2

The Authors are with the Robotics Systems Lab ETH Zürich.

Robotic Data Collection Robot Teleoperation Robotic Manipulation Adaptive Sampling Robot Imitation Learning Dataset Aggregation

Open paper AskPDF arXiv

Learning from demonstrations is effective for robotic manipulation, but collecting sufficient task-specific data remains a major bottleneck. Under distribution shift, small errors compound, performance degrades, and expert time is often spent on redundant, low-value corrections instead of the few critical failure cases. We present VR-DAgger, a human-in-the-loop framework centered on an immersive VR application for dexterous teleoperation, demonstration collection, and selective policy correction. The VR client provides intuitive hand control with synchronized scene visualization, while a backend workstation runs simulation and learning, enabling autonomous rollouts without continuous operator oversight. We use Monte Carlo (MC) dropout to score uncertainty during Isaac Lab rollouts of a diffusion policy and select informative failure segments for correction. These segments are replayed in VR as clips, where the operator selectively labels and corrects the policy's behavior, concentrating supervision where uncertainty is highest without full-rollout monitoring or a separate intervention classifier. We evaluate on three dexterous manipulation tasks (Pan pick-and-place, Drawer opening, Valve turning) with a 10-DoF XHand under standard and challenging initial configurations. Active labeling consistently improves over behavioral cloning across all tasks, with gains of up to 23 percentage points. Compared to unguided human-in-the-loop inspection, VR-DAgger reduces per-sample collection time by approximately 40% by focusing review on selected segments rather than full rollouts.

Abstract

Robotic Data Collection

Explore similar work

VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction

RoboPocket: Improve Robot Policies Instantly with Your Phone

Mulligan: Performance-Guided Data Collection for Efficient On-Robot Learning