Learning robot policies from demonstrations is often limited by the cost of collecting diverse, high-quality expert data. While an appealing alternative is to expand a single successful demonstration into a diverse set of successful trajectories, existing augmentation methods typically rely on manually designed transformation rules or complex trajectory planning, requiring substantial task-specific engineering and human effort. We introduce OneDemo, a reinforcement learning framework that learns to edit a single successful demonstration for new task configurations. We formulate demonstration editing as a structured reinforcement learning problem, in which an editor learns, from sparse task-success rewards, both when to transition between object-centric trajectory segments and how to geometrically adapt each segment. The learned editor serves both as a task policy and as a reusable demonstration generator. Across 15 Meta-World tasks, OneDemo reaches a 92.7% success rate within 53.7k environment interactions on average. Across 9 DexMimicGen tasks, it generates successful demonstrations with a 98.4% average success rate, and BC-RNN policies trained on the generated data achieve 99.6% average success at matched dataset scale. These results show that learning to edit a single successful demonstration provides a structured and effective basis for both policy learning and scalable demonstration generation.
Figures & tables
Figure 1: OneDemo learns to edit a single demonstration to generate varying successful trajectories under new spatial configurations. Left: OneDemo adapts the source trajectory to generate successful augmentations. Right: Recurrent behavior cloning (BC-RNN) trained on demonstrations augmented by OneDemo improves success from 76.3% to 99.6% compared with DexMimicGen data.
Figure 2: Overview of OneDemo . Semantic options define anchor objects and base transform, while the learned policy predicts an option duration and residual transformation at each option boundary. The residual is applied on top of the base transform and optimized from sparse task-success rewards.
Task
Naive
Seg.-only
OneDemo
Assembly
46.4±42.9
100.0±0.0
100.0±0.0
Basketball
68.8±13.2
76.8±5.2
84.4±5.5
Box Close
57.6±8.5
66.8±13.5
83.6±14.2
Coffee Push
64.8±41.7
99.2±1.0
100.0±0.0
Hand Insert
77.2±22.7
99.2±1.6
100.0±0.0
Peg Insert Side
43.6±16.4
98.4±3.1
98.4±3.1
Table 1: Meta-World success rate on the 15 tasks used by MoDem. Naive uses the randomly initialized duration policy with greedy sampling and applies only the baseline transformation, with no residual warping. Seg.-only uses the learned duration policy with no residual warping. OneDemo uses both the learned duration policy and learned residual warping. Values are mean ± 95% confidence-interval half-width over five seeds; each evaluation uses 50 episodes.
Figure 3: Meta-World sample efficiency aggregated over 15 tasks. OneDemo directly executes the edited demonstration as the task policy and achieves high success with substantially fewer environment interactions than our MoDem reproduction. Shaded regions denote 95% normal-approximation confidence intervals.
Trajectory Edit Success
Downstream Policy Success
Embodiment
Task
OneDemo
DexMimicGen Data ( Jiang et al., 2025 )
OneDemo Data
BC-RNN
Diffusion Policy
BC-RNN
Bimanual Panda + parallel-jaw grippers
Piece Assembly
94.1
66.0±2.0
80.7±0.9
97.3±0.9
Threading
100.0
68.0±1.6
69.3±1.9
100.0±0.0
Transport
99.7
57.3±4.1
83.3±0.9
100.0±0.0
Bimanual Panda + dexterous hands
Box Cleanup
100.0
94.7±0.9
92.0±4.3
100.0±0.0
Table 2: Demonstration-generation and downstream-policy results in DexMimicGen tasks (success rate, %). Trajectory edit success is the fraction of randomized task configurations for which OneDemo successfully edits the source demonstration into a successful trajectory. Bold indicates the best downstream result per task.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Source Demonstration
Naive
OneDemo
Box Close
Least-motion demo
56.8±8.1
83.6±14.5
Random demo 1
88.4±1.5
97.2±1.6
Random demo 2
72.8±5.2
97.6±1.5
Random demo 3
90.8±5.2
96.4±1.5
Basketball
Least-motion demo
67.6±15.7
84.4±5.6
Random demo 1
3.2±2.4
98.0±2.5
Appendix
Table 3: Sensitivity to source-demonstration selection on Meta-World Box Close and Basketball. We compare the least-motion demonstration used in the main experiments with three randomly selected successful source demonstrations. Values are mean ± standard deviation over five training seeds. Naive denotes performance before RL adaptation, and OneDemo denotes best checkpoint performance, consistent with Table 1 .
Hyperparameter
Value
Policy/value trunk
2-layer MLP
Hidden width
256
Activation
Tanh
Optimizer
Adam
Adam ϵ
10−5
PPO clipping coefficient
0.2
Appendix
Table 4: Hyperparameters shared by Option-PPO across benchmarks.
Hyperparameter
Meta-World
DexMimicGen
RoboTwin
Learning rate
3×10−4
3×10−4
3×10−4
LR schedule
constant
constant
linear decay
PPO epochs/update
4
4
4
Minibatches/update
2
32
2
Episodes per PPO update
100–188, task-dependent
2,048
60 or 120, task-dependent
Number of PPO updates
16
100
10 or 20, task-dependent
Appendix
Table 5: Benchmark-specific Option-PPO settings.
Figure 4: Object-centric behavioral dispersion and downstream imitation-learning performance across all nine DexMimicGen tasks. Each gray line connects DexMimicGen and OneDemo data for the same task. The left and right panels show mean phase-wise translational and rotational end-effector trajectory dispersion, respectively. OneDemo consistently produces lower behavioral dispersion while improving downstream BC-RNN success.
Task
Trajectory Edit Success
DP Base
DP OneDemo
Beat Block Hammer
96.7
42
56 (+14)
Handover Block
98.3
15
20 (+5)
Pick Dual Bottles
98.3
25
52 (+27)
Place Empty Cup
96.7
32
65 (+33)
Average
97.5
28.5
48.3 (+19.8)
Appendix
Table 6: RoboTwin results on four bimanual tasks. Trajectory edit success is the fraction of randomized task configurations for which OneDemo successfully edits the source demonstration into a successful trajectory. DP Base denotes our reproduced Diffusion Policy performance using RoboTwin’s released demonstrations, while DP OneDemo denotes Diffusion Policy performance using OneDemo-generated demonstrations. Numbers in parentheses indicate absolute percentage-point improvement over DP Base .