Learning robot policies from demonstrations is often limited by the cost of collecting diverse, high-quality expert data. While an appealing alternative is to expand a single successful demonstration into a diverse set of successful trajectories, existing augmentation methods typically rely on manually designed transformation rules or complex trajectory planning, requiring substantial task-specific engineering and human effort. We introduce OneDemo, a reinforcement learning framework that learns to edit a single successful demonstration for new task configurations. We formulate demonstration editing as a structured reinforcement learning problem, in which an editor learns, from sparse task-success rewards, both when to transition between object-centric trajectory segments and how to geometrically adapt each segment. The learned editor serves both as a task policy and as a reusable demonstration generator. Across 15 Meta-World tasks, OneDemo reaches a 92.7% success rate within 53.7k environment interactions on average. Across 9 DexMimicGen tasks, it generates successful demonstrations with a 98.4% average success rate, and BC-RNN policies trained on the generated data achieve 99.6% average success at matched dataset scale. These results show that learning to edit a single successful demonstration provides a structured and effective basis for both policy learning and scalable demonstration generation.
Figures & tables
Figure 1: OneDemo learns to edit a single demonstration to generate varying successful trajectories under new spatial configurations. Left: OneDemo adapts the source trajectory to generate successful augmentations. Right: Recurrent behavior cloning (BC-RNN) trained on demonstrations augmented by OneDemo improves success from 76.3% to 99.6% compared with DexMimicGen data.
Figure 2: Overview of OneDemo . Semantic options define anchor objects and base transform, while the learned policy predicts an option duration and residual transformation at each option boundary. The residual is applied on top of the base transform and optimized from sparse task-success rewards.
Task
Naive
Seg.-only
OneDemo
Assembly
46.4±42.9
100.0±0.0
100.0±0.0
Basketball
68.8±13.2
76.8±5.2
84.4±5.5
Box Close
57.6±8.5
66.8±13.5
83.6±14.2
Coffee Push
64.8±41.7
99.2±1.0
100.0±0.0
Hand Insert
77.2±22.7
99.2±1.6
100.0±0.0
Peg Insert Side
43.6±16.4
98.4±3.1
98.4±3.1
Table 1: Meta-World success rate on the 15 tasks used by MoDem. Naive uses the randomly initialized duration policy with greedy sampling and applies only the baseline transformation, with no residual warping. Seg.-only uses the learned duration policy with no residual warping. OneDemo uses both the learned duration policy and learned residual warping. Values are mean ± 95% confidence-interval half-width over five seeds; each evaluation uses 50 episodes.
Figure 3: Meta-World sample efficiency aggregated over 15 tasks. OneDemo directly executes the edited demonstration as the task policy and achieves high success with substantially fewer environment interactions than our MoDem reproduction. Shaded regions denote 95% normal-approximation confidence intervals.
Trajectory Edit Success
Downstream Policy Success
Embodiment
Task
OneDemo
DexMimicGen Data ( Jiang et al., 2025 )
OneDemo Data
BC-RNN
Diffusion Policy
BC-RNN
Bimanual Panda + parallel-jaw grippers
Piece Assembly
94.1
66.0±2.0
80.7±0.9
97.3±0.9
Threading
100.0
68.0±1.6
69.3±1.9
100.0±0.0
Transport
99.7
57.3±4.1
83.3±0.9
100.0±0.0
Bimanual Panda + dexterous hands
Box Cleanup
100.0
94.7±0.9
92.0±4.3
100.0±0.0
Table 2: Demonstration-generation and downstream-policy results in DexMimicGen tasks (success rate, %). Trajectory edit success is the fraction of randomized task configurations for which OneDemo successfully edits the source demonstration into a successful trajectory. Bold indicates the best downstream result per task.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Task
Source Demonstration
Naive
OneDemo
Box Close
Least-motion demo
56.8±8.1
83.6±14.5
Random demo 1
88.4±1.5
97.2±1.6
Random demo 2
72.8±5.2
97.6±1.5
Random demo 3
90.8±5.2
96.4±1.5
Basketball
Least-motion demo
67.6±15.7
84.4±5.6
Random demo 1
3.2±2.4
98.0±2.5
Appendix
Table 3: Sensitivity to source-demonstration selection on Meta-World Box Close and Basketball. We compare the least-motion demonstration used in the main experiments with three randomly selected successful source demonstrations. Values are mean ± standard deviation over five training seeds. Naive denotes performance before RL adaptation, and OneDemo denotes best checkpoint performance, consistent with Table 1 .
Hyperparameter
Value
Policy/value trunk
2-layer MLP
Hidden width
256
Activation
Tanh
Optimizer
Adam
Adam ϵ
10−5
PPO clipping coefficient
0.2
Appendix
Table 4: Hyperparameters shared by Option-PPO across benchmarks.
Hyperparameter
Meta-World
DexMimicGen
RoboTwin
Learning rate
3×10−4
3×10−4
3×10−4
LR schedule
constant
constant
linear decay
PPO epochs/update
4
4
4
Minibatches/update
2
32
2
Episodes per PPO update
100–188, task-dependent
2,048
60 or 120, task-dependent
Number of PPO updates
16
100
10 or 20, task-dependent
Appendix
Table 5: Benchmark-specific Option-PPO settings.
Figure 4: Object-centric behavioral dispersion and downstream imitation-learning performance across all nine DexMimicGen tasks. Each gray line connects DexMimicGen and OneDemo data for the same task. The left and right panels show mean phase-wise translational and rotational end-effector trajectory dispersion, respectively. OneDemo consistently produces lower behavioral dispersion while improving downstream BC-RNN success.
Task
Trajectory Edit Success
DP Base
DP OneDemo
Beat Block Hammer
96.7
42
56 (+14)
Handover Block
98.3
15
20 (+5)
Pick Dual Bottles
98.3
25
52 (+27)
Place Empty Cup
96.7
32
65 (+33)
Average
97.5
28.5
48.3 (+19.8)
Appendix
Table 6: RoboTwin results on four bimanual tasks. Trajectory edit success is the fraction of randomized task configurations for which OneDemo successfully edits the source demonstration into a successful trajectory. DP Base denotes our reproduced Diffusion Policy performance using RoboTwin’s released demonstrations, while DP OneDemo denotes Diffusion Policy performance using OneDemo-generated demonstrations. Numbers in parentheses indicate absolute percentage-point improvement over DP Base .
Learning robot manipulation policies typically requires substantial demonstration data, which are costly to collect on real robots. Recent methods generate robot demonstrations from human videos by adapting recovered motion and validating the resulting trajectories in simulation. However, methods centered on motion-reference adaptation can limit behavioral diversity by retaining the demonstrated contact strategies and subtask orders, while insufficient understanding of task requirements and scene relations can reduce demonstration generation efficiency by generating invalid candidates. To address these limitations, we propose KnowDemo, a framework that uses structured manipulation knowledge from human videos to generate diverse robot demonstrations for a target workspace. To distinguish task requirements from demonstration-specific choices, we develop a knowledge extraction and reasoning module based on a vision-language model (VLM) that associates object and action descriptions with inferred task conditions, demonstration references, and permissible execution variations. To translate this knowledge into executable demonstrations, we resolve the descriptions against target-scene entities and geometry to guide candidate generation and screening before motion planning and simulation. The resulting demonstrations exhibit multimodal behavior through alternative contact strategies and valid subtask orders, with structured execution labels. Experiments demonstrate additional verified execution modes beyond a reference-only configuration and improved candidate planning success through task-guided grasp sampling. To validate the generated data for policy learning, we fine-tune the pretrained π0.5 model on simulation data, achieving sim-to-real transfer across three tasks. Project page: https://zhiyuan-gao.github.io/knowdemo/
Zhiyuan Gao, Yanxiang Zhan, Mohammad Khoshnazar +2
University of Bremen · Robotics Institute Germany (RIG)
Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robot control. However, their performance remains fundamentally constrained by the availability of high-quality robot trajectory data. In current robot learning practice, such data are primarily collected through human teleoperation, which is labor-intensive, costly, and difficult to scale. In this paper, we propose RDGen, a sim-to-real reinforcement learning framework for generating high-quality robot demonstrations. Rather than employing reinforcement learning solely as the final control policy, RDGen leverages trained RL policies as a structured trajectory generator. The system consists of a VLM-based task parser that identifies task-relevant objects, a Grounding DINO-based object localizer, and an RL policy transferred from simulation to the real robot. Successful rollouts are then harvested as clean, high-quality demonstrations for downstream VLA training, while the simulation stage further provides a scalable source of additional trajectories at little marginal cost. Experiments on a pick-and-place task demonstrate that the transferred RL policy achieves a high task success rate. Compared with human teleoperation, RDGen produces significantly smoother trajectories and yields superior downstream VLA performance. These results indicate that RL-generated demonstrations can serve as more reliable and consistent supervisory signals for robot policy learning.
3D visuomotor policies offer a promising direction for complex robotic manipulation, as depth maps and point clouds provide rich geometric information for spatial reasoning. However, their success often depends on large-scale real-world demonstrations, which are costly and time-consuming to collect. To this end, existing methods commonly use demonstration generation strategies to improve data efficiency by applying object-centric transformations to human-collected demonstrations, such as varying object poses or scales. While effective for local variation, these transformations largely preserve the original scene structure and skill sequence, limiting their ability to synthesize diverse scene-skill-object combinations for complex tasks. In this paper, we propose Task-Edit, a novel demonstration generation framework that generates diverse trajectories from a task-centric editing perspective. The key insight of Task-Edit is to decompose a task into scene, skill and object components, and flexibly recombine them. In this way, Task-Edit enables scalable demonstration generation and significantly improves generalization for long-horizon manipulation tasks. We evaluate Task-Edit through extensive real-world experiments and demonstrate three advantages: (1) Effectiveness: Task-Edit significantly improves 3D visuomotor policies across various real-world tasks and robot embodiments. (2) Generalizability: Task-Edit improves model generalization across different scenario setups. (3) Applicability: Task-Edit enables models to handle scenarios that are difficult to collect in the real world, including disturbance resistance, obstacle avoidance and unseen cluttered scenes.
Jian-Jian Jiang, YiHan Yang, Lan Wei +6
Sun Yat-sen University · South China University of Technology · Nanyang Technological University +1