RESTA: Time-Scaling Augmentation for Data-Efficient Robot Imitation Learning

Yunpeng Jiang; Jianshu Hu; Han Fang; Zhiyuan Guan; Paul Weng; Yutong Ban

cs.ROOct 13, 2026

RESTA: Time-Scaling Augmentation for Data-Efficient Robot Imitation Learning

Authors: Yunpeng Jiang, Jianshu Hu, Han Fang, Zhiyuan Guan, Paul Weng, Yutong Ban

Organizations: Global College, Shanghai Jiao Tong University, Shanghai, China · Shanghai Institute of Physical Intelligence and Robotics (PAIR), Shanghai, China · Duke Kunshan University, Jiangsu, China

Abstract

Robot manipulation policies trained through imitation learning commonly rely on costly demonstrations recorded in one task direction and at one temporal resolution. We propose Reversal and Stride-based Trajectory Augmentation (RESTA), which exploits time-scaling data augmentation to improve data efficiency through trajectory reversal and temporal resampling. Trajectory-reversal augmentation reverses and relabels demonstrations between paired tasks with opposite objectives, reusing recorded rollouts without additional environment interaction. Because contact-rich manipulation is not exactly reversible, we treat reversal as an approximate paired-task prior rather than a physical symmetry. Temporal-stride augmentation varies observation and action sampling strides and conditions the policy on the resulting offsets, enabling fixed or randomized temporal schedules. Experiments across 16 paired bimanual manipulation tasks show that trajectory reversal is particularly effective in the low-data regime and that temporal-stride augmentation improves success under randomized temporal schedules. With 25 and 50 independently collected demonstrations per task, RESTA achieves 78.4% and 85.2% average success, respectively, matching or exceeding baselines trained with twice as many demonstrations (77.3% and 85.4%). These results show that RESTA can improve data efficiency in robot imitation learning.

Figures & tables

Figure 1: Overview and aggregate results for RESTA. Left: Open Laptop and Close Laptop trajectories illustrate a reversible task pair. Middle: time scaling is illustrated through trajectory reversal, which connects representative states across forward and reversed time, and temporal-stride augmentation, which samples a shared timeline uniformly or variably. Right: bars report mean evaluation success and approximate simulated duration over the 16-task suite; higher success and lower duration are preferred. $B_{n}$ denotes DP3 trained with $n$ native demonstrations per task, and $\mathrm{RESTA}_{n}$ adds trajectory-reversal and temporal-stride augmentation.

Figure 2: Overview of RESTA. (a): paired task directions provide demonstrations that are reversed and relabeled to construct augmented datasets. (b): during training, sampled temporal offsets determine the observation and action samples and condition the policy. (c): at inference, a schedule selects action offsets, and the executed action chunk determines the observation spacing for the next policy cycle.

Task	$B_{25}$	$RESTA{}_{25}$	$B_{50}$	$RESTA{}_{50}$	$B_{100}$
Place Fan	23	43	34	55	65
Remove Fan	87	96	92	99	99
Place Object Stand	43	58	61	79	80
Remove Object Stand	64	75	82	76	87
Place Empty Cup	47	77	77	91	93
Remove Empty Cup	91	97	90	100	100

Table 1: Main results on all 16 paired tasks at 3,000 training epochs.

B_{n}

is DP3 trained on

n

native trajectories.

RESTA{}_{n}

is DP3 trained on the same

n

native trajectories and their offline time-reversed counterparts, with temporal-stride augmentation and observation-offset conditioning. The subscript counts independently collected trajectories;

B_{2n}

is DP3 trained on

2n

native trajectories. Bold text with blue shading marks the best result at each current budget;

B_{100}

is a reference upper bound and is not highlighted. Values are success rates (%). Higher is better.

Measure	$B_{25}$	$\mathrm{RESTA{}}_{25}$	$B_{50}$	$\mathrm{RESTA{}}_{50}$	$B_{100}$
Policy actions	217.8	100.3	209.2	99.3	196.4
Physics steps	14251.3	9337.0	13223.3	9063.5	12879.4

Table 2: Average execution steps over the sixteen paired tasks. Each task mean uses all successful episodes. RESTA uses the rand offset schedule. Lower is better.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Setting
Inputs	1,024 point-cloud points; 14-D robot state and action
Chunking	3 observation steps; horizon 8; predict and execute 6 action targets
DP3 backbone	FiLM-conditioned U-Net with channel widths $512$ , $1024$ , and $2048$ ; 128-D point-cloud feature
Diffusion	100 training steps with the squared-cosine schedule; 10 DDIM inference steps
Optimizer	AdamW; learning rate $10^{-4}$ ; weight decay $10^{-6}$ ; betas $(0.95,0.999)$
Learning-rate schedule	Cosine decay with 500 warm-up steps

Appendix

Table 3: Model, optimization, and temporal-offset settings. The settings are shared across data budgets; augmentation is enabled only for the corresponding method variants.

Task pair	Forward task	Reverse task
Fan manipulation	Place Fan	Remove Fan
Object-stand manipulation	Place Object Stand	Remove Object Stand
Empty-cup manipulation	Place Empty Cup	Remove Empty Cup
Mug manipulation	Hang Mug	Unhang Mug
QR-code manipulation	Rotate QR Code	Rotate QR Code Back
Block handover	Handover Block	Handover Block Back

Appendix

Table 4: Forward tasks selected from RoboTwin 2.0 and their independently implemented reverse-task counterparts. Direction: not applicable; this table lists task correspondences.

Panel of Figure 3 — Figure 3: Four-frame expert demonstrations for all sixteen tasks, arranged as eight paired rows with the two task directions side by side.

Task	$B_{25}$	$\mathrm{TR}_{25}$	$\mathrm{TSAO}_{25}$	$RESTA{}_{25}$	$B_{50}$	$\mathrm{TR}_{50}$	$\mathrm{TSAO}_{50}$	$RESTA{}_{50}$	$B_{100}$
Place Fan	23	34	30	43	34	58	38	55	65
Remove Fan	87	82	100	96	92	85	100	99	99
Place Object Stand	43	61	50	58	61	72	67	79	80
Remove Object Stand	64	67	68	75	82	80	85	76	87
Place Empty Cup	47	83	71	77	77	74	85	91	93
Remove Empty Cup	91	94	95	97	90	98	99	100	100

Appendix

Table 5: Per-task data-efficiency comparison at 3,000 training epochs.

B_{n}

is DP3 trained with

n

native trajectories per task.

\mathrm{TR}_{n}

adds

n

time-reversed trajectories, while

\mathrm{TSAO}_{n}

uses only the

n

native trajectories with temporal-stride augmentation and observation-offset conditioning.

RESTA{}_{n}

combines trajectory reversal with this augmentation. TSAO and

RESTA{}_{n}

use the rand inference schedule.

B_{100}

is the larger-data reference and is not highlighted. Bold text with blue shading indicates the best result at each current budget. Values are success rates (%). Higher is better.

Task	$B_{50}$	offset1	offset2	offset3
Place Fan	34	33	34	32
Remove Fan	92	100	100	100
Place Object Stand	61	66	67	63
Remove Object Stand	82	78	84	82
Place Empty Cup	77	79	85	87
Remove Empty Cup	90	97	97	98

Appendix

Table 6: Per-task success rates for the inference-schedule sweep in the

n=50

setting, where

n

is the number of native trajectories per task. The baseline is DP3; the remaining columns use TSAO and the indicated inference schedule. Bold and blue shading indicate the best result for each task or average across both blocks. Values are success rates (%). Higher is better.

Task	$B_{25}$	$\mathrm{TSA}_{25}$	$\mathrm{TSAO}_{25}$	$B_{50}$	$\mathrm{TSA}_{50}$	$\mathrm{TSAO}_{50}$
Place Fan	23	18	30	34	30	38
Remove Fan	87	100	100	92	100	100
Place Object Stand	43	55	50	61	67	67
Remove Object Stand	64	66	68	82	81	85
Place Empty Cup	47	69	71	77	80	85
Remove Empty Cup	91	95	95	90	97	99

Appendix

Table 7: Per-task effect of temporal-stride augmentation and observation-offset conditioning at two trajectory budgets. The TSA and TSAO entries are the selected rand inference results from the corresponding budget-specific evaluations. For each

n\in\{25,50\}

B_{n}

is the baseline,

\mathrm{TSA}_{n}

uses temporal-stride augmentation with consecutive observations, and

\mathrm{TSAO}_{n}

additionally conditions on observation offsets. Bold text with blue shading indicates the best result within each budget for each task or average across tasks. Values are success rates (%). Higher is better.

Task	$B_{25}$	$\mathrm{RESTA{}}_{25}$	$B_{50}$	$\mathrm{RESTA{}}_{50}$	$B_{100}$
Place Fan	131.5	71.9	143.8	68.7	139.3
Remove Fan	140.1	70.4	135.9	65.9	117.9
Place Object Stand	132.9	86.7	129.9	64.9	130.9
Remove Object Stand	159.2	60.9	131.8	63.2	112.7
Place Empty Cup	149.6	63.0	139.8	58.0	144.0
Remove Empty Cup	135.5	63.4	142.1	65.9	119.3

Appendix

Table 8: Mean policy actions per successful episode for baseline and RESTA checkpoints on the 16-task paired suite. RESTA uses the rand schedule; each task mean is based on all successful episodes. For each task and for the average across tasks, bold text with blue shading marks the lower value within each

n=25

and

n=50

baseline/RESTA pair. Lower is better.

Task	$B_{25}$	$\mathrm{RESTA{}}_{25}$	$B_{50}$	$\mathrm{RESTA{}}_{50}$	$B_{100}$
Place Fan	10734.4	7908.4	10897.0	7406.1	10494.5
Remove Fan	11734.1	5995.5	9287.3	6789.6	9386.8
Place Object Stand	10198.1	8883.6	9574.5	6259.4	9130.2
Remove Object Stand	13009.2	6246.6	9436.0	6097.9	8730.4
Place Empty Cup	9728.8	5878.8	8935.4	5460.3	8629.3
Remove Empty Cup	9251.7	5984.6	9447.8	6354.0	8864.5

Appendix

Table 9: Mean simulator physics steps per successful episode for baseline and RESTA checkpoints on the 16-task paired suite. Physics steps include low-level interpolation. RESTA uses the rand schedule; each task mean is based on all successful episodes. For each task and for the average across tasks, bold text with blue shading marks the lower value within each

n=25

and

n=50

baseline/RESTA pair. Lower is better.

Explore similar work

MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning
Jun 23, 2026Qingyang Wang, Xingang Liu, Changwei Yao +4Synthetic Data Augmentation Data Augmentation
School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen 518055, China · Pengcheng Laboratory, Shenzhen 518000, China · Carnegie Mellon University, Pittsburgh, PA 15213, USA
Improving Robotic Imitation Learning via Trajectory Standardization
Jun 22, 2026Licheng Yang, Lingfeng Qian, Fei Zheng +4Robotic Manipulation Robot Imitation Learning
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · D-Robotics
Data Scaling Laws in Imitation Learning for Robotic Manipulation
Oct 24, 2024Fanqi Lin, Yingdong Hu, Pingyue Sheng +3Generalization in Robotic Manipulation Robotic Manipulation
Tsinghua University · Shanghai Qi Zhi Institute · Shanghai Artificial Intelligence Laboratory

Jun 23, 2026J/K move · Enter open · S save

MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning

Qingyang Wang, Xingang Liu, Changwei Yao, Zikai Ouyang+3

School of Automation and Intelligent Manufacturing, Southern University of Science and Technology, Shenzhen 518055, China · Pengcheng Laboratory, Shenzhen 518000, China · Carnegie Mellon University, Pittsburgh, PA 15213, USA

Synthetic Data Augmentation Data Augmentation Robotic Manipulation Robot Imitation Learning

Open paper AskPDF arXiv

Imitation learning enables robots to acquire complex manipulation skills from demonstrations, but its effectiveness is limited by the cost of collecting high-quality data. Trajectory-level data augmentation methods alleviate this challenge by recombining expert demonstrations under varied initial states. However, such methods typically insert interpolations or other non-expert transition segments between disjoint parts, and such non-expert segments could reduce the quality of the generated data. This paper introduces Minimizing Interpolation (MinInter), an effective trajectory selection method that, for each sampled initial configuration, chooses the source demonstration requiring the least interpolation to form a complete trajectory. By explicitly minimizing interpolations during data generation, MinInter produces higher-quality synthetic demonstrations while remaining compatible with existing data generation frameworks. Experiments on 12 manipulation tasks with 26 variants from the MimicGen benchmark show that MinInter consistently improves both data generation success rates and policy success rates, with the largest gains on contact-rich, long-horizon and high-variance settings. Compared to the recent SkillGen framework, MinInter achieves higher policy success rates despite its conceptual simplicity, underscoring the value of interpolation minimization for data augmentation.

Abstract

Explore similar work

MinInter: Minimizing Trajectory Interpolation During Data Augmentation for Imitation Learning

Improving Robotic Imitation Learning via Trajectory Standardization

Data Scaling Laws in Imitation Learning for Robotic Manipulation