SynIL: Leveraging Synergy for Offline Imitation Learning from Imperfect Demonstration Datasets

Yuto Tanaka; Kyo Kutsuzawa; Martina Doku; Dai Owaki; Mitsuhiro Hayashibe

cs.ROSep 28, 2026

SynIL: Leveraging Synergy for Offline Imitation Learning from Imperfect Demonstration Datasets

Authors: Yuto Tanaka, Kyo Kutsuzawa, Martina Doku, Dai Owaki, Mitsuhiro Hayashibe

Organizations: Department of Robotics, Graduate School of Engineering, Tohoku University, 6-6-01 Aoba, Aramaki, Aoba-ku, Sendai, 980-8579, Miyagi, Japan

Abstract

Imitation learning enables robots to acquire complex skills directly from massive demonstration datasets, but its performance degrades severely when datasets are contaminated with suboptimal or noisy demonstrations. While prior quality-assessment methods attempt to filter or reweight data, they typically rely on manual pre-selection of expert reference data or task-specific heuristics, limiting scalability. To address this challenge, we introduce SynIL (Synergy-based Imitation Learning), a novel framework for automated, label-free demonstration quality assessment in offline reinforcement learning. Grounded in neuroscientific evidence that motor synergy, a low-dimensional coordinated structure in movement, correlates directly with motor proficiency, SynIL algorithmically quantifies synergy manifestation to generate dense, transition-level reward signals via self-supervised reward regression. Comprehensive evaluations on D4RL locomotion benchmarks and multi-human Robomimic manipulation datasets demonstrate that synergy-derived rewards correlate strongly with ground-truth rewards. Furthermore, SynIL substantially outperforms Behavior Cloning (BC) and achieves performance comparable to, and in sparse-reward human teleoperation scenarios, superior to, offline reinforcement learning trained on true environment rewards.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of SynIL. Synergy scores $\xi$ are calculated from trajectories of a dataset $\mathbb{D}$ (top-right), and a reward model is trained based on the synergy scores (bottom-right). Offline reinforcement learning is finally performed with rewards computed by the trained reward model (bottom-left).

	Hyperparameter	Value in D4RL	Value in Robomimic
RewardNet	Hidden layers	2	2
	Hidden Size	512	1024
	Activation	ReLU	ReLU
	Last Activation	Linear	Linear
	Epochs	500	10000
	Batch Size	64	64

Table 1: Hyperparameters for SSRR

Hyperparameter	SynIL and IQL	BC
Hidden size	256	256
Hidden layers	2	2
Activation	ReLU	ReLU
Optimizer	Adam	Adam
Epochs	$10^{6}$	$10^{6}$
Evaluate period	5000	5000

Table 2: Hyperparameters for imitation learning

Panel of Figure 2 — Figure 2: Comparison of synergy-based/true rewards for different datasets in D4RL. The vertical axes correspond to the cumulative rewards over 200 steps estimated by SSRR, and the horizontal axes correspond to the true cumulative rewards over 200 steps.

Dataset	Correlation coefficient
halfcheetah-medium-expert-v2	$0.9959$
walker2d-medium-expert-v2	$0.8655$
hopper-medium-expert-v2	$0.8718$
halfcheetah-medium-v2	$0.9482$
walker2d-medium-v2	$0.5809$
hopper-medium-v2	$0.9526$

Table 3: Correlation coefficients between the synergy-based and true rewards in D4RL

Dataset	BC	SynIL (Ours)	IQL (ground-truth reward)
halfcheetah-medium-v2	$42.7\pm 0.3$	${\bf 45.4\pm 0.2}$	$47.4\pm 0.2$
hopper-medium-v2	${\bf 54.4\pm 1.7}$	$46.6\pm 3.8$	$54.4\pm 3.5$
walker2d-medium-v2	$71.7\pm 6.3$	${\bf 72.0\pm 5.4}$	$80.0\pm 4.9$
halfcheetah-medium-replay-v2	$37.1\pm 2.4$	${\bf 40.6\pm 1.4}$	$44.3\pm 0.5$
hopper-medium-replay-v2	$14.4\pm 2.4$	${\bf 99.9\pm 0.5}$	$66.5\pm 8.1$
walker2d-medium-replay-v2	$15.5\pm 6.4$	${\bf 74.5\pm 3.0}$	$69.5\pm 7.9$

Table 4: Performance of each algorithm in D4RL (Average sum of rewards over 10 episodes)

Dataset	BC	SynIL (Ours)	IQL (sparse reward)
Can (MH)	$56.7\pm 10.0$	${\bf 94.0\pm 3.3}$	$67.3\pm 5.0$
Square (MH)	$24.0\pm 4.3$	${\bf 49.3\pm 8.2}$	$16.7\pm 5.2$

Table 5: Performance comparison in Robomimic (average success rate, 3 seeds

\times

50 rollouts)

Dataset	BC	SynIL (Ours)	IQL (sparse reward)
Can (MH)	$2.1\pm 0.2$	${\bf 1.3\pm 0.0}$	$2.2\pm 0.2$
Square (MH)	$2.9\pm 0.0$	${\bf 2.0\pm 0.1}$	$3.4\pm 0.2$

Table 6: Execution time in Robomimic (seconds)

Figure 3: Learning curves of all environments.

Figure 4: Learning curves of Robomimic.

Explore similar work

SynthICL: Scalable In-context Imitation Learning with Synthetic Data
Jun 6, 2026Cheng Qian, Ruomeng Fan, Yifei Ren +2ICL for Robotics Robot Skill Learning
The Robot Learning Lab Imperial College London
Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation
Apr 4, 2026Pierrick Lorang, Johannes Huemer, Timothy Duggan +3Robot Imitation Learning Robotic Manipulation
Department of Computer Science, Tufts University, Medford, MA, USA · Complex Dynamical Systems, Austrian Institute of Technology GmbH (AIT), Vienna, Austria
Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy
May 2, 2026Noushad Sojib, Ola Ghattas, Momotaz BegumRobot Policy Learning Robot Skill Learning
Department of Computer Science, University of New Hampshire, USA

Jun 6, 2026J/K move · Enter open · S save

SynthICL: Scalable In-context Imitation Learning with Synthetic Data

Cheng Qian, Ruomeng Fan, Yifei Ren, Yilong Wang+1

The Robot Learning Lab Imperial College London

ICL for Robotics Robot Skill Learning Generalization in Robotic Manipulation Synthetic Data Generation for Robotics AI Control Imitation Learning+2

Open paper AskPDF arXiv

In-context imitation learning (ICIL) enables robots to learn new tasks from a small number of demonstrations by conditioning a pre-trained policy on task-specific examples, without retraining at test time. Despite this promise, training generalizable and scalable in-context imitation policies remains an open challenge. We present SynthICL, a scalable framework that trains ICIL policies entirely from RGB-only synthetic data. Specifically, we build a data generation pipeline to produce high-fidelity ICIL data and train a flow-matching transformer policy on the resulting dataset. SynthICL avoids the need for depth sensing, precise camera calibration, and real-world training data in prior approaches, offering a simpler and more scalable alternative. We further incorporate subgoal prediction by training the model to predict the next subgoal images, enabling more precise and visually grounded control. Evaluated on 16 unseen real-world manipulation tasks, SynthICL achieves an average success rate of 79% with only one demonstration provided at test time and outperforms prior methods. Project page: https://synth-icl.github.io

Abstract

Explore similar work

SynthICL: Scalable In-context Imitation Learning with Synthetic Data

Build on Priors: Vision--Language--Guided Neuro-Symbolic Imitation Learning for Data-Efficient Real-World Robot Manipulation

Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy