cs.LGNov 11, 2026

Imitating the Imperfect: Offline-to-Online Robust Imitation Learning from Heterogeneous Demonstrators

Authors: Cheng Pan, Nan Lu, Kai Arulkumaran, Josie Hughes

Organizations: Swiss Federal Institute of Technology in Lausanne (EPFL), Lausanne, Switzerland. · University of Bristol, Bristol, UK. · Araya Inc., Tokyo, Japan.

Abstract

Imitation learning (IL) typically relies on large-scale offline demonstrations collected from multiple human or algorithmic demonstrators. However, most existing approaches assume these demonstrators are homogeneous or near-optimal experts, a convenient but unrealistic assumption in real-world applications. In practice, demonstrations are often collected from heterogeneous demonstrators with unknown and varying levels of expertise, resulting in highly mixed-quality data. In this work, we study a challenging yet practical setting of imitation learning from such heterogeneous imperfect demonstrators. We propose Latent Expertise and Optimality Scoring based Imitation Learning (LEOS-IL), a principled framework that jointly learns (i) a state-action optimality scoring model and (ii) latent expertise levels for each demonstrator from unlabeled suboptimal offline demonstrations. The learned scoring model is then integrated into an online policy optimization procedure, where the agent is trained to maximize estimated optimality scores while its online rollouts are iteratively leveraged to refine the scoring model, thereby mitigating distributional shift during online adaptation. We further provide theoretical guarantees for optimality score identification and conditional optimal occupancy and policy recovery under explicit sufficient conditions. Experiments on continuous control benchmarks demonstrate that our approach consistently outperforms existing baselines across highly heterogeneous and low-quality demonstration regimes.

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SynIL: Leveraging Synergy for Offline Imitation Learning from Imperfect Demonstration Datasets

    Sep 28, 2026Yuto Tanaka, Kyo Kutsuzawa, Martina Doku +2Robot Imitation LearningRobotic Control

  2. Latent Wasserstein Adversarial Imitation Learning

    Mar 5, 2026Siqi Yang, Kai Yan, Alexander G. Schwing +1Imitation LearningAdversarial Imitation Learning

  3. How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learning

    May 25, 2026Hyung-Suk Yoon, Seung-Woo SeoDistribution Shift RobustnessImitation Learning