cs.LGNov 9, 2026

RAIL: Representation-Aligned Imitation Learning for Student-Compatible Teacher Policies

Authors: Meraj Mammadov, Pedro Zuidberg Dos Martires, Johannes Andreas Stork

Organizations: Örebro University

Abstract

Reinforcement learning (RL) from raw sensory inputs such as images or onboard sensors can be challenging due to high-dimensional observations and sparse rewards. A common strategy is to train a teacher policy with access to privileged state information and then distill it into a student policy that acts from raw inputs alone. However, when the teacher relies on information unavailable to the student, exact imitation may be impossible, creating an irreducible imitation gap. Existing approaches typically mitigate this mismatch through careful reward shaping or additional reinforcement learning on the student policy after the teacher has been trained. Instead, we introduce Representation-Aligned Imitation Learning (RAIL), which addresses the mismatch during teacher training itself. RAIL learns a latent representation shared across teacher and student observations using contrastive learning, and trains the teacher policy directly in this space. Because the teacher policy is constrained to operate on this shared space, it learns behaviors that are reproducible from student observations. This substantially reduces the imitation gap while preserving task performance. Across multiple environments, RAIL outperforms strong baselines without reward modification or post-hoc student fine-tuning, and surpasses direct reinforcement learning from raw observations. The learned representations further enable zero-shot transfer to new tasks through teacher-only training. Project site: https://merajmammadov.com/rail/.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learning

    May 27, 2026Meraj Mammadov, Pedro Zuidberg Dos Martires, Johannes Andreas StorkImitation LearningRobot Imitation Learning

  2. Teaching a Robot Dog New Tricks: Diverse Quadruped Skills via Combined Reinforcement and Imitation Learning with Adversarial Task Selection

    Oct 6, 2026Lemon Foxmere, Anthony Furman, Yizheng Du +3Robot Skill LearningMulti-Task RL

  3. Feedback Manipulation Regularization: Enabling Offline Agent Alignment for Imitation Learning

    Jul 8, 2026Benjamin Poole, Minwoo LeeImitation LearningSafe RL