cs.RONov 7, 2026

Explicitly capturing demonstrator multi-modality for distillation into diffusion policies

Authors: Shao Yuan Chew Chia, Rebecca H. Jiang, Abhinav Agarwal, Andreas Hofmann, Leslie Pack Kaelbling, Russ Tedrake

Organizations: Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA, USA · Pickle Robot Company, Charlestown, MA, USA

Abstract

The diversity of a distilled diffusion policy's outputs matters beyond how it impacts the policy's success rate. In particular, downstream applications like test-time search and policy improvement require exploration. Thus, we want to distill multi-modal demonstrators into diffusion policies which reproduce their conditional action distributions. Standard data collection for behavior cloning applies a sliding window to demonstration episodes, which pairs unique observation histories with actions. These datasets can be fit by a policy that is either multi-modal, or unimodal with high Lipschitz constant to capture a diversity of actions seen at nearby observation samples. Prior work finds that diffusion policies trained on sparse data learn the latter, memorizing one action per observation. We adapt vine sampling to behavior cloning data collection to resolve this ambiguity: at some states the demonstrator visits, query it kk times from exactly the same observation. On a synthetic task, we study how the number of actions per observation, kk, affects the learned policy, and find that larger kk yields a policy that better matches the demonstrator's conditional distribution, including its multi-modality and diversity. On a 2D truck unloading task, vine sampling likewise leads to a policy closer to the demonstrator and substantially improves both best-of-NN and expert iteration performance.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Understanding Multimodality in Generative Behavioral Cloning

    May 21, 2026Lorenzo Mazza, Massimiliano Datres, Ariel Rodriguez +3Latent Variable ModelsImitation Learning

  2. Immiscible Diffusion Policy: Preserving Multimodal Robot Actions through Label-Free Noise Assignment

    Oct 7, 2026Xiao Zhang, Yuxin Chen, Zhixuan Liang +5Diffusion PolicyMode Collapse

  3. Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies

    May 12, 2026Alberta Longhini, David Emukpere, Jean-Michel Renders +1RL for RoboticsReinforcement Learning