cs.CVNov 18, 2026

Training Denoising Models of Any Architecture Without Stored Images

Authors: Christos Koutlis, Ioannis Kompatsiaris, Symeon Papadopoulos

Organizations: Information Technologies Institute, CERTH, Thessaloniki, Greece

Abstract

Can a pretrained denoising model replace the dataset used to train it? We investigate this question through teacher-supervised training, showing for the first time that continuous-time diffusion and flow models of any architecture and size can be trained from random initialization without stored images. Our students regress the teacher's velocity on a rolling pool of noisy samples, advanced along trajectories of random step counts to obtain independent, dense and approximately continuous noise levels. This saves considerable storage for large datasets and applies even when the data are unreleased or private. Our experiments include 12 teachers of 6 families and 6 datasets. Our method outperforms standard training of the same network on CIFAR-10 for students 4x smaller than the teacher, comes within 0.75 to 1.92 FID of it at the teacher's size and surpasses training on teacher-generated images and step reduction methods without teacher-based initialization. Our code is available at https://github.com/mever-team/ddm.

Figures & tables

Explore similar work

CardsList
  1. Consistent Distribution Matching for Data-Free Diffusion Distillation

    Oct 6, 2026Yuxiang Fu, Qi Yan, Zike Wu +4Diffusion Model DistillationDiffusion Model Inference Acceleration

  2. Improved Distributional Diffusion Models

    Sep 29, 2026Tommaso Martorella, Alexandre Galashov, Felix Krause +4Few-Step Diffusion SamplingDiffusion-Based Image Generation

  3. Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations

    May 8, 2026Yuan Zhang, Chenyi Li, Guoqing Ma +7Diffusion Model DistillationDiffusion Sampling