Overview

Dataset distillation aims to replace a large training set with a compact set of examples that lets models learn effectively while reducing storage, communication, or training costs. Common approaches match gradients or training trajectories, align feature or data distributions, select representative prototypes, or generate examples with diffusion models; newer work also targets text, graphs, time series, video, and multimodal data. Researchers additionally study how to compress labels and bits, and how to preserve robustness, fairness, and performance across model architectures. A central open tension is whether distilled sets reliably outperform simpler random subsets or coresets, especially at large scale, while balancing downstream accuracy against faithful coverage of the original data.

Latest papers

CardsList

73 papers5 papers in the last four weeks, down 29% on the four weeks before. 0.0% of all new papers.

  1. Dataset Distillation for Handwritten Text Recognition: An Empirical Study

    Nov 16, 2026Félix Fuentes-Hurtado, Aric Ayllón, Juan C. AlfaroDataset DistillationDistribution Matching

  2. Room for Supervision: Training-Free Dataset Distillation for Object Detection

    Nov 16, 2026Ziang Gan, Qi Zhu, Libao ZhangDataset DistillationObject Detection

  3. Dataset Distillation via Drifting

    Nov 12, 2026George Cazenavette, Angelina Quan, Giannis Daras +2Dataset DistillationDistribution Matching

  4. Model Distribution-Aware Multimodal Dataset Distillation

    Nov 12, 2026Jiajun Shen, Yonglin Wu, Ruonan Yu +1Dataset DistillationMultimodal Pretraining

  5. Disentangled Embedding Optimization for Diffusion-Based Dataset Distillation

    Nov 12, 2026Hongcheng Li, Yucan Zhou, Yibin Wang +3Diffusion-Based Dataset DistillationDataset Distillation

  6. Online Label-Free Gradient Matching for Continual Language Model Training

    Nov 2, 2026Sajjad Pakdamansavoji, Mohammad Reza Rezaei, Ilija Subasic +3Continual Learning for LLMsDataset Distillation

  7. S2^2-RL: Sample-Set Dual Reinforcement Learning for Generative Semantic Segmentation Dataset Distillation

    Oct 26, 2026Haoyu Wang, Fei Zhou, Qingqing Qiu +3Dataset DistillationDiffusion-Based Dataset Distillation

  8. ProMoDD: Dataset Distillation for 3D Human Motion via Prototype-Conditioned Diffusion Models

    Oct 12, 2026Yijin Wei, Yawen Zou, Wenqi Cai +5Dataset DistillationSkeleton-Based Action Recognition

  9. X-AVDD: Cross-Attentive Audio-Visual Dataset Distillation

    Oct 9, 2026Saumyaranjan Mohanty, Aravind Reddy, Konda Reddy MopuriDataset DistillationAudio-Visual Alignment

  10. MGPO: Manifold-Guided Diffusion Alignment for Task-Aware Dataset Distillation

    Oct 4, 2026Yunyi Chen, Chenru Wang, Xinyi Ye +2Diffusion Model AlignmentDataset Distillation

  11. Nonparametric Distribution Matching for Self-Supervised Whole-Slide Image Condensation

    Sep 30, 2026Duong M. Nguyen, Trong Nghia Hoang, Hang Thi Nguyen +3Computational PathologyDistribution Matching

  12. The Composition Gap in Dataset Distillation

    Sep 28, 2026Guang Li, Takahiro Ogawa, Miki HaseyamaDataset Distillation

  13. Difficulty-Aware Statistical Matching for Dataset Distillation

    Sep 28, 2026Hongxu Ma, Guang Li, Shijie Wang +6Distribution MatchingDataset Distillation

  14. Latent Dataset Distillation for Human Motion Prediction

    Sep 22, 2026Ge Tian, Guang Li, Takahiro Ogawa +1Dataset DistillationHuman Motion Prediction

  15. Towards Scalable RLVR: Multimodal Instruction Following Data Synthesis and Distillation

    Sep 13, 2026Yirong Zeng, Zhang Sai, Yuxian Wang +4Multimodal Instruction TuningRL for VLMs

  16. DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation

    Aug 26, 2026Chuixuan Fan, Guang Li, Shijie Wang +5Dataset DistillationFine-Grained Image Classification

  17. Efficient Video Dataset Distillation via Cluster-Guided Prototype Blending

    Aug 4, 2026Chongle Ren, Guang Li, Wenbo Huang +3Dataset Distillation

  18. Self-Supervised Representation-Guided Generative Dataset Distillation

    Aug 4, 2026Mingzhuo Li, Guang Li, Linfeng Ye +4Diffusion Model GuidanceDataset Distillation

  19. Long-Tailed 3D Point Cloud Dataset Distillation

    Jul 29, 2026Jiahao You, Xu Han, Jinfeng Xu +1Long-Tailed ClassificationDataset Distillation

  20. Dataset Distillation Based on Saliency-Driven Prototype Alignment

    Jul 28, 2026Yawen Zou, Wenqi Cai, Guang Li +3Diffusion-Based Dataset DistillationPrototype Learning

  21. Making Single-Cell Data Distillation Auditable: Traceable Real-Cell Coresets via Discrete Min--Max Selection

    Jul 20, 2026Yaodi Luo, Peize He, Lingbei Meng +4Dataset DistillationCoreset Selection

  22. Dataset Distillation by Influence Matching

    Jul 18, 2026Haoru Tan, Wang Wang, Sitong Wu +5Dataset DistillationInfluence Functions

  23. Condensing Large-Scale Datasets Directly with Minimal Information Loss

    Jul 1, 2026Xinyi Shang, Peng Sun, Bei Shi +2Distribution ShiftDistribution Matching

  24. Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets

    Jun 29, 2026Kaifeng Chen, Lechao Cheng, Jiyang Li +6Dataset DistillationLearning with Noisy Labels

  25. Rank-Aware Hyperbolic Alignment for Vision-Language Dataset Distillation

    Jun 28, 2026Jongoh Jeong, Sun-Kyung Lee, Kuk-Jin YoonDataset DistillationVision-Language Alignment

  26. Structural Assessment for Understanding and Guiding Dataset Distillation in Discrete Token Space

    Jun 19, 2026Yue Cao, Jianyang Gu, Vyacheslav Kungurtsev +4Data SelectionDataset Distillation

  27. TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillation

    Jun 13, 2026Tri-Nhan Vo, Dang Nguyen, Sunil GuptaDataset Distillation