Dataset Distillation

Follow this topic Ask about this topic

Dataset distillation aims to replace a large training set with a compact set of examples that lets models learn effectively while reducing storage, communication, or training costs. Common approaches match gradients or training trajectories, align feature or data distributions, select representative prototypes, or generate examples with diffusion models; newer work also targets text, graphs, time series, video, and multimodal data. Researchers additionally study how to compress labels and bits, and how to preserve robustness, fairness, and performance across model architectures. A central open tension is whether distilled sets reliably outperform simpler random subsets or coresets, especially at large scale, while balancing downstream accuracy against faithful coverage of the original data.

Latest papers

67 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization
Feb 17, 2026Muhammad J. Alahmadi, Peng Gao, Feiyi Wang +1Dataset Distillation
North Carolina State University · King Abdulaziz University · Oak Ridge National Laboratory
Efficient Dataset Distillation for Pre-Trained Self-Supervised Models via Statistical Flow Matching
Feb 5, 2026Qianxin Xia, Jiawei Du, Yuhan Zhang +6Flow Matching Dataset Distillation
University of Electronic Science and Technology of China, Chengdu, China · CFAR, Singapore · SWJTU, Chengdu, China
Toward Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion
Jan 22, 2026Yonghao Xu, Pedram Ghamisi, Qihao WengRemote Sensing Image Understanding Diffusion Model Distillation
Computer Vision and Learning Systems, Department of Electrical Engineering, Linköping University, 581 83 Linköping, Sweden · Helmholtz-Zentrum Dresden-Rossendorf, Responsible AI Group, 09599 Freiberg, Germany · University of Iceland, Faculty of Electrical and Computer Engineering, Reykjavik 102, Iceland +1
Foreground-Aware Dataset Distillation via Dual-Path Patch Selection
Jan 6, 2026Mingzhuo Li, Guang Li, Longzhen Li +4Dataset Distillation
Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-Ku, Sapporo, 060-0814, Japan · Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-Ku, Sapporo, 060-0814, Japan
A Discrepancy-Based Perspective on Dataset Condensation
Sep 12, 2025Tong Chen, Raghavendra SelvanDistribution Matching Dataset Distillation
Department of Computer Science, University of Copenhagen, Denmark
OD3: Optimization-free Dataset Distillation for Object Detection
Jun 2, 2025Salwa K. Al Khatib, Ahmed ElHagry, Shitong Shao +1Dataset Distillation
Mohamed bin Zayed University of Artificial Intelligence · Hong Kong University of Science and Technology (Guangzhou)
Towards Consistent and Efficient Dataset Distillation via Diffusion-Driven Selection
Dec 13, 2024Xinhao Zhong, Shuoyang Sun, Zhaoyang Xu +4Diffusion-Based Dataset Distillation Dataset Distillation
Harbin Institute of Technology, Shenzhen · Shanghai Jiao Tong University

Feb 17, 2026J/K move · Enter open · S save

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization

Muhammad J. Alahmadi, Peng Gao, Feiyi Wang, Dongkuan Xu

North Carolina State University · King Abdulaziz University · Oak Ridge National Laboratory

Dataset Distillation

Open paper AskPDF arXiv

Dataset distillation compresses the original data into compact synthetic datasets, reducing training time and storage while retaining model performance, enabling deployment under limited resources. Although recent decoupling-based distillation methods enable dataset distillation at large scale, they continue to face an efficiency gap: optimization-based decoupling methods achieve higher accuracy but demand intensive computation, whereas optimization-free decoupling methods are efficient but sacrifice accuracy. To overcome this trade-off, we propose Exploration--Exploitation Distillation (E

^2

D), a simple, practical method that minimizes redundant computation through an efficient pipeline that begins with full-image initialization to preserve semantic integrity and feature diversity. It then uses a two-phase optimization strategy: an exploration phase that performs uniform updates and identifies high-loss regions, and an exploitation phase that focuses updates on these regions to accelerate convergence. We evaluate E

^2

D on large-scale benchmarks, surpassing the state-of-the-art on ImageNet-1K while being

18\times

faster, and on ImageNet-21K, our method substantially improves accuracy while remaining

4.3\times

faster. These results demonstrate that targeted, redundancy-reducing updates, rather than brute-force optimization, bridge the gap between accuracy and efficiency in large-scale dataset distillation. Code is available at https://github.com/ncsu-dk-lab/E2D.