Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillation

Muquan Li; Hang Gou; Yingyi Ma; Rongzheng Wang; Ke Qin; Tao He

cs.CVFeb 27, 2026

Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillation

Authors: Muquan Li, Hang Gou, Yingyi Ma, Rongzheng Wang, Ke Qin, Tao He

Organizations: The Laboratory of Intelligent Collaborative Computing of UESTC

Abstract

Decoupled dataset distillation (DD) compresses large corpora into a few synthetic images by matching a frozen teacher's statistics. However, current residual-matching pipelines rely on static real patches, creating a fit-complexity gap and a pull-to-anchor effect that reduce intra-class diversity and hurt generalization. To address these issues, we introduce RETA -- a Retrieval and Topology Alignment framework for decoupled DD. First, Dynamic Retrieval Connection (DRC) selects a real patch from a prebuilt pool by minimizing a fit-complexity score in teacher feature space; the chosen patch is injected via a residual connection to tighten feature fit while controlling injected complexity. Second, Persistent Topology Alignment (PTA) regularizes synthesis with persistent homology: we build a mutual k-NN feature graph, compute persistence images of components and loops, and penalize topology discrepancies between real and synthetic sets, mitigating pull-to-anchor effect. Across CIFAR-100, Tiny-ImageNet, ImageNet-1K, and multiple ImageNet subsets, RETA consistently outperforms various baselines under comparable time and memory, especially reaching 64.3% top-1 accuracy on ImageNet-1K with ResNet-18 at 50 images per class, +3.1% over the best prior.

Figures & tables

Figure 1: Comparison between existing pipelines and our proposed RETA. (a) Decoupled methods squeeze the original dataset into a compact set. (b) Previous residual matching methods [ 6 ] connect a fixed real patch to the distilled images at every step. (c) RETA augments residual matching with Dynamic Retrieval Connection (DRC) and Persistent Topology Alignment (PTA).

Figure 2: Overview of RETA. Dynamic Retrieval Connection (top) computes a fit–complexity score to adaptively retrieve real patches as anchors for residual matching, while Persistent Topology Alignment (bottom) aligns the persistent diagrams of real and synthetic features. The two modules are jointly optimized to produce synthetic datasets that retain both accuracy and topology-aware structure.

Architecture	#Params	RDED	EDC	CV-DD	FADRM+	RETA
EfficientNet-B0	39.6M	42.8	51.1	43.2	51.9	53.6
MobileNetV2	3.4M	34.4	45.0	39.0	45.5	47.4
ShuffleNetV2-0.5x	1.4M	19.6	29.8	27.4	30.2	33.6
Swin-Tiny	28.0M	29.2	38.3	–	39.1	41.5
Wide ResNet50-2	68.9M	50.0	–	53.9	59.1	60.7
DenseNet121	8.0M	49.4	–	50.9	55.4	56.8

Table 2: Test accuracies (%) on ImageNet-1K with IPC=

10

for cross-architecture generalization. The bold font indicates the best performance achieved on each architecture.

Panel of Figure 3 — Figure 3: Visualization of images distilled by FADRM and RETA. The synthetic images on Tiny-ImageNet, ImageNette and ImageNet- $1$ K are demonstrated here.

Panel of Figure 4 — Figure 4: Ablation studies for hyperparameters on ImageNet- $1$ K. Curves show the mean test accuracy, and the error bars denote one standard deviation over five independent runs.

DRC	PTA	CIFAR-100	Tiny-ImageNet	ImageNette	ImageNet-1K
×	×	67.9	52.8	69.0	50.9
✓	×	69.0 ( +1.1 )	54.5 ( +1.7 )	70.9 ( +1.9 )	51.8 ( +0.9 )
×	✓	68.5 ( +0.6 )	53.7 ( +0.9 )	69.8 ( +0.8 )	51.6 ( +0.7 )
✓	✓	70.3 ( +2.4 )	56.2 ( +3.4 )	72.5 ( +3.5 )	53.2 ( +2.3 )

Table 3: Ablation study on the contributions of different components in our method. The experiments are conducted on CIFAR-

100

, Tiny-ImageNet, ImageNette and ImageNet-

1

K, with IPC=

10

for all datasets using ResNet-

18

. All results are obtained under optimal hyperparameter settings, and improvements are marked with blue numbers in parentheses.

Figure 5: Computational efficiency comparison between baseline methods and RETA when distilling ImageNet-

1

k. The time cost is measured in seconds, representing the duration required to generate a single image on a single RTX

4090

GPU.

Datasets	MTT [ 3 ]	IDC [ 21 ]	FADRM+ [ 6 ]	RETA
ImageNette-C	28.0±1.6	34.5±0.6	37.4±0.4	40.1±0.8
ImageWoof-C	20.8±1.0	18.7±0.4	19.9±0.7	22.5±0.5
ImageFruit-C	22.7±1.1	28.5±0.9	30.3±0.8	34.4±0.6
ImageYellow-C	25.6±1.7	36.8±1.4	37.1±1.3	41.3±1.1
ImageMeow-C	23.2±1.1	22.2±1.2	25.2±1.1	28.2±0.6
ImageSquawk-C	25.7±0.8	26.8±0.5	27.3±0.6	29.9±0.5

Table 4: Accuracy (%) on ImageNet-Subset-C with

\text{IPC}{=}1

. Each score averages over

15

corruption types and

5

severities. Bold denotes the best result for each subset.

Figure 6: Application of continual learning on ImageNet-1k.

Diffusion-Based Dataset Distillation

See all 14 papers

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization
Feb 17, 2026Muhammad J. Alahmadi, Peng Gao, Feiyi Wang +1Dataset Distillation
North Carolina State University · King Abdulaziz University · Oak Ridge National Laboratory
FD $^2$ : A Dedicated Framework for Fine-Grained Dataset Distillation
Mar 26, 2026Hongxu Ma, Guang Li, Shijie Wang +5Fine-Grained Image Classification Dataset Distillation
Zhejiang University · Hokkaido University · The University of Queensland +2
PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty
Sep 28, 2026Hongxu Ma, Guang Li, Shijie Wang +6Distribution Matching Dataset Distillation
Zhejiang University · Hokkaido University · The University of Queensland +3

Feb 17, 2026J/K move · Enter open · S save

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization

Muhammad J. Alahmadi, Peng Gao, Feiyi Wang, Dongkuan Xu

North Carolina State University · King Abdulaziz University · Oak Ridge National Laboratory

Dataset Distillation

Open paper AskPDF arXiv

Dataset distillation compresses the original data into compact synthetic datasets, reducing training time and storage while retaining model performance, enabling deployment under limited resources. Although recent decoupling-based distillation methods enable dataset distillation at large scale, they continue to face an efficiency gap: optimization-based decoupling methods achieve higher accuracy but demand intensive computation, whereas optimization-free decoupling methods are efficient but sacrifice accuracy. To overcome this trade-off, we propose Exploration--Exploitation Distillation (E

^2

D), a simple, practical method that minimizes redundant computation through an efficient pipeline that begins with full-image initialization to preserve semantic integrity and feature diversity. It then uses a two-phase optimization strategy: an exploration phase that performs uniform updates and identifies high-loss regions, and an exploitation phase that focuses updates on these regions to accelerate convergence. We evaluate E

^2

D on large-scale benchmarks, surpassing the state-of-the-art on ImageNet-1K while being

18\times

faster, and on ImageNet-21K, our method substantially improves accuracy while remaining

4.3\times

faster. These results demonstrate that targeted, redundancy-reducing updates, rather than brute-force optimization, bridge the gap between accuracy and efficiency in large-scale dataset distillation. Code is available at https://github.com/ncsu-dk-lab/E2D.

Abstract

Diffusion-Based Dataset Distillation

Explore similar work

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization

FD2^22: A Dedicated Framework for Fine-Grained Dataset Distillation

PSM: Dataset Distillation Based on Precise Statistical Matching by Difficulty

FD $^2$ : A Dedicated Framework for Fine-Grained Dataset Distillation