DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation

Chuixuan Fan; Guang Li; Shijie Wang; Dongzhan Zhou; Baoli Sun; Takahiro Ogawa; Miki Haseyama; Zhihui Wang

cs.CVAug 26, 2026

DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation

Authors: Chuixuan Fan, Guang Li, Shijie Wang, Dongzhan Zhou, Baoli Sun, Takahiro Ogawa, Miki Haseyama, Zhihui Wang

Organizations: University of Science and Technology of China · Hokkaido University · The University of Queensland · Shanghai AI Laboratory ·

Abstract

Dataset distillation compresses a large training set into a compact synthetic set while preserving its downstream utility. However, existing methods primarily preserve global image statistics and may overlook the localized evidence essential for fine-grained visual classification (FGVC), such as object parts, subtle textures, and region-specific structures. We formulate fine-grained dataset distillation as budgeted discriminative-evidence preservation and propose Discriminative Evidence Composition (DeCO). DeCO uses attention rollout from a pretrained TransFG teacher to identify informative patches, applies spatial diversification to reduce redundant coverage, and organizes the resulting regions into class-wise evidence banks. Multiple same-class regions are then packed into compact grid-composed images. The teacher is used only for dataset construction, whereas downstream students are trained with standard hard-label supervision without teacher logits. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars show that DeCO consistently outperforms representative coreset and dataset-distillation baselines under different IPC budgets.

Figures & tables

Figure 1: Overview of DeCO. A TransFG teacher scores patches through attention rollout. Spatially diverse high-response regions are collected into class-wise evidence banks and packed into grid-composed images for hard-label student training.

Dataset	IPC	Uniform	RDED	SRe 2 L++ †	FADRM+ †	DeCO rand	DeCO
CUB-200-2011	1	1.45	38.25	53.44	54.80	45.31	65.53
	3	1.97	52.56	60.04	64.06	74.60	86.83
	5	2.55	63.85	63.50	66.40	78.98	87.99
FGVC-Aircraft	1	1.82	22.11	52.60	55.02	56.02	66.04
	3	3.80	36.39	66.63	72.90	76.17	87.49
	5	4.35	38.56	68.35	74.01	78.62	88.09

Table 1: Top-1 accuracy (%). Unmarked results are means of at least three runs under a unified TransFG hard-label protocol;

\dagger

denotes reported results under their original protocols.

Figure 2: Student attention visualization. Grad-CAM Selvaraju and others (2017) maps of students trained on the RDED and DeCO distilled sets for three CUB-200-2011 categories. Each pair shows an input image and its activation map; DeCO-trained students generally exhibit more localized responses on discriminative object regions.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Mean top-1 accuracy under different cropped-region area ratios on CUB-200-2011 and FGVC-Aircraft.

Dataset	ViT-B/16	ViT-B/32
CUB-200-2011	65.4	65.2
FGVC-Aircraft	66.0	66.1

Appendix

Table 2: Mean top-1 accuracy (%) under different teacher patch sizes.

Rollout Depth $l$	CUB-200-2011	FGVC-Aircraft
1	64.12	65.18
3	64.93	65.71
5	65.38	65.96
7	65.47	66.02
9	65.51	66.01
11	65.53	66.04

Appendix

Table 3: Sensitivity to attention rollout depth. Results are mean top-1 accuracy (%) at IPC

=1

Figure 4: Qualitative comparison with RDED. Left: distilled images produced by RDED and DeCO using the same four-region budget. DeCO retains more localized class-specific evidence, whereas RDED may preserve larger background or weakly localized regions. Right: top-20 predicted class probabilities for a representative CUB-200-2011 category.

Figure 5: Visualization of distilled samples from the first 100 classes of CUB-200-2011 at IPC=1.

Figure 6: Visualization of distilled samples from the last 100 classes of CUB-200-2011 at IPC=1.

Figure 7: Visualization of distilled samples from the first 100 classes of Stanford Cars at IPC=1.

Figure 8: Visualization of distilled samples from the remaining 96 classes of Stanford Cars at IPC=1.

Figure 9: Visualization of distilled samples from all 100 classes of FGVC-Aircraft at IPC=1.

Dataset Distillation

Dataset distillation compresses training data into a small synthetic or selected set that retains the information needed to train useful models.

See all 67 papers

FD $^2$ : A Dedicated Framework for Fine-Grained Dataset Distillation
Mar 26, 2026Hongxu Ma, Guang Li, Shijie Wang +5Fine-Grained Image Classification Dataset Distillation
Zhejiang University · Hokkaido University · The University of Queensland +2
Dataset Distillation Based on Saliency-Driven Prototype Alignment
Jul 28, 2026Yawen Zou, Wenqi Cai, Guang Li +3Diffusion-Based Dataset Distillation Prototype Learning
University of Toyama · Hokkaido University · University of Fukui
Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization
Feb 17, 2026Muhammad J. Alahmadi, Peng Gao, Feiyi Wang +1Dataset Distillation
North Carolina State University · King Abdulaziz University · Oak Ridge National Laboratory

Mar 26, 2026J/K move · Enter open · S save

FD $^2$ : A Dedicated Framework for Fine-Grained Dataset Distillation

Hongxu Ma, Guang Li, Shijie Wang, Dongzhan Zhou+4

Zhejiang University · Hokkaido University · The University of Queensland+2

Fine-Grained Image Classification Dataset Distillation Image Classification

Open paper AskPDF arXiv

Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks. Decoupled DD further improves efficiency by splitting the pipeline into pretraining, sample distillation, and soft-label generation. However, existing decoupled methods largely rely on coarse class-label supervision and optimize samples within each class in a nearly identical manner. On fine-grained datasets, this often yields distilled samples that (i) retain large intra-class variation with subtle inter-class differences and (ii) become overly similar within the same class, limiting localized discriminative cues and hurting recognition. To solve the above-mentioned problems, we propose FD

^{2}

, a dedicated framework for Fine-grained Dataset Distillation. FD

^{2}

localizes discriminative regions and constructs fine-grained representations for distillation. During pretraining, counterfactual attention learning aggregates discriminative representations to update class prototypes. During distillation, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others, and a similarity constraint diversifies attention across same-class samples. Experiments on multiple fine-grained and general datasets show that FD

^{2}

integrates seamlessly with decoupled DD and improves performance in most settings, indicating strong transferability. Code is available at https://github.com/Guang000/FD2.

Abstract

Dataset Distillation

Explore similar work

FD2^22: A Dedicated Framework for Fine-Grained Dataset Distillation

Dataset Distillation Based on Saliency-Driven Prototype Alignment

Accelerating Large-Scale Dataset Distillation via Exploration-Exploitation Optimization

FD $^2$ : A Dedicated Framework for Fine-Grained Dataset Distillation