cs.CVMar 7, 2026

Post Training Quantization for Efficient Dataset Condensation

Authors: Linh-Tam Tran, Sung-Ho Bae

Organizations: Kyung Hee University, Republic of Korea

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Dataset Condensation (DC) distills knowledge from large datasets into smaller ones, accelerating training and reducing storage requirements. However, despite notable progress, prior methods have largely overlooked the potential of quantization for further reducing storage costs. In this paper, we take the first step to explore post-training quantization in dataset condensation, demonstrating its effectiveness in reducing storage size while maintaining representation quality without requiring expensive training cost. However, we find that at extremely low bit-widths (e.g., 2-bit), conventional quantization leads to substantial degradation in representation quality, negatively impacting the networks trained on these data. To address this, we propose a novel patch-based post-training quantization approach that ensures localized quantization with minimal loss of information. To reduce the overhead of quantization parameters, especially for small patch sizes, we employ quantization-aware clustering to identify similar patches and subsequently aggregate them for efficient quantization. Furthermore, we introduce a refinement module to align the distribution between original images and their dequantized counterparts, compensating for quantization errors. Our method is a plug-and-play framework that can be applied to synthetic images generated by various DC methods. Extensive experiments across diverse benchmarks including CIFAR-10/100, Tiny ImageNet, and ImageNet subsets demonstrate that our method consistently outperforms prior works under the same storage constraints. Notably, our method nearly doubles the test accuracy of existing methods at extreme compression regimes (e.g., 26.0% →\rightarrow 54.1% for DM at IPC=1), while operating directly on 2-bit images without additional distillation.

Figures & tables

Figure 1: Performance comparison when applying our framework to DSA ( Zhao and Bilen 2021 ) , DM ( Zhao and Bilen 2022 ) , and DATM ( Guo et al. 2024 ) . Notably, our framework doubles the performance for DSA and DM at a budget of 1 Image Per Class (IPC), demonstrating its effectiveness in extremely low-storage scenarios.
MethodRedundancyStrategyBitCost
IDCSpatialSpatial Downsample3232Med.
AutoPaletteColorPalette Reduction3232High
DDiFFeatureNeural Field3232High
OursBit-levelPatch Quantization2Low
Table 1: Comparison of redundancy reduction strategies in PDC.
Figure 2: Overview of the proposed patch-based quantization framework for dataset condensation. (I) Synthetic images are first refined using a quantization-aware loss. (II) Patches are grouped based on quantization parameters via kk -means clustering. (III) Each group is quantized with shared parameters, and the result is entropy encoded to produce the final compressed dataset.
Figure 3: Visualization comparison of quantization strategies on synthetic images: (a) original images, (b) Median Cut ( Heckbert 1982 ) , (c) asymmetric quantization, and (d) our patch-based quantization. All quantized images use 2-bit precision. (e) shows distortion (MSE) versus bit-width across methods. Images are generated by DM ( Zhao and Bilen 2022 ) on CIFAR-10.
DatasetCIFAR-10CIFAR-100Tiny ImageNet
IPC1105011050110
DistillationDD–36.8 ±1.2{}_{\pm\text{1.2}}––––––
DM26.0 ±0.8{}_{\pm\text{0.8}}48.9 ±0.6{}_{\pm\text{0.6}}63.0 ±0.4{}_{\pm\text{0.4}}11.4 ±0.3{}_{\pm\text{0.3}}29.7 ±0.3{}_{\pm\text{0.3}}43.6 ±0.4{}_{\pm\text{0.4}}3.9 ±0.2{}_{\pm\text{0.2}}12.9 ±0.4{}_{\pm\text{0.4}}
DSA28.8 ±0.7{}_{\pm\text{0.7}}52.1 ±0.5{}_{\pm\text{0.5}}60.6 ±0.5{}_{\pm\text{0.5}}13.9 ±0.3{}_{\pm\text{0.3}}32.3 ±0.3{}_{\pm\text{0.3}}42.8 ±0.4{}_{\pm\text{0.4}}––
MTT46.3 ±0.8{}_{\pm\text{0.8}}65.3 ±0.7{}_{\pm\text{0.7}}71.6 ±0.2{}_{\pm\text{0.2}}24.3 ±0.3{}_{\pm\text{0.3}}40.1 ±0.4{}_{\pm\text{0.4}}47.7 ±0.2{}_{\pm\text{0.2}}8.8 ±0.3{}_{\pm\text{0.3}}23.2 ±0.2{}_{\pm\text{0.2}}
DATM46.9 ±0.5{}_{\pm\text{0.5}}66.8 ±0.2{}_{\pm\text{0.2}}76.1 ±0.3{}_{\pm\text{0.3}}27.9 ±0.2{}_{\pm\text{0.2}}47.2 ±0.4{}_{\pm\text{0.4}}55.0 ±0.2{}_{\pm\text{0.2}}17.1 ±0.3{}_{\pm\text{0.3}}31.1 ±0.3{}_{\pm\text{0.3}}
Table 2: Test accuracy (%) comparison on CIFAR-10, CIFAR-100, and Tiny ImageNet. Results are reported in mean ±std{}_{\pm\text{std}} format. Experiments on CIFAR-100 with IPC=50 are omitted, as this would require 1,000 images per class under a compression rate of 20, which exceeds the original 500 images per class.
Table 6
Table 7
MethodNetwork
ConvNetAlexNetVGG11ResNet18
DC44.9 ±0.5{}_{\pm\text{0.5}}22.4 ±1.4{}_{\pm\text{1.4}}35.9 ±0.7{}_{\pm\text{0.7}}18.4 ±0.4{}_{\pm\text{0.4}}
MTT65.3 ±0.7{}_{\pm\text{0.7}}34.2 ±0.6{}_{\pm\text{0.6}}50.3 ±0.8{}_{\pm\text{0.8}}46.6 ±0.6{}_{\pm\text{0.6}}
DATM66.8 ±0.2{}_{\pm\text{0.2}}32.7 ±3.8{}_{\pm\text{3.8}}38.8 ±2.1{}_{\pm\text{2.1}}51.4 ±1.7{}_{\pm\text{1.7}}
FRePo65.5 ±0.4{}_{\pm\text{0.4}}61.9 ±0.7{}_{\pm\text{0.7}}59.4 ±0.7{}_{\pm\text{0.7}}58.1 ±0.6{}_{\pm\text{0.6}}
Spectral73.4 ±0.2{}_{\pm\text{0.2}}71.4 ±0.3{}_{\pm\text{0.3}}67.8 ±0.2{}_{\pm\text{0.2}}64.9 ±1.3{}_{\pm\text{1.3}}
Table 7: Cross-architecture performance comparison between our framework and other methods at IPC=10.
GAQRefinementECCIFAR-10I-Nette
✗✗✗71.8 ±0.3{}_{\pm\text{0.3}}75.2 ±0.8{}_{\pm\text{0.8}}
✓✗✗76.1 ±0.3{}_{\pm\text{0.3}}76.5 ±1.1{}_{\pm\text{1.1}}
✓✓✗77.2 ±0.3{}_{\pm\text{0.3}}77.2 ±1.0{}_{\pm\text{1.0}}
✓✓✓79.0 ±0.3{}_{\pm\text{0.3}}81.1 ±1.2{}_{\pm\text{1.2}}
Table 8: Effect of each component on final performance. The first row shows the baseline using global asymmetric quantization. IPC is set to 10 for both CIFAR-10 and I-Nette.
Refinement before GAQ✓✗✓
Refinement after GAQ✗✓✓
Test Accuracy (%)68.9 ±0.4{}_{\pm\text{0.4}}68.7 ±0.5{}_{\pm\text{0.5}}68.9 ±0.3{}_{\pm\text{0.3}}
Table 9: Test accuracy on CIFAR-10 at IPC=1 with refinement applied at different stages.
Figure 4: Measured test accuracy (%) and storage across different numbers of groups. The vertical dotted line marks the maximum allowable storage under the given budget.
Figure 5: Visualization of (a) original synthetic images and their quantized versions using (b) Median Cut, (c) Asymmetric Quantization (AQ), and (d) Group-Aware Quantization (GAQ). While Median Cut preserves texture but distorts color, and AQ preserves color but loses fine details, GAQ achieves a better balance between texture and color fidelity.
More in this topic

Dataset Condensation

Dataset condensation creates tiny training sets that preserve the learning value of much larger datasets.

See all 19 papers

Explore similar work

CardsList
Feb 24, 2026

Dataset Color Quantization: A Training-Oriented Framework for Dataset-Level Compression

Large-scale image datasets are fundamental to deep learning, but their high storage demands pose challenges for deployment in resource-constrained environments. While existing approaches reduce dataset size by discarding samples, they often ignore the significant redundancy within each image -- particularly in the color space. To address this, we propose Dataset Color Quantization (DCQ), a unified framework that compresses visual datasets by reducing color-space redundancy while preserving information crucial for model training. DCQ achieves this by enforcing consistent palette representations across similar images, selectively retaining semantically important colors guided by model perception, and maintaining structural details necessary for effective feature learning. Extensive experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K show that DCQ significantly improves training performance under aggressive compression, offering a scalable and robust solution for dataset-level storage reduction.

Chenyue Yu, Lingao Xiao, Jinhong Deng +2

National University of Singapore · CFAR, Agency for Science, Technology and Research, Singapore · IHPC, Agency for Science, Technology and Research, Singapore +2

Topic pending
AskPDF
Mar 2, 2026

From Fewer Samples to Fewer Bits: Reframing Dataset Distillation as Joint Optimization of Precision and Compactness

Dataset Distillation (DD) compresses large datasets into compact synthetic ones that maintain training performance. However, current methods mainly target sample reduction, with limited consideration of data precision and its impact on efficiency. We propose Quantization-aware Dataset Distillation (QuADD), a unified framework that jointly optimizes dataset compactness and precision under fixed bit budgets. QuADD integrates a differentiable quantization module within the distillation loop, enabling end-to-end co-optimization of synthetic samples and quantization parameters. Guided by the rate-distortion perspective, we empirically analyze how bit allocation between sample count and precision influences learning performance. Our framework supports both uniform and adaptive non-uniform quantization, where the latter learns quantization levels from data to represent information-dense regions better. Experiments on image classification and 3GPP beam management tasks show that QuADD surpasses existing DD and post-quantized baselines in accuracy per bit, establishing a new standard for information-efficient dataset distillation.

My H. Dinh, Aditya Sant, Akshay Malhotra +2

InterDigital Communications, Inc.

Quantization-Aware DistillationDataset DistillationRate-Distortion Optimization
AskPDF
Mar 20, 2026

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but are agnostic to the base model, allowing quantization noise to disproportionately corrupt the small-magnitude parameter deltas (ΔWΔW) that encode post-training behavior -- an effect we analyze through the lens of quantization as implicit regularization. DAQ replaces reconstruction-based objectives with two delta-aware metrics -- Sign Preservation Rate and Cosine Similarity -- that directly optimize for directional fidelity of ΔWΔW, requiring only the base and post-trained weight matrices. In a pilot FP8 study, DAQ recovers style-specific capabilities lost under standard quantization while maintaining general performance.

Xiaoming Yu, Shize Tang, Guanghua Yu +4

Yuanbao & Hunyuan AI Infra Team

QuantizationLLM QuantizationNeural Network QuantizationPost-Training QuantizationWeight-Only Quantization
AskPDF