SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

Minjun Kim; Jongjin Kim; U Kang

cs.CVMar 19, 2026

SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning

Authors: Minjun Kim, Jongjin Kim, U Kang

Organizations: Seoul National University, Seoul, South Korea

Abstract

How can we accurately quantize a pre-trained model without any data? Quantization algorithms are widely used for deploying neural networks on resource-constrained edge devices. Zero-shot Quantization (ZSQ) addresses the crucial and practical scenario where training data are inaccessible for privacy or security reasons. However, three significant challenges hinder the performance of existing ZSQ methods: 1) noise in the synthetic dataset, 2) predictions based on off-target patterns, and the 3) misguidance by erroneous hard labels. In this paper, we propose SynQ (Synthesis-aware Fine-tuning for Zero-shot Quantization), a carefully designed ZSQ framework to overcome the limitations of existing methods. SynQ minimizes the noise from the generated samples by exploiting a low-pass filter. Then, SynQ trains the quantized model to improve accuracy by aligning its class activation map with the pre-trained model. Furthermore, SynQ mitigates misguidance from the pre-trained model's error by leveraging only soft labels for difficult samples. Extensive experiments show that SynQ provides the state-of-the-art accuracy, over existing ZSQ methods.

Figures & tables

Figure 1: Comparison between (a) real images in ImageNet dataset and (b) generated samples in the synthetic dataset from TexQ ( Chen et al., 2023 ) . Each set displays samples labeled as timber wolf, tobacco shop, aircraft carrier, and beaker. We present the average magnitude spectrum for a randomly selected batch of 256 images from each dataset, highlighting their distinct differences.

Figure 2: Grad-CAM ( Selvaraju et al., 2017 ) plot of the (a) input by the (b) pre-trained ResNet-18 model on ImageNet dataset, the (c) 3bit quantized model by TexQ, and the (d) 3bit quantized model by SynQ . While TexQ fails to capture the correct image region, SynQ captures the region closely matching the pre-trained model.

Figure 3: Error rates of pre-trained ResNet-20 on CIFAR-10 (yellow) and CIFAR-100 (green), and ResNet-18 on ImageNet (purple) by difficulty. Error rate rapidly grows as the difficulty exceeds 0.5.

Figure 4: Overall architecture of SynQ . Our main ideas are 1) low-pass filter, 2) alignment of class activation map, and 3) soft labels for difficult samples. See Section 4 for details.

Figure 5: Comparison of amplitude distribution among (a) ImageNet dataset, (b) synthetic dataset by TexQ, and (c) filtered samples. After filtering, the distribution closely aligns with that of real images.

Method	R-20 (CIFAR-10)		R-20 (CIFAR-100)		R-18 (ImageNet)		R-50 (ImageNet)		MV2 (ImageNet)
Method	W4A4	W3A3	W4A4	W3A3	W4A4	W3A3	W4A4	W3A3	W4A4	W3A3
Full Precision (W32A32)	93.89		70.33		71.47		77.73		73.03
GDFQ ( Xu et al., 2020 )	90.11	75.11	63.75	47.61	60.60	20.23	54.16	0.31	59.43	1.46
ARC ( Zhu et al., 2021 )	88.55	-	62.76	40.15	61.32	23.37	64.37	1.63	60.13	14.30
Qimera ( Choi et al., 2021 )	91.26	74.43	65.10	46.13	63.84	1.17	66.25	-	61.62	-
ARC + AIT ( Choi et al., 2022 )	90.49	-	61.05	41.34	65.73	-	68.27	-	66.47	-

Table 1: Zero-shot Quantization accuracy [%] of ResNet-20 (R-20) on CIFAR-10 and CIFAR-100, and ResNet-18 (R-20), ResNet-50 (R-50), and MobileNetV2 (MV2) on ImageNet. WBAB indicates that both weights and activations are quantized to Bbit. Note that SynQ achieves the highest accuracy.

Bits	Method	DeiT-Tiny	DeiT-Small	Swin-Tiny	Swin-Small	Average
Full Precision		72.21	79.85	81.35	83.20	79.15
W4A8	PSAQ-ViT ( Li et al., 2022 )	65.57 $\pm$ 0.10	72.04 $\pm$ 0.19	69.78 $\pm$ 1.67	75.03 $\pm$ 0.63	70.61
W4A8	SynQ (Proposed)	65.90 $\pm$ 0.07	72.28 $\pm$ 0.34	70.76 $\pm$ 1.61	75.82 $\pm$ 0.54	71.19
W8A8	PSAQ-ViT ( Li et al., 2022 )	71.56 $\pm$ 0.03	75.97 $\pm$ 0.20	73.54 $\pm$ 1.61	76.68 $\pm$ 0.53	74.44
W8A8	SynQ (Proposed)	71.74 $\pm$ 0.03	76.16 $\pm$ 0.29	74.11 $\pm$ 1.82	77.32 $\pm$ 0.59	74.83

Table 2: Zero-shot Quantization accuracy [%] of ViT models on ImageNet dataset. WBAB indicates that both weights and activations are quantized to Bbit. Note that SynQ shows consistent improvements in quantization performance across various models.

Figure 6: ZSQ accuracy comparison on different CAM techniques. See Section 5.3 for details.

I1	I2	I3	Accuracy [%]
Baseline			43.63
✓			49.43
	✓		48.26
		✓	46.42
✓	✓		51.24
✓		✓	50.81

Table 3: Ablation study on the main ideas of SynQ . All ideas contribute to the improved performance.

Figure 7: Hyperparameter analysis on (a) balancing hyperparameters $\lambda_{CE}$ and $\lambda_{CAM}$ , (b) difficulty threshold $\tau$ , and (c) filtering hyperparameter $D_{0}$ . See Section 5.5 for details.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$\theta$	A pre-trained model
$\theta^{q}$	The quantized model
$\{\mathbf{x}_{i}\}_{i=1}^{N}$	Synthetic samples
$\{\mathbf{y}_{i}\}_{i=1}^{N}$	One-hot encoded labels of synthetic samples
$q(\mathbf{x}_{i};\theta)$	Probability distribution of a sample $\mathbf{x}_{i}$ predicted by parameters $\theta$
$\delta(\mathbf{x}_{i};\theta)$	Difficulty of a sample $\mathbf{x}_{i}$ predicted by parameters $\theta$

Appendix

Table 4: Frequently used notations.

Figure 8: Runtime analysis of SynQ . The overhead from SynQ is marginal.

Panel of Figure 9 — Figure 9: Amplitude distribution of various real image datasets. See Appendix C.3 for details.

Panel of Figure 10 — Figure 10: Amplitude distribution of various synthetic datasets. See Appendix C.3 for details.

Method	Real dataset	Synthetic dataset
Method		Baseline	+ SynQ
IntraQ ( Zhong et al., 2022b )	3.1973	4.0251	3.2891 (-18.29%)
HAST ( Li et al., 2023a )	3.0976	3.9867	3.3133 (-16.89%)
TexQ ( Chen et al., 2023 )	2.9952	3.8436	3.1542 (-17.94%)

Appendix

Table 5: The average per-batch KL-divergence [

\times 10^{-2}

] between the saliency maps of the pre-trained and quantized models trained with different methods and datasets. See Section C.4 for details.

FA	I2	I1 & I3	Accuracy [%]
Baseline			43.63
✓			46.77 $\pm$ 0.30
	✓		48.26 $\pm$ 0.29
✓		✓	51.20 $\pm$ 0.30
	✓	✓	52.02 $\pm$ 0.34

Appendix

Table 6: Ablation study of two alignment techniques. CAM alignment shows superior performance.

Type	Method	W3A3			W4A4
Type	Method	Baseline	+ SynQ	Imp.	Baseline	+ SynQ	Imp.
Generator-based	GDFQ ( Xu et al., 2020 )	20.23	25.57 $\pm$ 0.28	5.34	60.60	61.23 $\pm$ 0.21	0.63
	Qimera ( Choi et al., 2021 )	1.17	32.34 $\pm$ 0.30	31.17	63.84	64.28 $\pm$ 0.18	0.44
	AdaDFQ ( Qian et al., 2023a )	38.10	40.56 $\pm$ 0.28	2.46	66.53	66.79 $\pm$ 0.22	0.26
Noise Optimization	IntraQ ( Zhong et al., 2022b )	45.51	50.44 $\pm$ 0.42	4.93	66.47	66.73 $\pm$ 0.19	0.26
	HAST ( Li et al., 2023a )	42.58	50.69 $\pm$ 0.38	8.11	66.91	67.19 $\pm$ 0.21	0.28

Appendix

Table 7: Comparison of ZSQ accuracy [%] of the ResNet-18 model pre-trained on the ImageNet dataset, before and after applying SynQ . Regardless of the baseline method and quantization bits, SynQ consistently improves the ZSQ accuracy.

Method	W2A2	W2A4	W3A3	W4A4	Average
Genie ( Jeon et al., 2023b )	54.01	65.10	66.84	69.66	63.90
+ SynQ (Proposed)	54.97 $\pm$ 0.35	65.88 $\pm$ 0.27	67.42 $\pm$ 0.21	69.88 $\pm$ 0.19	64.54

Appendix

Table 8: Zero-shot Quantization accuracy [%] of a ResNet-18 model on ImageNet quantized with Genie ( Jeon et al., 2023b ) as baseline. WPAQ indicates that weights and activations are quantized each into Pbit and Qbit, respectively. SynQ shows consistent improvements in quantization performance when applied to Genie, across various quantization bits.

Method	Baseline	Noise
Method		Gaussian	Speckle	S & P	Uniform
TexQ ( Chen et al., 2023 )	50.28	33.01 (-34.35%)	29.80 (-40.74%)	40.82 (-18.81%)	39.85 (-20.74%)
SynQ (Proposed)	52.02	43.29 (-16.78%)	35.62 (-31.53%)	46.81 (-10.01%)	45.11 (-13.28%)

Appendix

Table 9: 3bit ZSQ accuracy [%] of a ResNet-18 model pre-trained on ImageNet dataset when four different types of noise is injected into its synthetic dataset. See Section C.8 for details.

Figure 11: ZSQ accuracy regarding the size of synthetic dataset. See Appendix C.9 for details.

Panel of Figure 12 — Figure 12: Visualization of samples within the synthetic dataset before (left) and after (right) the low-pass filter, generated by (a, b) a ResNet-20 model pre-trained on CIFAR-10 dataset, (c, d) a ResNet-20 model pre-trained on CIFAR-100 dataset, and (e, f) a ResNet-18 model pre-trained on ImageNet dataset. See Appendix C.10 for details.

Panel of Figure 13 — Figure 13: Hyperparameter analysis of difficulty threshold $\tau$ with ResNet-20 (R-20) model pre-trained on CIFAR-10 and CIFAR-100 datasets, and MobileNetV2 (MV2) model pre-trained on ImageNet dataset. See Appendix C.11 for details.

Hyperparameter	Range
$\alpha_{1}$	[0.2, 0.4, 0.6, 0.8, 1]
$\alpha_{2}$	[0.01, 0.04, 0.1]
$\alpha^{C}$	[0.4, 1, 2.5]
$\lambda_{P}$	[0.25, 0.5, 0.75, 1, 1.5, 2, 3, 4]
$\lambda_{CE}$	[5, 5e-1, 5e-2, 5e-3]
$D_{0}$	[20, 40, 60, 80, 100]

Appendix

Table 10: Hyperparameter ranges for SynQ .

Neural Network Quantization

Neural network quantization represents weights, activations, and states with fewer bits to reduce memory use, computation, and deployment costs.

See all 404 papers

Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models
Jun 30, 2026Hyunho Lee, Kyomin Hwang, Hyeonjin Kim +3Quantization-Aware Training Synthetic Data Generation
Seoul National University, Seoul, South Korea · LG Electronics, Seoul, South Korea
Zero-Shot Quantization via Weight-Space Arithmetic
Apr 3, 2026Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli +3Quantization Robustness Quantization
Sapienza University of Rome · Sapienza University of Rome / Paradigma
JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization
Sep 27, 2026Kaicheng Yang, Kaisen Yang, Chunyu Liu +8Neural Network Quantization Diffusion Model Quantization
Shanghai Jiao Tong University · Tsinghua University · The Hong Kong Polytechnic University

Jun 30, 2026J/K move · Enter open · S save

Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models

Hyunho Lee, Kyomin Hwang, Hyeonjin Kim, Suyoung Kim+2

Seoul National University, Seoul, South Korea · LG Electronics, Seoul, South Korea

Quantization-Aware Training Synthetic Data Generation Object Detection Low-Bit Quantization Neural Network Quantization

Open paper AskPDF arXiv

With an increasing number of Object Detection (OD) models being deployed on edge devices, Zero-Shot Quantization for OD (ZSQ-OD) aims to quantize these models when access to the original training data is prohibited. Existing research on Zero-Shot Quantization-Aware Training (QAT) for OD synthesizes training sets through noise optimization. However, this approach struggles to maintain performance in low-bit regions. In this paper, we introduce GoodQ (Generative off-the-shelf models for object detector Quantization), a QAT pipeline that utilizes off-the-shelf generative models to construct a training set. We first identify three challenges that arise when introducing a generative model to the ZSQ-OD task: 1) each image contains dense information with multiple instances, 2) the class-wise distribution in the original dataset is imbalanced, and 3) the pseudo-labels assigned to the generated images can potentially act as noisy signals during QAT. GoodQ addresses these challenges by 1) introducing an Information-Dense Prompting strategy to generate multi-instance images, 2) applying Intrinsic Distribution-Aware Selection to match the pretrained class distribution, and 3) employing Teacher-guided Adaptive Noise Reduction to mitigate noise arising from the QAT process. Our framework achieves state-of-the-art performance in low-bit ZSQ (W4A4) and extends quantization to extreme bit-widths (W3A3). Furthermore, we conduct an extensive analysis to uncover the underlying factors contributing to the efficacy of GoodQ.

Abstract

Neural Network Quantization

Explore similar work

Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models

Zero-Shot Quantization via Weight-Space Arithmetic

JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization