Organizations: School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · College of Electronic Science and Technology, Aviation University of Air Force, Changchun, China, 130012.
Vision Transformers (ViTs) have demonstrated remarkable performance, yet their computational and memory demands pose significant challenges for deployment on mobile devices. Data-free quantization (DFQ) enables the compression of ViTs without access to their original training data. Existing DFQ methods often synthesize calibration samples by maximizing the entropy of patch similarity. However, similarity entropy describes the dispersion of similarity values without ensuring that the generated samples cover diverse feature directions, potentially limiting their effectiveness for quantization calibration. To address this limitation, we propose RankQ-ViT, a rank-guided DFQ framework for ViTs. We first observe that patch similarity matrices derived from Gaussian noise tend to have lower ranks than those derived from real images. Motivated by this observation, we introduce a block-aware self-contrastive objective that promotes feature diversity during synthetic data generation without explicitly computing matrix rank at each optimization step. We then select calibration samples based on their rank and semantic consistency and use them for block-wise reconstruction of the quantized model. Experiments on ImageNet, COCO, ModelNet40, and ShapeNet demonstrate the effectiveness of RankQ-ViT across multiple architectures and tasks, particularly under low-bit quantization. The code is available at https://github.com/Runqing-forMost/rankq-vit.git
Figures & tables
Fig. 1: Motivation and performance overview. (a) Rank distribution of the patch similarity matrix across different blocks, indicating that realistic images have higher ranks than noise ones. It motivates us to explicitly increase the ranks of the synthetic samples during sample generation. (b) Test accuracy (%) and time cost of data generation of RankQ-ViT, PSAQ-ViT v1 [ 16 ] , and PSAQ-ViT v2 [ 17 ] on DeiT-T and DeiT-S.
Fig. 2: Illustration of the distinction between patch-similarity entropy and feature-space rank. On the left, four normalized patch features lie in a two-dimensional plane and yield dispersed pairwise cosine similarities. On the right, four features span three dimensions despite having identical pairwise cosine similarities distributions. This example shows that dispersion of similarity values alone does not guarantee coverage of independent feature directions, motivating rank-aware synthetic data generation for quantization calibration.
XG
generated image
Y
label for XG
FXG(l)
features in the l -th block
S
patch similarity matrix
τ
temperature parameter
γ
A small positive number
Dp(⋅)
dropout with probability p
TABLE I: Key symbols in this work.
Fig. 3: Overview of RankQ-ViT. The workflow of RankQ-ViT involves two consecutive steps, namely block-aware rank promotion (the left panel) and block-wise training with sample selection (the right panel). First, the Gaussian noise images are encouraged to learn the high-rank similarity matrix by minimizing the Lrp . Then, generated samples with higher ranks (see Eq. 14 ) and minimal Jensen-Shannon Divergence (JSD) (see Eq. 15 ) are selected to form the training set D . Finally, the quantized model is forced to mimic the outputs of its full-precision version, recovering the accuracy degradation of quantization.
Fig. 4: The synthetic samples (224×224 pixels) shown in the figure are generated by PSAQ-ViT v1 and our proposed RankQ-ViT. The label associated with each generated image is indicated in the upper-right corner. The results demonstrate that images produced by PSAQ-ViT contain obscure semantic information, whereas those generated by our RankQ-ViT exhibit significantly stronger semantic content. Furthermore, the images from RankQ-ViT show richer texture details compared to those from PSAQ-ViT.
Model
Method
No Data
Data Num.
Precision
Top-1 Acc. (%)
Precision
Top-1 Acc. (%)
ViT-S
Full-Precision
-
-
W32/A32
81.39
W32/A32
81.39
Standard [ 16 ]
×
32
W4/A8
19.91
W8/A8
30.28
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A8
20.84
W8/A8
31.45
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A8
77.84
W8/A8
81.05
RepQ-ViT [ 19 ]
×
32
W4/A8
75.60
W8/A8
81.17
RankQ-ViT (ours)
✓
32
W4/A8
78.76 ±0.12
W8/A8
81.19 ±0.08
TABLE II: Quantization results on the ImageNet dataset with W4/A8 and W8/A8 quantization. “No Data” means that no realistic images are used. “Data Num.” means the number of images used for calibration. “Precision” denotes the bit-width of weights and activations. The top-1 accuracy (%) is reported as the evaluation metric.
Model
Method
No Data
Data Num.
Precision
Top-1 Acc. (%)
Precision
Top-1 Acc. (%)
ViT-S
Full-Precision
-
-
W32/A32
81.39
W32/A32
81.39
Standard
×
32
W4/A4
39.42
W5/A5
71.78
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A4
0.12
W5/A5
24.78
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A4
9.78
W5/A5
60.12
RankQ-ViT (ours)
✓
32
W4/A4
44.06 ±0.18
W5/A5
68.07 ±0.17
ViT-B
Full-Precision
-
-
W32/A32
84.53
W32/A32
84.53
TABLE III: Quantization results on the ImageNet dataset with W4/A4 and W5/A5 quantization.
Datasets
Method
No Data
Data Num.
Precision
mAcc (%)
OA (%)
Precision
mAcc (%)
OA (%)
ModelNet40
Full-Precision
-
-
W32/A32
83.99
88.00
W32/A32
83.99
88.00
Gaussian
✓
32
W4/A8
77.36
79.16
W8/A8
82.01
82.50
Real
×
32
W4/A8
80.00
84.51
W8/A8
83.67
87.18
Baseline
✓
32
W4/A8
79.85
83.98
W8/A8
83.26
87.00
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A8
79.60
83.65
W8/A8
83.21
87.16
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A8
80.00
84.65
W8/A8
83.28
87.01
TABLE IV: Quantization results of point cloud classification. “mACC” and “OA” denote the mean of class-wise accuracy (%) and the overall accuracy (%), respectively.
Model
Method
Prec.
AP box
AP mask
Swin-S Cascade Mask R-CNN
FP
W32/A32
51.9
48.0
Standard
W4/A8
47.3
40.9
Gaussian
W4/A8
38.1
35.2
PSAQ v1 [ 16 ]
W4/A8
42.6
36.9
PSAQ v2 [ 17 ]
W4/A8
47.9
41.4
RankQ-ViT
W4/A8
48.3 ±0.02
41.7 ±0.01
TABLE V: Quantization results on COCO dataset. “ APbox ” means the box average precision for object detection, and “ APmask ” means the mask average precision for instance segmentation.
Table 10
Fig. 5: Hyperparameter sensitivity analysis on W4/A8 quantization of DeiT-S. (a) Effect of different α values. (b) Effect of different Nsel values.
Fig. 6: Results of data generated by DeiT-T for other model quantization. W4/A8 quantization is used here for comparison. The results clearly indicate that our RankQ-ViT has superior cross-architecture generalization ability.
Fig. 7: Speed (FPS) and peak memory (MB) usage during inference on Intel 12th i9-12900K CPU and NVIDIA RTX-4090 GPU. (a) FPS comparison between FP32 and INT8 models. (b) The Peak GPU memory during inference. These results show that quantization significantly reduces the inference cost.
Department of Automation, Tsinghua University, China. · School of Computer Science and Information Engineering, Hefei University of Technology, China. · Li Auto, China.