Organizations: School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · College of Electronic Science and Technology, Aviation University of Air Force, Changchun, China, 130012.
Vision Transformers (ViTs) have demonstrated remarkable performance, yet their computational and memory demands pose significant challenges for deployment on mobile devices. Data-free quantization (DFQ) enables the compression of ViTs without access to their original training data. Existing DFQ methods often synthesize calibration samples by maximizing the entropy of patch similarity. However, similarity entropy describes the dispersion of similarity values without ensuring that the generated samples cover diverse feature directions, potentially limiting their effectiveness for quantization calibration. To address this limitation, we propose RankQ-ViT, a rank-guided DFQ framework for ViTs. We first observe that patch similarity matrices derived from Gaussian noise tend to have lower ranks than those derived from real images. Motivated by this observation, we introduce a block-aware self-contrastive objective that promotes feature diversity during synthetic data generation without explicitly computing matrix rank at each optimization step. We then select calibration samples based on their rank and semantic consistency and use them for block-wise reconstruction of the quantized model. Experiments on ImageNet, COCO, ModelNet40, and ShapeNet demonstrate the effectiveness of RankQ-ViT across multiple architectures and tasks, particularly under low-bit quantization. The code is available at https://github.com/Runqing-forMost/rankq-vit.git
Figures & tables
Fig. 1: Motivation and performance overview. (a) Rank distribution of the patch similarity matrix across different blocks, indicating that realistic images have higher ranks than noise ones. It motivates us to explicitly increase the ranks of the synthetic samples during sample generation. (b) Test accuracy (%) and time cost of data generation of RankQ-ViT, PSAQ-ViT v1 [ 16 ] , and PSAQ-ViT v2 [ 17 ] on DeiT-T and DeiT-S.
Fig. 2: Illustration of the distinction between patch-similarity entropy and feature-space rank. On the left, four normalized patch features lie in a two-dimensional plane and yield dispersed pairwise cosine similarities. On the right, four features span three dimensions despite having identical pairwise cosine similarities distributions. This example shows that dispersion of similarity values alone does not guarantee coverage of independent feature directions, motivating rank-aware synthetic data generation for quantization calibration.
XG
generated image
Y
label for XG
FXG(l)
features in the l -th block
S
patch similarity matrix
τ
temperature parameter
γ
A small positive number
Dp(⋅)
dropout with probability p
TABLE I: Key symbols in this work.
Fig. 3: Overview of RankQ-ViT. The workflow of RankQ-ViT involves two consecutive steps, namely block-aware rank promotion (the left panel) and block-wise training with sample selection (the right panel). First, the Gaussian noise images are encouraged to learn the high-rank similarity matrix by minimizing the Lrp . Then, generated samples with higher ranks (see Eq. 14 ) and minimal Jensen-Shannon Divergence (JSD) (see Eq. 15 ) are selected to form the training set D . Finally, the quantized model is forced to mimic the outputs of its full-precision version, recovering the accuracy degradation of quantization.
Fig. 4: The synthetic samples (224×224 pixels) shown in the figure are generated by PSAQ-ViT v1 and our proposed RankQ-ViT. The label associated with each generated image is indicated in the upper-right corner. The results demonstrate that images produced by PSAQ-ViT contain obscure semantic information, whereas those generated by our RankQ-ViT exhibit significantly stronger semantic content. Furthermore, the images from RankQ-ViT show richer texture details compared to those from PSAQ-ViT.
Model
Method
No Data
Data Num.
Precision
Top-1 Acc. (%)
Precision
Top-1 Acc. (%)
ViT-S
Full-Precision
-
-
W32/A32
81.39
W32/A32
81.39
Standard [ 16 ]
×
32
W4/A8
19.91
W8/A8
30.28
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A8
20.84
W8/A8
31.45
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A8
77.84
W8/A8
81.05
RepQ-ViT [ 19 ]
×
32
W4/A8
75.60
W8/A8
81.17
RankQ-ViT (ours)
✓
32
W4/A8
78.76 ±0.12
W8/A8
81.19 ±0.08
TABLE II: Quantization results on the ImageNet dataset with W4/A8 and W8/A8 quantization. “No Data” means that no realistic images are used. “Data Num.” means the number of images used for calibration. “Precision” denotes the bit-width of weights and activations. The top-1 accuracy (%) is reported as the evaluation metric.
Model
Method
No Data
Data Num.
Precision
Top-1 Acc. (%)
Precision
Top-1 Acc. (%)
ViT-S
Full-Precision
-
-
W32/A32
81.39
W32/A32
81.39
Standard
×
32
W4/A4
39.42
W5/A5
71.78
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A4
0.12
W5/A5
24.78
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A4
9.78
W5/A5
60.12
RankQ-ViT (ours)
✓
32
W4/A4
44.06 ±0.18
W5/A5
68.07 ±0.17
ViT-B
Full-Precision
-
-
W32/A32
84.53
W32/A32
84.53
TABLE III: Quantization results on the ImageNet dataset with W4/A4 and W5/A5 quantization.
Datasets
Method
No Data
Data Num.
Precision
mAcc (%)
OA (%)
Precision
mAcc (%)
OA (%)
ModelNet40
Full-Precision
-
-
W32/A32
83.99
88.00
W32/A32
83.99
88.00
Gaussian
✓
32
W4/A8
77.36
79.16
W8/A8
82.01
82.50
Real
×
32
W4/A8
80.00
84.51
W8/A8
83.67
87.18
Baseline
✓
32
W4/A8
79.85
83.98
W8/A8
83.26
87.00
PSAQ-ViT v1 [ 16 ]
✓
32
W4/A8
79.60
83.65
W8/A8
83.21
87.16
PSAQ-ViT v2 [ 17 ]
✓
-
W4/A8
80.00
84.65
W8/A8
83.28
87.01
TABLE IV: Quantization results of point cloud classification. “mACC” and “OA” denote the mean of class-wise accuracy (%) and the overall accuracy (%), respectively.
Model
Method
Prec.
AP box
AP mask
Swin-S Cascade Mask R-CNN
FP
W32/A32
51.9
48.0
Standard
W4/A8
47.3
40.9
Gaussian
W4/A8
38.1
35.2
PSAQ v1 [ 16 ]
W4/A8
42.6
36.9
PSAQ v2 [ 17 ]
W4/A8
47.9
41.4
RankQ-ViT
W4/A8
48.3 ±0.02
41.7 ±0.01
TABLE V: Quantization results on COCO dataset. “ APbox ” means the box average precision for object detection, and “ APmask ” means the mask average precision for instance segmentation.
Table 10
Fig. 5: Hyperparameter sensitivity analysis on W4/A8 quantization of DeiT-S. (a) Effect of different α values. (b) Effect of different Nsel values.
Fig. 6: Results of data generated by DeiT-T for other model quantization. W4/A8 quantization is used here for comparison. The results clearly indicate that our RankQ-ViT has superior cross-architecture generalization ability.
Fig. 7: Speed (FPS) and peak memory (MB) usage during inference on Intel 12th i9-12900K CPU and NVIDIA RTX-4090 GPU. (a) FPS comparison between FP32 and INT8 models. (b) The Peak GPU memory during inference. These results show that quantization significantly reduces the inference cost.
We present a framework for end-to-end joint quantization of Vision Transformers trained on ImageNet for the purpose of image classification. Unlike prior post-training or block-wise reconstruction methods, we jointly optimize over the entire set of all layers and inter-block dependencies without any labeled data, scaling effectively with the number of samples and completing in just one hour on a single GPU for ViT-small. We achieve state-of-the-art W4A4 and W3A3 accuracies on ImageNet and, to the best of our knowledge, the first PTQ results that maintain strong accuracy on ViT, DeiT, and Swin-T models under extremely low-bit settings (W1.58A8), demonstrating the potential for efficient edge deployment. Furthermore, we introduce a data-free calibration strategy that synthesizes diverse, label-free samples using Stable Diffusion Turbo guided by learned multi-mode prompts. By encouraging diversity in both the learned prompt embeddings and the generated image features, our data-free approach achieves performance on par with real-data ImageNet calibration and surpasses simple text-prompt baselines such as "a <adjective> photo of <adjective> <cls>".
Shile Li, Markus Karmann, Onay Urfalioglu
vivo Tech Research GmbH · Torr Vision Group, Universtiy of Oxford
Data-Free Quantization (DFQ) addresses data security concerns by synthesizing samples, without accessing real data. It has garnered increasing attention in the context of Vision Transformers (ViTs), owing to the superiority of the self-attention mechanism compared to classical convolutional operation. However, previous DFQ arts for ViTs often suffer from a distribution mismatch between synthetic samples and input distribution expected by quantized models Q, resulting in the suboptimal performance. In this paper, we propose a novel Masked Attention Alignment approach for Data-Free Quantization of ViTs, named MaskAQ, revealing that: 1) the semantics in the self-attention mechanism is predominantly localized to a sparse subset of patches, called informative regions; 2) the informative regions dominate the mutual information between synthetic samples and Q's outputs. To these ends, we incorporate differential entropy maximum over patch similarity of synthetic samples, to decouple informative regions from noisy background. To couple with varied Q, the informative regions are selected to align full-precision models with Q via a masked attention alignment objective, thus yielding high-quality synthetic samples. Furthermore, a periodic sample refreshing strategy comes up to endow MaskAQ with the capacity to continually adapt to the evolving state of Q throughout the training process, to preserve desirable mutual information with synthetic samples. Extensive experiments verify the merits of MaskAQ over state-of-the-art approaches across multiple backbones and downstream tasks. Our code is available at https://github.com/hfutqian/MaskAQ.
Biao Qian, Yang Wang, Yong Wu +1
Department of Automation, Tsinghua University, China. · School of Computer Science and Information Engineering, Hefei University of Technology, China. · Li Auto, China.
Vision Transformers (ViTs) achieve strong performance on image recognition and mobile vision applications, but their high-dimensional linear projections and attention computations still impose substantial storage and inference costs. Extremely low-bit quantization is a promising solution, yet ViTs often suffer severe accuracy degradation because conventional real-valued scalar codebooks are poorly matched to the directional geometry of Transformer projections. We present RPFQ-ViT, a Rotated Phase-Frame Quantization method that quantizes paired channels in two-dimensional phase planes, enabling low-bit codes to better preserve projection directions while recovering magnitude with lightweight scaling. RPFQ-ViT serves as a drop-in QAT replacement for nn.Linear and does not modify the standard real-valued attention, normalization, or activation computation graph. On ImageNet-1K, RPFQ-ViT-B/16 reaches 79.33% Top-1 / 94.48% Top-5 under W2/A4, Swin-T reaches 79.30% Top-1 / 94.79% Top-5 under W2/A8, and DeiT-S reaches 77.41% Top-1 / 93.11% Top-5 under W2/A8. Ablations, phase-geometry analysis, and direction-preservation metrics show that channel pairing, learnable rotation, phase-anchor learning, and residual phase refinement each improve quantization quality. We further deploy RPFQ-ViT image-classification models on native iOS and Android runtime stacks; with 2-bit packed weights, model size shrinks by roughly 5.4-7.1× relative to FP32 and end-to-end on-device latency drops by 1.4-1.6×. All ImageNet results trained in our codebase use a matched 300-epoch recipe and are reported as mean accuracies over three independent runs. These results show that RPFQ-ViT provides a favorable trade-off among accuracy, compression, and practical mobile deployment for extremely low-bit ViTs.