We enable accurate low-bit recovery from small synthetic pools by improving both image synthesis and student preparation. A grid with lower local response error can incur greater error in the student's block context. Functional Grid Calibration (FGC) selects activation scales and integer offsets against full-precision responses constructed from the student's current block input. With weights fixed during calibration, FGC evaluates candidates only at their consuming operators, bringing block context to calibration at nearly local-scoring cost. For DeiT and ViT, progressive inversion updates the same images through increasingly deep teacher prefixes. With 1,000 images, the complete pipeline exceeds MimiQ's reported accuracy in all 18 W4A4/W5A5 settings across nine backbones. With 256 images, DeiT-S and ViT-S reach 68.20% and 66.90% ImageNet top-1 accuracy at W4A4 in 26-27 minutes on one RTX 5090. The pipeline outperforms strengthened 1K-image MimiQ by 13.03 and 24.36 percentage points at lower preparation cost and the same recovery-update count. The same low-budget checkpoints retain their advantage on ImageNetV2.
Figures & tables
Figure 1: High accuracy from small synthetic pools at low preparation cost. Our 256-image configurations outperform reduced-budget MimiQ at lower total preparation cost. W4A4 timings include synthesis, calibration and recovery on one RTX 5090. Labels indicate pool size; each curve fixes the inversion updates per image. Our bars span two recovery seeds. 100× denotes amplified ordinary-activation scale gradients; external configurations are defined in Sec. 4.1 .
Figure 2: Generating a small pool and preparing the student to use it. (a) The same synthetic pool supplies calibration and recovery views. DeiT/ViT use progressive inversion; quantitative Swin results use full-depth pools. The final probability representation is installed after calibration, with row normalization active during recovery and inference. (b) Stage 2 illustrated at FC2, the MLP output projection. Both paths start from the same student block input. The cached FP target stays fixed within the block. Candidates are evaluated sequentially with shared current operands and fixed quantized weights; operands are refreshed before each coordinate search.
ViT
DeiT
Swin
W/A
Method
T
S
B
T
S
B
T
S
B
FP
Our FP teacher
75.34
81.38
84.41
72.02
79.72
81.74
80.92
82.90
84.75
4/4
PSAQ-ViT ‡
0.67
0.15
0.94
19.61
5.90
8.74
22.71
9.26
23.69
PSAQ-ViT V2 ‡
1.54
4.14
2.83
22.82
32.57
45.81
50.42
39.10
39.26
MimiQ
42.99
55.69
62.91
52.03
62.72
74.10
69.33
70.46
73.49
SARDFQ
–
50.32
51.84
52.06
62.29
72.17
68.74 †
74.74
76.42
Table 1: Data-free ImageNet Top-1 accuracy (%) across nine backbones. Transformer-specific methods are shown here; Tab. S14 gives the complete historical comparison. ‡ : results for prior methods reported by MimiQ [ 2 ] ; other external cells are author-reported, except † , our SARDFQ Swin-T adaptation. Our complete pipeline uses 1K images and reports the mean over two recovery seeds; FP identifies our teachers. Bold/underline mark the highest/second-highest reported value per column and bitwidth, excluding FP. A dash denotes an unreported setting. Quantizer, synthesis, and recovery protocols are method-specific.
Figure 3: Context changes which grid is preferred. (a) The contextual choice increases local operator-response error but improves the interaction enough to lower total error. (b) All 12 FC2 coordinates, including unchanged choices. Selection uses calibration views; evaluation uses 64 held-out synthetic inputs before recovery. Maps show individual inputs, with labels averaged over patch tokens; right panels pool all inputs and include CLS. Differences (contextual choice minus local choice) use contextual-reference normalization, in 10−3 units.
Figure 4: Allocating synthesis computation. (a) 256 images × 500 inversion updates versus 512 × 250 at near-equal preparation cost. Points show two-seed means; error bars span the seeds. Time includes synthesis, calibration and recovery. Right: depth schedules for the DeiT/ViT examples in (b). (b) Full333/PG500 at 3,996 logical block evaluations per image, paired by initial noise and requested class. Quantitative Swin recovery uses full-depth pools.
Figure 5: Prepared grids support earlier recovery. W4A4 on the primary 1K pools along the default recovery schedule. Points show two-seed means; bars span the seeds. At 6,174 updates, FGC exceeds tensor MSE at 12,474 on both backbones.
Backbone
Scale
Local
FGC
Δ
DeiT-S
Default
67.87
68.59
+0.72
100×
68.78
69.59
+0.81
ViT-S
Default
67.39
68.15
+0.76
100×
68.82
68.55
−0.27
Table 4: Scale adaptation changes the ranking of initializations. Two-seed Top-1 means (%) on the reference-study pools at 12,474 updates. Δ is FGC minus Local (pp). Only the ordinary-scale gradient multiplier changes.
Configuration
DeiT-S
ViT-S
(a) Small-budget checkpoints from Fig. 1
MimiQ 1K, grad. 100×
42.72
32.81
Our pipeline, 256 images
54.83
53.94
(b) Primary 1K and external checkpoints
MimiQ, native
48.49
39.96
MimiQ, grad. 100×
54.55
50.33
Table 5: The low-budget advantage persists on ImageNetV2. W4A4 Top-1 (%). (a) Exact Fig. 1 low-budget checkpoints, two-seed means. (b) Primary 1K and external checkpoints: long-budget MimiQ uses seed 17; SARDFQ averages two complete runs, and our primary results average two recovery seeds. Amplified MimiQ includes the static-control cache.
Surface
Granularity/support
Calibration
Recovery/inference
Weights
Per output channel; symmetric signed
Frozen teacher shadow weights and derived grid
Train shadow weights; dynamic min/max; cached final codes
Ordinary A
One affine (s,z) per tensor; fixed signedness
MSE proposal then operator-score sweep
Train s ; freeze rounded z
Probability P
One unsigned grid per surface; z=0
Native-P sweep, then mean-P reset
Train scale; normalize each quantized row; zero-row fallback
Table S1: Representation and optimization used by our complete preparation pipeline. Floating arithmetic is retained for fake quantization, biases, normalizations and residual operations.
Pool
Used for
FGC Top-1
Primary 1K
Tab. 1 and block readout
68.911 / 67.892
Independent PG500, 1K
Synthesis and calibration factorial
68.428 / 67.675
Matched-budget PG500, 1K
Reference origins and budget endpoints
68.591 / 68.151
Actual-size PG500, 256
Low-cost endpoint and same-pool control
68.201 / 66.901
Table S2: Pool identities for DeiT-S and ViT-S at W4A4. Each FGC pair gives the two backbone means in order.
Model
Seed
Tensor MSE
FGC
Δ (pp)
DeiT-S
17
66.884
68.082
+1.198
29
66.980
68.320
+1.340
Mean
66.932
68.201
+1.269
ViT-S
17
66.488
67.014
+0.526
29
66.058
66.788
+0.730
Mean
66.273
66.901
+0.628
Table S3: Top-1 (%) on the actual 256-image pools. Δ is the paired FGC gain in percentage points.
Images
Updates
DeiT-S
ViT-S
DeiT-B
ViT-B
256
250
67.003 / 18.84
65.228 / 19.31
75.909 / 38.95
69.190 / 39.67
256
500
68.201 / 26.36
66.901 / 26.80
76.407 / 54.81
71.730 / 55.91
512
250
67.016 / 26.25
65.626 / 26.80
75.952 / 54.92
70.407 / 56.05
512
500
68.411 / 41.20
66.887 / 41.78
76.691 / 86.62
70.250 / 88.07
1000
250
67.275 / 40.53
65.943 / 41.00
75.873 / 84.94
67.308 / 86.01
1000
500
68.591 / 69.34
68.151 / 70.44
76.356 / 146.65
73.417 / 148.21
Table S4: All four backbone budget grids. Each cell gives two-seed mean Top-1 (%) and complete preparation time (min).
Backbone
Method
Runs
Synthesis
Cal.
Recovery
Total
Top-1
DeiT-S
FGC, 256 / 500
2
14.99
0.12
11.24
26.36
68.201
DeiT-S
FGC, 1000 / 500
2
–
–
–
69.34
68.591
DeiT-S
MimiQ, small 100×
2
17.87
incl.
10.34
28.20
55.173
DeiT-S
MimiQ, long native
1
714.37
incl.
115.64
830.01
62.314
DeiT-S
MimiQ, long 100× + cache
1
714.37
incl.
104.49
818.85
67.572
DeiT-S
SARDFQ
2
2.35
incl.
1.46
3.80
64.028
Table S5: Measured preparation stages for representative Fig. 1 points (min). FGC rows use 500 inversion updates. For MimiQ and SARDFQ, recovery includes calibration. Long-budget MimiQ recovery is the reported total less the shared measured synthesis time. Dashes denote an unavailable two-run stage split. Top-1 is diagnostic ImageNet-50K accuracy (%).
Model
Native 1
Cached
Native 2
DeiT-S
0.05724
0.05144
0.05766
ViT-S
0.06007
0.05377
0.05925
Table S6: MimiQ control-cache probe. Each cell reports seconds per update.
DeiT-S
ViT-S
Synthesis
Preparation
Seed17
Seed29
Mean
Seed17
Seed29
Mean
Full333
Tensor MSE
64.416
64.610
64.513
66.058
65.834
65.946
Full333
FGC
66.510
66.404
66.457
66.540
66.616
66.578
PG500
Tensor MSE
66.958
67.082
67.020
67.090
66.812
66.951
PG500
FGC
68.498
68.358
68.428
67.810
67.540
67.675
Table S7: Four matched-work synthesis/calibration cells. W4A4 ImageNet Top-1 (%) over seeds 17 and 29.
Control
Model
Seed 17
Seed 29
Mean
Full-depth calibration / progressive recovery
DeiT-S
68.154
68.246
68.200
Full-depth calibration / progressive recovery
ViT-S
67.752
67.160
67.456
Full-depth synthesis (500 steps) / FGC
DeiT-S
67.504
67.340
67.422
Full-depth synthesis (500 steps) / FGC
ViT-S
67.124
66.794
66.959
Table S8: Fixed-initializer crossover and equal-update reference. W4A4 ImageNet Top-1 (%).
Figure S1 : Paired synthetic endpoints for fixed class IDs. Full333 and PG500 match logical block work, and Full500 and PG500 match the number of image updates.
Model
Pool
FP target
Seed 17
Seed 29
Mean
DeiT-S
Budget 1K
Student block input (default)
68.564
68.618
68.591
DeiT-S
Budget 1K
Local operator reference
67.932
67.804
67.868
DeiT-S
Budget 1K
Full FP trajectory
67.462
67.834
67.648
ViT-S
Budget 1K
Student block input (default)
68.328
67.974
68.151
ViT-S
Budget 1K
Local operator reference
67.364
67.420
67.392
ViT-S
Budget 1K
Full FP trajectory
62.602
62.506
62.554
Table S9: Per-seed reference-origin controls. The primary and budget studies use distinct 1K pools. Top-1 (%).
Model
Readout
Seed 17
Seed 29
Mean
DeiT-S
Operator
68.802
69.020
68.911
DeiT-S
Block
68.588
68.648
68.618
Swin-T
Operator
75.110
74.848
74.979
Swin-T
Block
75.098
74.538
74.818
ViT-S
Operator
67.782
68.002
67.892
ViT-S
Block
67.650
68.000
67.825
Table S10: Primary W4A4 readout ablation with the FP student-input reference. ImageNet Top-1 (%).
Model
Objective
Time (s)
DeiT-S
Local-reference operator
8.216±0.078
DeiT-S
Contextual operator (FGC)
8.324±0.049
DeiT-S
Contextual block readout
9.820±0.073
ViT-S
Local-reference operator
8.961±0.057
ViT-S
Contextual operator (FGC)
9.060±0.069
ViT-S
Contextual block readout
10.604±0.025
Table S11: Calibration-stage seconds, mean ± sample standard deviation over three repetitions.
Model
OFF 17
OFF 29
ON 17
ON 29
OFF mean
ON mean
DeiT-S
66.234
66.294
66.836
66.792
66.264
66.814
ViT-S
63.944
63.454
67.206
66.688
63.699
66.947
Table S12: Row normalization at a shared MSE initializer. ImageNet Top-1 (%).
Model
P staging
Seed 17
Seed 29
Mean
DeiT-S
Student block input (default)
68.564
68.618
68.591
DeiT-S
Skip P search; install final P late
68.566
68.534
68.550
DeiT-S
Final-P-first ordinary sweep
67.970
68.120
68.045
ViT-S
Student block input (default)
68.328
67.974
68.151
ViT-S
Skip P search; install final P late
67.922
67.786
67.854
ViT-S
Final-P-first ordinary sweep
66.846
66.702
66.774
Table S13: Probability stages on the matched-budget pools. ImageNet Top-1 (%).
Table S15: Matched W4A4 ImageNet Top-1 (%) on primary pools. Each pair lists seeds 17/29. Δ is FGC minus tensor MSE (pp).
Model
Seed 17
Seed 29
Mean
DeiT-B
79.896
79.910
79.903
DeiT-S
76.382
76.522
76.452
DeiT-T
67.554
67.446
67.500
Swin-B
82.140
82.118
82.129
Swin-S
81.304
81.276
81.290
Swin-T
78.892
78.568
78.730
Table S16: W5A5 FGC on primary pools. Top-1 (%) at the final checkpoint.
Model
W3A3
W4A4
W5A5
W6A6
W7A7
W4A8
W8A8
DeiT-S
31.678
68.911
76.452
78.217
78.978
77.963
79.340
ViT-S
15.367
67.892
76.034
79.318
80.413
77.935
80.666
Swin-T
54.161
74.979
78.730
80.087
80.526
78.838
80.627
Table S17: Fixed-recipe bitwidth sweep. ImageNet Top-1 (%), means over two seeds. P equals A.
Model
Preparation
Seed 17
Seed 29
Mean
DeiT-S
Tensor MSE
53.930
53.690
53.810
DeiT-S
FGC: block readout
55.700
55.330
55.515
DeiT-S
FGC
55.520
55.370
55.445
ViT-S
Tensor MSE
54.360
54.190
54.275
ViT-S
FGC: block readout
55.140
54.670
54.905
ViT-S
FGC
54.830
55.060
54.945
Table S18: Primary pools. W4A4 ImageNetV2 Top-1 (%) at the same checkpoints used for the primary comparisons.
Model
Preparation / reference
Seed 17
Seed 29
Mean
DeiT-S
Tensor MSE
53.790
53.850
53.820
DeiT-S
Local
54.900
54.950
54.925
DeiT-S
Full teacher
54.240
55.210
54.725
DeiT-S
FGC
55.620
55.390
55.505
ViT-S
Tensor MSE
54.410
54.140
54.275
ViT-S
Local
54.530
54.190
54.360
Table S19: Matched-budget 1K pools. ImageNetV2 Top-1 (%) for the reference-origin study.
Configuration
Seed
DeiT-S
ViT-S
MimiQ, native
17
48.490
39.960
MimiQ, scale grad. 100×
17
54.550
50.330
SARDFQ, rep. off
17
50.700
43.570
SARDFQ, rep. off
29
51.840
44.960
Table S20: External long-budget MimiQ and SARDFQ checkpoints from the accuracy–cost comparison. The primary FGC comparator is in Tab. S18 . ImageNetV2 Top-1 (%).
Model
Configuration
Seed17
Seed29
Mean
DeiT-S
MimiQ 1K/Full500, grad. 100×
42.740
42.690
42.715
DeiT-S
Our pipeline 256/PG500
54.880
54.780
54.830
ViT-S
MimiQ 1K/Full500, grad. 100×
33.010
32.610
32.810
ViT-S
Our pipeline 256/PG500
54.340
53.530
53.935
Table S21: Exact low-cost frontier checkpoints on ImageNetV2. Top-1 (%).
Figure S2 : The same legal FC2 candidate grids under three objectives for each backbone. Stars mark each score minimum. Color scales are specific to each objective.
Model
Ordinary-scale rule
MSE pair
FGC pair
MSE mean
FGC mean
Gap
DeiT-S
Native
66.836/66.792
68.564/68.618
66.814
68.591
+1.777
100×
68.140/68.176
69.616/69.572
68.158
69.594
+1.436
ViT-S
Native
67.206/66.688
68.328/67.974
66.947
68.151
+1.204
100×
68.494/68.482
68.522/68.570
68.488
68.546
+0.058
Swin-T
Native
74.318/74.316
75.110/74.848
74.317
74.979
+0.662
100×
74.552/74.842
74.954/74.962
74.697
74.958
+0.261
Table S22: MSE and FGC under native and amplified ordinary-scale gradients. ImageNet Top-1 (%) with seed pairs 17/29. Gap is FGC minus MSE (pp).
Model
Rule
Local 17/29
FGC 17/29
Local mean
FGC mean
Δ
DeiT-S
Default
67.932/67.804
68.564/68.618
67.868
68.591
+0.723
DeiT-S
100×
68.812/68.756
69.616/69.572
68.784
69.594
+0.810
ViT-S
Default
67.364/67.420
68.328/67.974
67.392
68.151
+0.759
ViT-S
100×
68.814/68.818
68.522/68.570
68.816
68.546
-0.270
Table S23: Local and FGC under each scale-gradient rule on the matched-reference pools. ImageNet Top-1 (%) at 12,474 updates. Δ is FGC minus Local (pp).
Preparation
Updates
DeiT-S
ViT-S
Tensor MSE
3150
66.723
66.345
Tensor MSE
6174
67.206
67.122
Tensor MSE
12474
66.992
67.299
FGC: block readout
3150
68.261
67.004
FGC: block readout
6174
68.643
67.886
FGC: block readout
12474
68.618
67.825
Table S24: Default recovery trajectory on the primary 1K pools. Two-seed mean ImageNet Top-1 (%).
We present a framework for end-to-end joint quantization of Vision Transformers trained on ImageNet for the purpose of image classification. Unlike prior post-training or block-wise reconstruction methods, we jointly optimize over the entire set of all layers and inter-block dependencies without any labeled data, scaling effectively with the number of samples and completing in just one hour on a single GPU for ViT-small. We achieve state-of-the-art W4A4 and W3A3 accuracies on ImageNet and, to the best of our knowledge, the first PTQ results that maintain strong accuracy on ViT, DeiT, and Swin-T models under extremely low-bit settings (W1.58A8), demonstrating the potential for efficient edge deployment. Furthermore, we introduce a data-free calibration strategy that synthesizes diverse, label-free samples using Stable Diffusion Turbo guided by learned multi-mode prompts. By encouraging diversity in both the learned prompt embeddings and the generated image features, our data-free approach achieves performance on par with real-data ImageNet calibration and surpasses simple text-prompt baselines such as "a <adjective> photo of <adjective> <cls>".
Shile Li, Markus Karmann, Onay Urfalioglu
vivo Tech Research GmbH · Torr Vision Group, Universtiy of Oxford
Low-bit post-training quantization reduces language-model memory but can sharply degrade downstream task accuracy. Calibration usually relies on generic text, but when task-informed data recover accuracy and which quantizers benefit remain poorly understood. We introduce FragCal-DC, which supplements generic text with training documents from task families with low retention of full-precision accuracy in a two-bit scan. We use a two- to four-bit sweep to locate functional regimes and four-seed calibration comparisons to evaluate six data-dependent methods across six dense models (1.7B-8B) and ten tasks. At their lowest tested functional bit widths, iterative SignRound V2 and OmniQuant improve at two bits, whereas single-pass GPTQ and AWQ show small mean responses at three bits. At two bits, SignRound V2 gains 1.83 percentage points in mean ten-task accuracy over generic-text calibration across six models. At 70 billion parameters, GPTQ and SignRound V2 both improve at two bits, with functional generic-calibration baselines. For two-bit SignRound V2, task-level controls show that recovery concentrates on families included in calibration, while incorrect answer designations retain about 90% of the mean accuracy gain. Changes in reconstruction error do not consistently track accuracy gains. These findings support task-informed calibration as a design principle for preserving downstream accuracy in low-bit quantization.
Yehong Jiang, Yen-Kuang Chen, Xinmin Tian
Massachusetts Institute of Technology · Intel Corporation
Diffusion large language models dLLMs) have emerged as a promising alternative to autoregressive language models through bidirectional diffusion-based token generation. However, their growing model sizes and high inference costs make efficient deployment challenging: full-sequence denoising repeatedly invokes compute-intensive forward passes, while block-diffusion models additionally introduce a memory-intensive KV-cache. Low-bit weight-activation quantization is therefore attractive, yet existing dLLM post-training quantization methods rely on calibration data despite activation distributions shifting across masking states and denoising steps. We present SoloQ, a calibration-free quantization framework that maps weights and activations into a normalized rotated basis with a predictable marginal distribution, enabling data-independent quantization. SoloQ combines a structured K-RPBH rotation with a lightweight rescaling correction for calibration-free quantization. Its predictable post-rotation distribution supports both distribution-matched codebooks and hardware-native NVFP4. For block-diffusion models, SoloQ further applies commit-time KV-cache quantization to compress persistent states without perturbing the actively denoised block. Across full-sequence dLLMs (LLaDA and Dream) and block-diffusion dLLMs(Fast-dLLM v2 and Nemotron-Labs-Diffusion), SoloQ retains accuracy under 4-bit quantization and outperforms calibration-based baselines on knowledge- and reasoning-intensive benchmarks. With NVFP4, SoloQ reduces peak memory by up to 2.61X and accelerates end-to-end inference by up to 2.24X.
Donghyun Lee, Arkapravo Ghosh, Varun Manjunath +5
University of Southern California · Yale University · Korea University