Calibrate in Context: Efficient Data-Free Quantization from Small Synthetic Pools
Organizations: Soochow University · D-Robotics · Heriot-Watt University · Fraunhofer Institute for Applied Information Technology
Abstract
We enable accurate low-bit recovery from small synthetic pools by improving both image synthesis and student preparation. A grid with lower local response error can incur greater error in the student's block context. Functional Grid Calibration (FGC) selects activation scales and integer offsets against full-precision responses constructed from the student's current block input. With weights fixed during calibration, FGC evaluates candidates only at their consuming operators, bringing block context to calibration at nearly local-scoring cost. For DeiT and ViT, progressive inversion updates the same images through increasingly deep teacher prefixes. With 1,000 images, the complete pipeline exceeds MimiQ's reported accuracy in all 18 W4A4/W5A5 settings across nine backbones. With 256 images, DeiT-S and ViT-S reach 68.20% and 66.90% ImageNet top-1 accuracy at W4A4 in 26-27 minutes on one RTX 5090. The pipeline outperforms strengthened 1K-image MimiQ by 13.03 and 24.36 percentage points at lower preparation cost and the same recovery-update count. The same low-budget checkpoints retain their advantage on ImageNetV2.
Figures & tables
| ViT | DeiT | Swin | ||||||||
| W/A | Method | T | S | B | T | S | B | T | S | B |
| FP | Our FP teacher | 75.34 | 81.38 | 84.41 | 72.02 | 79.72 | 81.74 | 80.92 | 82.90 | 84.75 |
| 4/4 | PSAQ-ViT ‡ | 0.67 | 0.15 | 0.94 | 19.61 | 5.90 | 8.74 | 22.71 | 9.26 | 23.69 |
| PSAQ-ViT V2 ‡ | 1.54 | 4.14 | 2.83 | 22.82 | 32.57 | 45.81 | 50.42 | 39.10 | 39.26 | |
| MimiQ | 42.99 | 55.69 | 62.91 | 52.03 | 62.72 | 74.10 | 69.33 | 70.46 | 73.49 | |
| SARDFQ | – | 50.32 | 51.84 | 52.06 | 62.29 | 72.17 | 68.74 † | 74.74 | 76.42 | |
| Backbone | Scale | Local | FGC | |
|---|---|---|---|---|
| DeiT-S | Default | 67.87 | 68.59 | |
| 68.78 | 69.59 | |||
| ViT-S | Default | 67.39 | 68.15 | |
| 68.82 | 68.55 |
| Configuration | DeiT-S | ViT-S |
|---|---|---|
| (a) Small-budget checkpoints from Fig. 1 | ||
| MimiQ 1K, grad. | 42.72 | 32.81 |
| Our pipeline, 256 images | 54.83 | 53.94 |
| (b) Primary 1K and external checkpoints | ||
| MimiQ, native | 48.49 | 39.96 |
| MimiQ, grad. | 54.55 | 50.33 |
| Surface | Granularity/support | Calibration | Recovery/inference |
|---|---|---|---|
| Weights | Per output channel; symmetric signed | Frozen teacher shadow weights and derived grid | Train shadow weights; dynamic min/max; cached final codes |
| Ordinary A | One affine per tensor; fixed signedness | MSE proposal then operator-score sweep | Train ; freeze rounded |
| Probability P | One unsigned grid per surface; | Native-P sweep, then mean-P reset | Train scale; normalize each quantized row; zero-row fallback |
| Pool | Used for | FGC Top-1 |
|---|---|---|
| Primary 1K | Tab. 1 and block readout | 68.911 / 67.892 |
| Independent PG500, 1K | Synthesis and calibration factorial | 68.428 / 67.675 |
| Matched-budget PG500, 1K | Reference origins and budget endpoints | 68.591 / 68.151 |
| Actual-size PG500, 256 | Low-cost endpoint and same-pool control | 68.201 / 66.901 |
| Model | Seed | Tensor MSE | FGC | (pp) |
|---|---|---|---|---|
| DeiT-S | 17 | 66.884 | 68.082 | +1.198 |
| 29 | 66.980 | 68.320 | +1.340 | |
| Mean | 66.932 | 68.201 | +1.269 | |
| ViT-S | 17 | 66.488 | 67.014 | +0.526 |
| 29 | 66.058 | 66.788 | +0.730 | |
| Mean | 66.273 | 66.901 | +0.628 |
| Images | Updates | DeiT-S | ViT-S | DeiT-B | ViT-B |
|---|---|---|---|---|---|
| 256 | 250 | 67.003 / 18.84 | 65.228 / 19.31 | 75.909 / 38.95 | 69.190 / 39.67 |
| 256 | 500 | 68.201 / 26.36 | 66.901 / 26.80 | 76.407 / 54.81 | 71.730 / 55.91 |
| 512 | 250 | 67.016 / 26.25 | 65.626 / 26.80 | 75.952 / 54.92 | 70.407 / 56.05 |
| 512 | 500 | 68.411 / 41.20 | 66.887 / 41.78 | 76.691 / 86.62 | 70.250 / 88.07 |
| 1000 | 250 | 67.275 / 40.53 | 65.943 / 41.00 | 75.873 / 84.94 | 67.308 / 86.01 |
| 1000 | 500 | 68.591 / 69.34 | 68.151 / 70.44 | 76.356 / 146.65 | 73.417 / 148.21 |
| Backbone | Method | Runs | Synthesis | Cal. | Recovery | Total | Top-1 |
|---|---|---|---|---|---|---|---|
| DeiT-S | FGC, 256 / 500 | 2 | 14.99 | 0.12 | 11.24 | 26.36 | 68.201 |
| DeiT-S | FGC, 1000 / 500 | 2 | – | – | – | 69.34 | 68.591 |
| DeiT-S | MimiQ, small | 2 | 17.87 | incl. | 10.34 | 28.20 | 55.173 |
| DeiT-S | MimiQ, long native | 1 | 714.37 | incl. | 115.64 | 830.01 | 62.314 |
| DeiT-S | MimiQ, long + cache | 1 | 714.37 | incl. | 104.49 | 818.85 | 67.572 |
| DeiT-S | SARDFQ | 2 | 2.35 | incl. | 1.46 | 3.80 | 64.028 |
| Model | Native 1 | Cached | Native 2 |
|---|---|---|---|
| DeiT-S | 0.05724 | 0.05144 | 0.05766 |
| ViT-S | 0.06007 | 0.05377 | 0.05925 |
| DeiT-S | ViT-S | ||||||
|---|---|---|---|---|---|---|---|
| Synthesis | Preparation | Seed17 | Seed29 | Mean | Seed17 | Seed29 | Mean |
| Full333 | Tensor MSE | 64.416 | 64.610 | 64.513 | 66.058 | 65.834 | 65.946 |
| Full333 | FGC | 66.510 | 66.404 | 66.457 | 66.540 | 66.616 | 66.578 |
| PG500 | Tensor MSE | 66.958 | 67.082 | 67.020 | 67.090 | 66.812 | 66.951 |
| PG500 | FGC | 68.498 | 68.358 | 68.428 | 67.810 | 67.540 | 67.675 |
| Control | Model | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|
| Full-depth calibration / progressive recovery | DeiT-S | 68.154 | 68.246 | 68.200 |
| Full-depth calibration / progressive recovery | ViT-S | 67.752 | 67.160 | 67.456 |
| Full-depth synthesis (500 steps) / FGC | DeiT-S | 67.504 | 67.340 | 67.422 |
| Full-depth synthesis (500 steps) / FGC | ViT-S | 67.124 | 66.794 | 66.959 |
| Model | Pool | FP target | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|---|
| DeiT-S | Budget 1K | Student block input (default) | 68.564 | 68.618 | 68.591 |
| DeiT-S | Budget 1K | Local operator reference | 67.932 | 67.804 | 67.868 |
| DeiT-S | Budget 1K | Full FP trajectory | 67.462 | 67.834 | 67.648 |
| ViT-S | Budget 1K | Student block input (default) | 68.328 | 67.974 | 68.151 |
| ViT-S | Budget 1K | Local operator reference | 67.364 | 67.420 | 67.392 |
| ViT-S | Budget 1K | Full FP trajectory | 62.602 | 62.506 | 62.554 |
| Model | Readout | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|
| DeiT-S | Operator | 68.802 | 69.020 | 68.911 |
| DeiT-S | Block | 68.588 | 68.648 | 68.618 |
| Swin-T | Operator | 75.110 | 74.848 | 74.979 |
| Swin-T | Block | 75.098 | 74.538 | 74.818 |
| ViT-S | Operator | 67.782 | 68.002 | 67.892 |
| ViT-S | Block | 67.650 | 68.000 | 67.825 |
| Model | Objective | Time (s) |
|---|---|---|
| DeiT-S | Local-reference operator | |
| DeiT-S | Contextual operator (FGC) | |
| DeiT-S | Contextual block readout | |
| ViT-S | Local-reference operator | |
| ViT-S | Contextual operator (FGC) | |
| ViT-S | Contextual block readout |
| Model | OFF 17 | OFF 29 | ON 17 | ON 29 | OFF mean | ON mean |
|---|---|---|---|---|---|---|
| DeiT-S | 66.234 | 66.294 | 66.836 | 66.792 | 66.264 | 66.814 |
| ViT-S | 63.944 | 63.454 | 67.206 | 66.688 | 63.699 | 66.947 |
| Model | P staging | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|
| DeiT-S | Student block input (default) | 68.564 | 68.618 | 68.591 |
| DeiT-S | Skip P search; install final P late | 68.566 | 68.534 | 68.550 |
| DeiT-S | Final-P-first ordinary sweep | 67.970 | 68.120 | 68.045 |
| ViT-S | Student block input (default) | 68.328 | 67.974 | 68.151 |
| ViT-S | Skip P search; install final P late | 67.922 | 67.786 | 67.854 |
| ViT-S | Final-P-first ordinary sweep | 66.846 | 66.702 | 66.774 |
| ViT | DeiT | Swin | ||||||||
| W/A | Method | T | S | B | T | S | B | T | S | B |
| FP | Local teacher | 75.34 | 81.38 | 84.41 | 72.02 | 79.72 | 81.74 | 80.92 | 82.90 | 84.75 |
| 4/4 | GDFQ ‡ | 2.95 | 4.62 | 11.73 | 25.96 | 22.12 | 30.04 | 42.08 | 41.93 | 36.04 |
| Qimera ‡ | 0.57 | 7.02 | 5.61 | 15.18 | 11.37 | 32.49 | 47.98 | 39.64 | 29.27 | |
| AdaDFQ ‡ | 2.00 | 1.78 | 6.21 | 19.57 | 14.44 | 19.22 | 38.88 | 39.40 | 32.26 | |
| PSAQ-ViT ‡ | 0.67 | 0.15 | 0.94 | 19.61 | 5.90 | 8.74 | 22.71 | 9.26 | 23.69 | |
| Model | MSE pair | Mean | FGC pair | Mean | |
|---|---|---|---|---|---|
| ViT-T | 45.716/45.648 | 45.682 | 44.642/44.678 | 44.660 | -1.022 |
| ViT-S | 67.370/67.228 | 67.299 | 67.782/68.002 | 67.892 | +0.593 |
| ViT-B | 69.142/68.894 | 69.018 | 71.300/71.224 | 71.262 | +2.244 |
| DeiT-T | 57.910/57.778 | 57.844 | 58.512/58.730 | 58.621 | +0.777 |
| DeiT-S | 66.950/67.034 | 66.992 | 68.802/69.020 | 68.911 | +1.919 |
| DeiT-B | 76.062/76.094 | 76.078 | 76.496/76.458 | 76.477 | +0.399 |
| Model | Seed 17 | Seed 29 | Mean |
|---|---|---|---|
| DeiT-B | 79.896 | 79.910 | 79.903 |
| DeiT-S | 76.382 | 76.522 | 76.452 |
| DeiT-T | 67.554 | 67.446 | 67.500 |
| Swin-B | 82.140 | 82.118 | 82.129 |
| Swin-S | 81.304 | 81.276 | 81.290 |
| Swin-T | 78.892 | 78.568 | 78.730 |
| Model | W3A3 | W4A4 | W5A5 | W6A6 | W7A7 | W4A8 | W8A8 |
|---|---|---|---|---|---|---|---|
| DeiT-S | 31.678 | 68.911 | 76.452 | 78.217 | 78.978 | 77.963 | 79.340 |
| ViT-S | 15.367 | 67.892 | 76.034 | 79.318 | 80.413 | 77.935 | 80.666 |
| Swin-T | 54.161 | 74.979 | 78.730 | 80.087 | 80.526 | 78.838 | 80.627 |
| Model | Preparation | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|
| DeiT-S | Tensor MSE | 53.930 | 53.690 | 53.810 |
| DeiT-S | FGC: block readout | 55.700 | 55.330 | 55.515 |
| DeiT-S | FGC | 55.520 | 55.370 | 55.445 |
| ViT-S | Tensor MSE | 54.360 | 54.190 | 54.275 |
| ViT-S | FGC: block readout | 55.140 | 54.670 | 54.905 |
| ViT-S | FGC | 54.830 | 55.060 | 54.945 |
| Model | Preparation / reference | Seed 17 | Seed 29 | Mean |
|---|---|---|---|---|
| DeiT-S | Tensor MSE | 53.790 | 53.850 | 53.820 |
| DeiT-S | Local | 54.900 | 54.950 | 54.925 |
| DeiT-S | Full teacher | 54.240 | 55.210 | 54.725 |
| DeiT-S | FGC | 55.620 | 55.390 | 55.505 |
| ViT-S | Tensor MSE | 54.410 | 54.140 | 54.275 |
| ViT-S | Local | 54.530 | 54.190 | 54.360 |
| Configuration | Seed | DeiT-S | ViT-S |
|---|---|---|---|
| MimiQ, native | 17 | 48.490 | 39.960 |
| MimiQ, scale grad. | 17 | 54.550 | 50.330 |
| SARDFQ, rep. off | 17 | 50.700 | 43.570 |
| SARDFQ, rep. off | 29 | 51.840 | 44.960 |
| Model | Configuration | Seed17 | Seed29 | Mean |
|---|---|---|---|---|
| DeiT-S | MimiQ 1K/Full500, grad. | 42.740 | 42.690 | 42.715 |
| DeiT-S | Our pipeline 256/PG500 | 54.880 | 54.780 | 54.830 |
| ViT-S | MimiQ 1K/Full500, grad. | 33.010 | 32.610 | 32.810 |
| ViT-S | Our pipeline 256/PG500 | 54.340 | 53.530 | 53.935 |
| Model | Ordinary-scale rule | MSE pair | FGC pair | MSE mean | FGC mean | Gap |
|---|---|---|---|---|---|---|
| DeiT-S | Native | 66.836/66.792 | 68.564/68.618 | 66.814 | 68.591 | +1.777 |
| 68.140/68.176 | 69.616/69.572 | 68.158 | 69.594 | +1.436 | ||
| ViT-S | Native | 67.206/66.688 | 68.328/67.974 | 66.947 | 68.151 | +1.204 |
| 68.494/68.482 | 68.522/68.570 | 68.488 | 68.546 | +0.058 | ||
| Swin-T | Native | 74.318/74.316 | 75.110/74.848 | 74.317 | 74.979 | +0.662 |
| 74.552/74.842 | 74.954/74.962 | 74.697 | 74.958 | +0.261 |
| Model | Rule | Local 17/29 | FGC 17/29 | Local mean | FGC mean | |
|---|---|---|---|---|---|---|
| DeiT-S | Default | 67.932/67.804 | 68.564/68.618 | 67.868 | 68.591 | +0.723 |
| DeiT-S | 68.812/68.756 | 69.616/69.572 | 68.784 | 69.594 | +0.810 | |
| ViT-S | Default | 67.364/67.420 | 68.328/67.974 | 67.392 | 68.151 | +0.759 |
| ViT-S | 68.814/68.818 | 68.522/68.570 | 68.816 | 68.546 | -0.270 |
| Preparation | Updates | DeiT-S | ViT-S |
|---|---|---|---|
| Tensor MSE | 3150 | 66.723 | 66.345 |
| Tensor MSE | 6174 | 67.206 | 67.122 |
| Tensor MSE | 12474 | 66.992 | 67.299 |
| FGC: block readout | 3150 | 68.261 | 67.004 |
| FGC: block readout | 6174 | 68.643 | 67.886 |
| FGC: block readout | 12474 | 68.618 | 67.825 |
Neural Network Quantization
Neural network quantization represents weights, activations, and states with fewer bits to reduce memory use, computation, and deployment costs.
See all 438 papers