Test-Time Adaptation of Quantized ViTs via Single-Pass Quantizer-Aligned Recalibration

Hyeongheon Cha; Young D. Kwon; Sung-Ju Lee

cs.CVOct 6, 2026

Test-Time Adaptation of Quantized ViTs via Single-Pass Quantizer-Aligned Recalibration

Authors: Hyeongheon Cha, Young D. Kwon, Sung-Ju Lee

Organizations: KAIST · Samsung AI Center-Cambridge · University of Cambridge

Abstract

Post-training quantization is a standard route to fitting vision transformers (ViTs) into edge compute and memory budgets, yet quantized models become especially brittle under distribution shift. Test-time adaptation (TTA) addresses such shifts without labels, but most existing approaches are poorly aligned with the constraints of quantized inference. Prevailing TTA methods recover accuracy through backpropagation, while backprop-free methods often still incur overhead from extra forward passes or parameter updates, and lightweight feature- or logit-level methods recover only part of the loss. Across these approaches, a quantization-specific failure mode that amplifies the drop is not directly targeted: under shift, activations occupy frozen quantizers' calibrated ranges differently, distorting their code distribution. We propose Quantizer-Aligned Recalibration (QuAR), a single-pass TTA method tailored to quantized ViTs that neither backpropagates nor updates any model parameters. QuAR recalibrates activations at the input to a frozen quantizer, mapping the test stream's running per-channel statistics back toward the source calibration. On ImageNet-C with ViT-B, QuAR achieves the highest mean accuracy among state-of-the-art backprop-free TTA methods at 3-, 4-, 6- and 8-bit weight/activation precision, outperforming the strongest baseline by 2.28 points at 8 bits and 4.00 at 3 bits, with 46% lower latency and a memory overhead of only 0.17 MB (0.01% of peak inference memory). Analysis and diagnostics trace the gain to a reduced per-channel mismatch at these quantizers, which restores the code distribution the baselines leave unchanged or distort further. A single fixed configuration remains ahead across continual streams, non-i.i.d. label shift, seven out-of-distribution suites, and three other backbones.

Figures & tables

Figure 1: Quantization makes activation drift costlier; QuAR acts on the quantization path to undo it. (a) ViT-B accuracy on ImageNet-C by precision: unadapted accuracy falls from

55.6

28.4

at W3A3, while QuAR’s gain grows as precision drops, from

+3.8

to over

+6

at every quantized setting. (b) At the recalibrated quantizer inputs (W6A6), corruption drifts the code distribution off its source calibration; bars show the fraction removed (Wasserstein-

1

between code histograms, Appendix P ). Only QuAR recovers much of that drift,

{\sim}9\times

the best baseline.

Figure 2: Code resolution lost under shift (W6A6 ViT-B, ImageNet-C sev. 5). (a) Both rows carry the same nine positions, one grid step apart, against the same frozen grid: the shift moves and narrows it, so distinct values collapse onto one code (Proposition 1 ), and the same contraction would be free in full precision, where resolution is relative rather than absolute. Both magnitudes are measured

99

th percentiles of their per-channel distributions. (b) Fraction of all

21{,}504

channels at the frozen quantizers forfeiting at least a given resolution; other baselines in Appendix U .

Figure 3: Quantizer-Aligned Recalibration (QuAR). (a) QuAR recalibrates frozen quantizer inputs locally in the late ViT blocks and globally at the classifier input. (b) Under distribution shift, the running target moments

(\mu_{t},\sigma_{t})

drift from the source calibration

(\mu_{s},\sigma_{s})

, distorting the frozen quantizer’s code distribution. At each recalibrated interface, QuAR maps the activation toward the source operating point before quantization, restoring code usage toward the source.

	ImageNet-C accuracy (%), bit-width				Test-time cost			Mechanism
Method	W3A3	W4A4	W6A6	W8A8	#FP	Upd. par.	Lat. (ms)	Code dist.	Closed
Full precision	No Adapt $55.58$ , Ours $59.38_{\,\pm 0.01}$ (W32A32)				1	0	n/a	n/a	n/a
No Adapt	$28.35_{\,\pm 0.00}$	$47.30_{\,\pm 0.01}$	$47.68_{\,\pm 0.00}$	$54.36_{\,\pm 0.00}$	1	0	6.0	0 / 0	n/a
T3A	$28.49_{\,\pm 0.01}$	$47.42_{\,\pm 0.04}$	$47.79_{\,\pm 0.01}$	$54.16_{\,\pm 0.02}$	1	0	8.2	0 / 0	yes
FOA	$31.52_{\,\pm 0.09}$	$50.25_{\,\pm 0.07}$	$50.39_{\,\pm 0.05}$	$57.85_{\,\pm 0.16}$	2	2.3K	13.0	1 / 48	no
ZOA	$31.45_{\,\pm 0.05}$	$50.87_{\,\pm 0.09}$	$51.37_{\,\pm 0.04}$	$58.16_{\,\pm 0.05}$	2	$\approx$ 25K	12.3	2 / 25	no

Table 1: Mean top-1 (%) on ImageNet-C by bit-width. Lower block: component ablation . Columns: #FP, forwards/batch; Upd. par., test-time-updated parameters; Lat., per-image latency (ms); Code dist., percentage of the corruption-induced code-distribution drift removed at W6A6, as intermediate / head ; Closed, no iterative or zeroth-order optimization. Full precision : W32A32 reference.

BW	Method	#FP	R1	R2	R3	R4	R5	R6	R7	R8	R9	R10	Avg.
	No Adapt	1	28.35	28.35	28.35	28.35	28.35	28.35	28.35	28.35	28.35	28.35	$28.35_{\,\pm 0.00}$
	ZOA	2	31.27	31.61	31.31	32.10	32.06	32.08	32.10	32.15	32.34	32.40	$31.94_{\,\pm 0.25}$
	NEO-Continual	1	31.69	31.64	31.69	31.66	31.67	31.69	31.66	31.67	31.69	31.63	$31.67_{\,\pm 0.01}$
W3A3	QuAR	1	35.66	35.64	35.64	35.66	35.67	35.70	35.75	35.70	35.63	35.65	$\mathbf{35.67}_{\,\pm 0.02}$
	No Adapt	1	47.68	47.68	47.68	47.68	47.68	47.68	47.68	47.68	47.68	47.68	$47.68_{\,\pm 0.00}$
	ZOA	2	51.59	52.24	52.62	52.67	52.63	52.85	52.95	53.27	53.29	53.23	$52.73_{\,\pm 0.12}$

Table 2: Long-term continual adaptation: a

10

-round

\times

15

-corruption ImageNet-C stream with state carried throughout. Round-

r

is the 15-corruption mean of pass

r

; Avg. is the R1–R10 mean

\pm

std over three seeds. #FP: forwards/batch; No Adapt is stateless. QuAR and NEO-Continual use the EMA estimator at a matched

\alpha{=}0.1

. Best per bit-width bold .

Table 6

Setting	W4A4	W6A6	Trailing-block depth	W4A4	W6A6
Modules (blocks 8–11)			1 block (11)	$48.31_{\,\pm 0.03}$	$48.54_{\,\pm 0.02}$
attn.qkv + attn.proj	$50.18_{\,\pm 0.05}$	$50.72_{\,\pm 0.05}$	2 blocks (10–11)	$50.85_{\,\pm 0.03}$	$51.53_{\,\pm 0.05}$
+ mlp.fc1	$51.27_{\,\pm 0.05}$	$51.86_{\,\pm 0.05}$	3 blocks (9–11)	$52.03_{\,\pm 0.03}$	$52.56_{\,\pm 0.05}$
+ mlp.fc2	$\mathbf{52.87}_{\,\pm 0.03}$	$\mathbf{53.42}_{\,\pm 0.04}$	4 blocks (8–11)	$\mathbf{52.87}_{\,\pm 0.03}$	$\mathbf{53.42}_{\,\pm 0.04}$
Block position (4 blocks)			5 blocks (7–11)	$52.75_{\,\pm 0.02}$	$53.39_{\,\pm 0.07}$
early (blocks 0–3)	$30.93_{\,\pm 0.08}$	$32.12_{\,\pm 0.04}$	6 blocks (6–11)	$52.11_{\,\pm 0.06}$	$53.28_{\,\pm 0.03}$

Table 5: Recalibration-scope ablation, Local-only (no global branch), 3-seed mean

\pm

std; No Adapt is

47.30

47.68

at W4A4/W6A6. Left group: which modules within blocks 8–11, and the block position of the four recalibrated blocks, all modules. Right group: trailing-block depth , the number of late blocks recalibrated, all modules. Deployed default configuration bold .

Table 8

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Method	W3A3	W4A4	W6A6	W8A8
No Adapt	28.26	47.24	47.60	54.21
T3A	$29.68_{\,\pm 0.01}$	$48.62_{\,\pm 0.00}$	$49.11_{\,\pm 0.00}$	$55.76_{\,\pm 0.01}$
FOA	$31.25_{\,\pm 0.07}$	$50.28_{\,\pm 0.06}$	$50.50_{\,\pm 0.22}$	$58.17_{\,\pm 0.00}$
ZOA	$32.06_{\,\pm 0.15}$	$51.10_{\,\pm 0.14}$	$52.91_{\,\pm 0.05}$	$58.69_{\,\pm 0.02}$
NEO	$31.73_{\,\pm 0.01}$	$50.32_{\,\pm 0.01}$	$51.43_{\,\pm 0.00}$	$58.06_{\,\pm 0.00}$
FOZO	$31.88_{\,\pm 0.04}$	$50.03_{\,\pm 0.08}$	$49.94_{\,\pm 0.26}$	$58.05_{\,\pm 0.15}$

Appendix

Table 8: Full ImageNet-C evaluation (

50

k images per corruption, all

15

corruptions, sev. 5, quantized ViT-B, batch 64, episodic; 3-seed mean

\pm

std over seeds

0/1/2

, No Adapt within

0.01

across seeds). Mean top-1 (%) by bit-width, replicating the accuracy comparison of Table 1 on the full test set rather than the

5

k subset. Lower block: component ablation (Local-only, Global-only, Ours). QuAR is the best method at every bit-width and the ranking matches the

5

k main table; the W6A6 column also agrees with the i.i.d. column of Table 24 . Best per column bold .

Method	Gauss	Shot	Impul.	Defoc.	Glass	Motion	Zoom	Snow	Frost	Fog	Bright.	Contr.	Elastic	Pixel	JPEG	Avg.
No Adapt	14.52	13.42	13.76	27.42	18.56	27.20	27.90	25.60	35.30	37.82	63.76	4.78	28.90	35.30	51.02	28.35
T3A	14.36	13.32	13.86	27.24	18.40	27.50	28.00	26.18	35.56	38.62	63.84	4.72	29.14	35.50	51.20	28.50
FOA	16.74	15.06	15.66	30.20	20.66	31.48	31.40	31.46	38.48	44.92	64.98	8.40	32.70	37.76	52.88	31.52
ZOA	16.56	15.34	16.12	29.48	20.92	30.64	31.24	32.58	37.90	43.86	64.94	7.90	33.20	38.30	52.04	31.40
NEO	16.98	16.16	16.52	30.84	20.98	31.14	31.26	31.98	39.12	45.04	65.02	8.00	33.80	38.30	53.38	31.90
FOZO	17.72	16.32	16.92	28.78	21.40	31.90	32.14	33.18	39.94	43.20	65.24	7.40	34.90	39.54	53.70	32.15

Appendix

Table 9: Full per-corruption top-1 accuracy (%) on ImageNet-C, by quantization bit-width. Ours uses

\lambda{=}0.5

; baselines as in Table 1 ; seed 0. Best per (bit-width, corruption) in bold .

Method	Gauss	Shot	Impul.	Defoc.	Glass	Motion	Zoom	Snow	Frost	Fog	Bright.	Contr.	Elastic	Pixel	JPEG	Avg.
Local-only	46.48	45.06	47.24	43.98	39.68	48.98	45.12	61.82	63.14	63.72	76.44	33.80	54.24	63.38	68.46	53.44
Global-only	45.40	44.30	46.00	42.02	34.42	45.92	39.28	57.92	62.70	66.64	75.60	36.26	43.66	61.36	65.92	51.16
QuAR	46.80	45.74	47.26	44.88	40.64	49.60	46.00	62.36	64.04	66.32	76.90	37.46	55.26	63.56	68.68	54.37

Appendix

Table 10: Component ablation of QuAR: per-corruption accuracy on ImageNet-C sev. 5 (W6A6 PTQ4ViT ViT-B, 5k subset, bs=64, seed 0, episodic reset; cumulative running target moments, lagged). Companion to Table 1 (lower block).

Variance match $\lambda$	0	0.1	0.3	0.5	0.7	1.0
Mean top-1 (%)	$51.20_{\,\pm 0.02}$	$52.61_{\,\pm 0.01}$	$54.00_{\,\pm 0.03}$	$\mathbf{54.35}_{\,\pm 0.02}$	$54.20_{\,\pm 0.06}$	$53.52_{\,\pm 0.01}$
Batch size	1	4	8	16	64	128
Mean top-1 (%)	$54.27_{\,\pm 0.04}$	$54.39_{\,\pm 0.04}$	$54.47_{\,\pm 0.02}$	$54.35_{\,\pm 0.01}$	$\mathbf{54.35}_{\,\pm 0.02}$	$54.29_{\,\pm 0.06}$
Source-stat batches	16	32	64	128	200	256
Mean top-1 (%)	$53.65_{\,\pm 0.02}$	$53.88_{\,\pm 0.08}$	$54.18_{\,\pm 0.02}$	$54.29_{\,\pm 0.06}$	$\mathbf{54.35}_{\,\pm 0.02}$	$54.33_{\,\pm 0.04}$

Appendix

Table 11: Full QuAR sensitivity sweep, 3-seed mean

\pm

std over seeds

0/1/2

(W6A6 ViT-B, ImageNet-C 15-corruption 5k, cumulative estimator; expanded from Table 7 ). Deployed defaults in bold ; per-cell std

\leq 0.08

	i.i.d. stream		class-sorted stream
Batch	$\rho{=}3$	$\rho{=}\infty$	$\rho{=}3$	$\rho{=}\infty$
$64$	$54.37$	$54.54$	$54.18$	$54.37$
$8$	$54.49$	$54.51$	$53.36$	$53.19$
$4$	$54.35$	$54.40$	$52.70$	$52.53$
$1$	$\mathbf{54.31}$	$53.29$	$\mathbf{51.86}$	$50.26$

Appendix

Table 12: Variance-ratio clamp ablation (W6A6, 15-corruption mean, cumulative estimator, seed 0): default clamp

\rho{=}3

vs. no clamp

\rho{=}\infty

, across batch size and stream order (i.i.d. vs. class-sorted). The clamp is essentially free at batch

\geq 4

(

\leq 0.2

points), but at batch

1

recovers

+1.0

(i.i.d.) and

+1.6

(class-sorted) points, where few tokens or class concentration drive per-channel

\sigma_{t,c}\!\to\!0

and the unclamped ratio diverges; the estimator’s batch-size invariance thus relies on it.

Local est.	Global est.	bs $64$	bs $1$
cumulative	cumulative	$\mathbf{54.35}_{\,\pm 0.02}$	$\mathbf{54.27}_{\,\pm 0.04}$
Welford	Welford	$54.33_{\,\pm 0.03}$	$54.21_{\,\pm 0.02}$
Welford	cumulative	$54.33_{\,\pm 0.02}$	$54.22_{\,\pm 0.01}$
cumulative	Welford	$54.34_{\,\pm 0.03}$	$54.25_{\,\pm 0.04}$

Appendix

Table 13: Target-moment estimator ablation (W6A6, 15-corruption mean top-1 %, ImageNet-C 5k subset, 3-seed mean

\pm

std over seeds

0/1/2

). Deployed cumulative estimator vs. a streaming Welford (exact pooled) estimate, applied to the local branch, the global (CLS) branch, or both, at batch sizes

64

and

1

. All within

{\sim}0.15

points; at batch size

64

the four variants are statistically tied, and at batch size

1

the deployed estimator is best. Deployed configuration bold .

	i.i.d. stream		label shift
Local target moments	bs $64$	bs $1$	$10^{3}$	$5{\times}10^{5}$
Current batch only	$52.37_{\,\pm 0.01}$	$38.11_{\,\pm 0.01}$	$42.39$	$23.89$
Running, cumulative (deployed)	$\mathbf{54.35}_{\,\pm 0.02}$	$\mathbf{54.27}_{\,\pm 0.04}$	$\mathbf{54.72}$	$\mathbf{54.82}$
Converged over an extra pass	$54.43$	$54.53$	$54.74$	$54.66$

Appendix

Table 14: How much the target moments need to be pooled (W6A6 ViT-B, 15-corruption mean top-1 %). The local branch’s target moments are the deployed running estimate, the moments of the batch at hand, or the values the running estimate converges to over a full extra pass through the stream; the global branch is the deployed estimate in every row. Correcting with the batch at hand needs no extra forward, since the hook sees the whole batch before it corrects. Batch size

64

and

1

: ImageNet-C 5k subset, 3-seed mean

\pm

std, except the converged row, which is seed 0. Label shift: full

50

k images per corruption, SAR imbalance ratio

10^{3}

and

5{\times}10^{5}

, seed 0. Deployed configuration bold .

	i.i.d. stream			label shift
Design	$K{=}1$	$K{=}5$	full ( $K{=}79$ )	$10^{3}$	$5{\times}10^{5}$	bs 1
Deployed (global inclusive, local lagged)	51.11	53.33	54.35	54.72	54.82	54.31
Global branch also lagged	48.51	52.59	54.31	54.75	54.97	54.30
Local branch also inclusive	53.09	53.47	54.33	53.88	51.96	54.24

Appendix

Table 15: When each branch updates (W6A6 ViT-B, ImageNet-C sev. 5, 15 corruptions). Deployed : the global branch corrects with running moments that include the current batch, the local branch with the moments of the preceding batches only. The two alternatives make the branches symmetric. First batch (

K{=}1

), first five batches (

K{=}5

) and the full stream (

K{=}79

5

k subset, batch 64, 3-seed mean. Label shift: full

50

k images per corruption, imbalance ratio

10^{3}

and

5{\times}10^{5}

, seed 0. Batch size 1:

5

k subset, seed 0. Mean top-1 (%).

Local	Global	bs $64$	bs $1$
off	off	47.68	47.75
mean	off	$50.55_{\,\pm 0.02}$	$50.56_{\,\pm 0.01}$
mean+var	off	$53.42_{\,\pm 0.04}$	$53.38_{\,\pm 0.04}$
off	mean	$51.15_{\,\pm 0.02}$	$49.61_{\,\pm 0.08}$
off	mean+var	$51.20_{\,\pm 0.02}$	$51.25_{\,\pm 0.01}$
mean	mean	$51.78_{\,\pm 0.00}$	$50.22_{\,\pm 0.01}$

Appendix

Table 16: Branch

\times

recalibration-mode factorial (W6A6, 15-corruption mean top-1 %, ImageNet-C 5k subset, 3-seed mean

\pm

std over seeds

0/1/2

; cumulative estimator). Each branch is off, mean (recenter only), or mean+var (recenter and rescale). QuAR is (local mean+var, global mean+var), bold ; (off, off) is No Adapt.

Figure 4: Effect of the recalibration strength

\lambda

on W6A6 accuracy across three backbones (ViT-B, DeiT-B, ViT-S; seed 0); the default

\lambda{=}0.5

is marked dotted.

Trailing blocks	w/o fc2	with fc2	$\Delta$
1 (block 11)	49.08	48.55	$-0.53$
2 (blocks 10–11)	50.88	51.48	$+0.60$
3 (blocks 9–11)	51.34	52.59	$+1.25$
4 (blocks 8–11)	51.85	53.44	$+1.59$
5 (blocks 7–11)	52.10	53.39	$+1.29$
6 (blocks 6–11)	51.79	53.31	$+1.52$

Appendix

Table 17: Full per-depth contrast of recalibrating the post-GELU mlp.fc2 interface (Local-only, W6A6; mean top-1 %, seed 0). Each row recalibrates the trailing blocks at {qkv, proj, fc1}, without (w/o fc2 ) vs. with the fc2 interface added;

\Delta

is the fc2 gain. The deployed blocks 8–11 configuration is in bold .

Modules (blocks 8–11)
Setting	W3A3	W4A4	W6A6	W8A8
No Adapt	$28.35_{\,\pm 0.00}$	$47.30_{\,\pm 0.00}$	$47.68_{\,\pm 0.00}$	$54.36_{\,\pm 0.00}$
attn.qkv + attn.proj	$31.75_{\,\pm 0.02}$	$50.18_{\,\pm 0.05}$	$50.72_{\,\pm 0.05}$	$57.10_{\,\pm 0.02}$
+ mlp.fc1	$32.97_{\,\pm 0.04}$	$51.27_{\,\pm 0.05}$	$51.86_{\,\pm 0.05}$	$58.01_{\,\pm 0.01}$
+ mlp.fc2	$\mathbf{35.30}_{\,\pm 0.06}$	$\mathbf{52.87}_{\,\pm 0.03}$	$\mathbf{53.42}_{\,\pm 0.04}$	$\mathbf{59.11}_{\,\pm 0.02}$
Block position (4 blocks, all modules)

Appendix

Table 18: Recalibration-scope ablation at all four bit-widths, Local-only with no global CLS branch, 15-corruption mean top-1 (%) on the 5k subset, batch 64, episodic, 3-seed mean

\pm

std. Deployed default configuration bold ; W4A4 and W6A6 reproduce Table 5 .

$\overline{\mathrm{C}}$ (ImageNet- $\overline{\mathrm{C}}$ )
Method	W3A3	W4A4	W6A6	W8A8
No Adapt	$36.63_{\,\pm 0.00}$	$55.65_{\,\pm 0.01}$	$59.07_{\,\pm 0.00}$	$63.94_{\,\pm 0.00}$
T3A	$36.82_{\,\pm 0.03}$	$56.09_{\,\pm 0.03}$	$59.83_{\,\pm 0.02}$	$64.54_{\,\pm 0.04}$
FOA	$40.86_{\,\pm 0.07}$	$59.92_{\,\pm 0.07}$	$63.40_{\,\pm 0.09}$	$67.85_{\,\pm 0.11}$
ZOA	$40.65_{\,\pm 0.17}$	$59.97_{\,\pm 0.16}$	$\underline{64.02}_{\,\pm 0.08}$	$67.84_{\,\pm 0.06}$
NEO	$41.17_{\,\pm 0.02}$	$\underline{60.11}_{\,\pm 0.02}$	$63.94_{\,\pm 0.04}$	$67.80_{\,\pm 0.01}$

Appendix

Table 19: Full per-method results on the four OOD generalization benchmarks (quantized ViT-B; 3-seed mean

\pm

std over seeds 0/1/2).

\overline{\mathrm{C}}

: ImageNet-

\overline{\mathrm{C}}

(10 algorithmic corruptions); R: ImageNet-R; Sk.: ImageNet-Sketch; 3DCC: ImageNet-3DCC (12 3D common corruptions). Best per column bold , second underlined .

Method	W3A3	W4A4	W6A6	W8A8
No Adapt	$35.76_{\,\pm 0.00}$	$56.53_{\,\pm 0.00}$	$56.73_{\,\pm 0.00}$	$62.74_{\,\pm 0.00}$
T3A	$35.84_{\,\pm 0.02}$	$56.53_{\,\pm 0.01}$	$56.74_{\,\pm 0.01}$	$62.79_{\,\pm 0.04}$
FOA	$38.28_{\,\pm 0.08}$	$57.95_{\,\pm 0.23}$	$57.88_{\,\pm 0.11}$	$63.85_{\,\pm 0.06}$
ZOA	$38.45_{\,\pm 0.19}$	$58.33_{\,\pm 0.11}$	$58.83_{\,\pm 0.07}$	$64.25_{\,\pm 0.12}$
NEO	$38.90_{\,\pm 0.04}$	$\underline{58.26}_{\,\pm 0.07}$	$\underline{58.93}_{\,\pm 0.01}$	$64.11_{\,\pm 0.03}$
FOZO	$\underline{39.31}_{\,\pm 0.12}$	$58.10_{\,\pm 0.24}$	$57.40_{\,\pm 0.19}$	$\underline{64.39}_{\,\pm 0.14}$

Appendix

Table 20: Held-out ImageNet-C: the four extra ImageNet-C corruptions (Gaussian blur, saturate, spatter, speckle noise) not in the standard 15, sev. 5; mean over the 4 corruptions. Quantized ViT-B; 3-seed mean

\pm

std over seeds 0/1/2. Best per column bold , second underlined .

CIFAR-10-C
Method	W3A3	W4A4	W6A6	W8A8
No Adapt	$76.34_{\,\pm 0.00}$	$81.05_{\,\pm 0.00}$	$82.86_{\,\pm 0.00}$	$84.39_{\,\pm 0.00}$
T3A	$76.67_{\,\pm 0.01}$	$81.29_{\,\pm 0.01}$	$83.15_{\,\pm 0.01}$	$84.54_{\,\pm 0.01}$
FOA	$78.04_{\,\pm 0.05}$	$82.22_{\,\pm 0.02}$	$83.84_{\,\pm 0.05}$	$85.19_{\,\pm 0.03}$
ZOA	$78.35_{\,\pm 0.03}$	$82.57_{\,\pm 0.07}$	$84.54_{\,\pm 0.11}$	$85.82_{\,\pm 0.05}$
NEO	$78.42_{\,\pm 0.01}$	$82.42_{\,\pm 0.00}$	$84.05_{\,\pm 0.01}$	$85.26_{\,\pm 0.01}$

Appendix

Table 21: Generalization beyond ImageNet: CIFAR-10-C and CIFAR-100-C (severity 5, 15 corruptions, full

10

k images per corruption) and DomainNet-126 (mean over the three target domains, real as source), each on a ViT-B/16 fine-tuned on its own source data and quantized as in the main protocol; batch 64, episodic, 3-seed mean

\pm

std over sample order. Best bold , second underlined .

ImageNet-C
Method	W3A3	W4A4	W6A6	W8A8
FOA ( $K{=}2$ )	$31.52_{\,\pm 0.09}$	$50.25_{\,\pm 0.07}$	$50.39_{\,\pm 0.05}$	$57.85_{\,\pm 0.16}$
FOA ( $K{=}28$ )	$31.88_{\,\pm 0.13}$	$50.47_{\,\pm 0.07}$	$52.30_{\,\pm 0.24}$	$59.00_{\,\pm 0.11}$
QuAR	$\mathbf{35.98}_{\,\pm 0.00}$	$\mathbf{53.58}_{\,\pm 0.04}$	$\mathbf{54.35}_{\,\pm 0.02}$	$\mathbf{60.44}_{\,\pm 0.01}$
$\overline{\mathrm{C}}$ (ImageNet- $\overline{\mathrm{C}}$ )
FOA ( $K{=}2$ )	$40.86_{\,\pm 0.07}$	$59.92_{\,\pm 0.07}$	$63.40_{\,\pm 0.09}$	$67.85_{\,\pm 0.11}$

Appendix

Table 22: FOA at its minimum-forward setting (

K{=}2

, two forwards/batch) vs. its recommended CMA-ES population (

K{=}28

{\sim}29

forwards; Niu et al. (2024) default), and QuAR (one forward), across ImageNet-C and the four generalization suites (quantized ViT-B; W6A6/W8A8 PTQ4ViT, W4A4/W3A3 AdaLog; 3-seed mean

\pm

std). At

{\sim}14\times

the forwards,

K{=}28

stays below QuAR at every bit-width except ImageNet-R. Best of the three bold .

Batch size	$1$	$64$	$128$	$512$
No Adapt	47.75	47.69	47.69	47.68
AQR ( Mehrbod et al., 2026 ) (our impl.)	$18.73_{\,\pm 0.00}$	$51.37_{\,\pm 0.04}$	$51.63_{\,\pm 0.06}$	$51.75_{\,\pm 0.02}$
QuAR	$\mathbf{54.27}_{\,\pm 0.05}$	$\mathbf{54.35}_{\,\pm 0.02}$	$\mathbf{54.29}_{\,\pm 0.08}$	$\mathbf{53.77}_{\,\pm 0.02}$

Appendix

Table 23: Comparison with AQR on quantized W6A6 ViT-B (ImageNet-C,

15

-corruption mean top-1 %,

5

k subset, sev. 5; 3-seed mean

\pm

std over seeds

0/1/2

). AQR is our implementation of the paper-default ViT configuration (after-LayerNorm, top half blocks 6–11,

101

percentiles, standard raw-tail handling); No Adapt is measured at each batch size at seed

0

, varying by

0.07

over this range. Best per column bold .

Method	$1$	$333$	$10^{3}$	$3{\times}10^{3}$	$5{\times}10^{5}$
No Adapt	$47.55$
T3A	$48.67_{\,\pm 0.01}$	$48.61_{\,\pm 0.03}$	$48.19_{\,\pm 0.02}$	$47.60_{\,\pm 0.09}$	$47.24_{\,\pm 0.11}$
FOA	$50.19_{\,\pm 0.11}$	$50.32_{\,\pm 0.24}$	$50.33_{\,\pm 0.14}$	$50.10_{\,\pm 0.35}$	$49.86_{\,\pm 0.33}$
ZOA	$52.81_{\,\pm 0.06}$	$52.86_{\,\pm 0.05}$	$52.68_{\,\pm 0.11}$	$52.56_{\,\pm 0.12}$	$52.05_{\,\pm 0.04}$
NEO	$51.39_{\,\pm 0.00}$	$51.45_{\,\pm 0.08}$	$51.39_{\,\pm 0.04}$	$51.33_{\,\pm 0.11}$	$51.24_{\,\pm 0.04}$
FOZO	$49.49_{\,\pm 0.16}$	$49.77_{\,\pm 0.26}$	$49.66_{\,\pm 0.21}$	$49.45_{\,\pm 0.26}$	$48.92_{\,\pm 0.21}$

Appendix

Table 24: Non-i.i.d. online label shift by imbalance ratio,

1

being i.i.d. and

5{\times}10^{5}

near class-sorted. W6A6, Full ImageNet-C (

50

k/corruption), 15-corruption mean (%), 3 seeds; best bold . These are the values behind Figure 5 .

Figure 5: Online label shift (W6A6, full ImageNet-C, 3-seed mean) by imbalance ratio; labels give each method’s accuracy change from i.i.d. to

5{\times}10^{5}

. QuAR is the most accurate at every ratio and the only method whose accuracy does not fall.

	W3A3	W4A4	W6A6	W8A8
No Adapt	$76.76$	$82.88$	$82.74$	$84.12$
QuAR	$76.62$	$82.96$	$82.66$	$83.70$
$\Delta$ (QuAR $-$ No Adapt)	$-0.14$	$+0.08$	$-0.08$	$-0.42$

Appendix

Table 25: Clean-source (no-shift) sanity: top-1 accuracy (%) on clean ImageNet validation (no corruption; RobustBench 5k subset, ViT-B, batch size 64, seed 0; W6A6/W8A8 PTQ4ViT, W4A4/W3A3 AdaLog). QuAR uses its deployed configuration.

\Delta

is QuAR minus No Adapt; a value near zero means the recalibration does not harm the in-distribution operating point.

Method	W3A3		W4A4		W6A6		W8A8
	AdaLog	PTQ4ViT	AdaLog	PTQ4ViT	AdaLog	PTQ4ViT	AdaLog	PTQ4ViT
No Adapt	28.35	0.35	47.30	18.93	54.79	47.69	53.91	54.36
T3A	28.50	0.42	47.42	18.95	55.05	47.80	53.54	54.17
FOA	31.49	0.71	50.33	22.75	57.83	50.40	56.96	57.88
ZOA	31.40	0.79	50.78	23.44	57.93	51.35	57.30	58.21
NEO	31.90	0.84	50.29	23.59	58.24	51.36	57.40	58.10

Appendix

Table 26: Cross-quantizer comparison: mean top-1 (%) on ImageNet-C with both PTQ methods at every bit-width. QuAR: per-quantizer source stats (

\lambda{=}0.5

, cumulative moments, blocks 8–11

+

CLS). PTQ4ViT degrades sharply at low bit-widths (near-chance at W3A3), hence AdaLog in the main protocol; ZOA’s source stats are collected on the deployed quantizer, so it is slightly mismatched here. Best per column bold , second underlined .

Method	W4A4	W4A8	W8A8
No Adapt	47.30	54.12	54.36
T3A	$47.42_{\,\pm 0.04}$	$53.74_{\,\pm 0.01}$	$54.16_{\,\pm 0.02}$
FOA	$50.25_{\,\pm 0.07}$	$57.55_{\,\pm 0.14}$	$57.85_{\,\pm 0.16}$
ZOA	$50.87_{\,\pm 0.09}$	$58.10_{\,\pm 0.01}$	$58.16_{\,\pm 0.05}$
FOZO	$50.09_{\,\pm 0.02}$	$57.88_{\,\pm 0.14}$	$57.74_{\,\pm 0.20}$
NEO	$50.29_{\,\pm 0.02}$	$58.01_{\,\pm 0.02}$	$58.09_{\,\pm 0.03}$

Appendix

Table 27: Asymmetric bit-widths. Mean top-1 (%) on ImageNet-C sev. 5, 15 corruptions, 5k subset, batch 64, episodic, 3-seed mean

\pm

std; No Adapt within

0.01

across seeds. W4A4 and W8A8 are reproduced from Table 1 .

	FP (W32)	W8A8	W6A6	W4A4	W3A3
Clean	$84.3$	$84.1$	$82.7$	$82.9$	$76.8$
ImageNet-C	$55.6$	$54.4$	$47.7$	$47.3$	$28.4$
Gap (clean $-$ corrupt)	$28.7$	$29.8$	$35.1$	$35.6$	$48.4$

Appendix

Table 28: Unadapted clean vs. ImageNet-C (sev. 5) accuracy by precision (5k subset, No Adapt, seed 0). The clean

\to

corrupt gap widens as precision drops.

Method	Acc.	Mean Mis. $\downarrow$	Std Mis. $\downarrow$	Comb. $\downarrow$	Reduction (%) $\uparrow$
No Adapt	47.69	0.235	0.112	0.347	0.0
Local-only	53.35	0.095	0.057	0.153	56.0
Global-only	51.16	0.235	0.112	0.347	0.0
QuAR	54.36	0.095	0.057	0.153	56.0
NEO	51.36	0.235	0.112	0.347	0.0

Appendix

Table 29: Intermediate activation mismatch at the

16

pre-quant / input-side interfaces the local branch acts on (blocks 8–11

\times

{attn.qkv, attn.proj, mlp.fc1, mlp.fc2};

\lambda{=}0.5

) on W6A6 ViT-B. Mismatch is the per-channel mean absolute deviation of the target moments from the source calibration moments, averaged over channels. Accuracy is mean top-1 over 15 corruptions (seed 0).

	$\\|e_{\ell}\\|$		clipped (%)		harmful flips (%)
	clean	corrupt	clean	corrupt	clean	corrupt
W8A8	$2404$	$2457$	$0.011$	$0.011$	$1.08$	$3.82$
W6A6	$6773$	$6600$	$0.045$	$0.035$	$3.50$	$11.79$
W4A4	$12384$	$11992$	$0.900$	$0.817$	$3.18$	$11.86$
W3A3	$17436$	$17233$	$2.717$	$2.369$	$10.18$	$29.70$

Appendix

Table 30: The shift collapses the margin while leaving the quantization error where it was (ViT-B,

5000

-image subset, clean and all 15 ImageNet-C corruptions at severity 5, No Adapt).

\|e_{\ell}\|

and the clipped fraction are measured at the sixteen recalibration interfaces; harmful flips are predictions correct in full precision and wrong once quantized. The full-precision margin is

3.78

clean against

2.05

corrupted, independent of the bit-width.

Precision	No Adapt	Global-only	Local-only	QuAR	Local Gain	Marg. Local Gain
W8A8	54.36	58.10	59.11	60.44	+4.75	+2.34
W6A6	47.68	51.20	53.42	54.35	+5.74	+3.15
W4A4	47.30	50.21	52.87	53.58	+5.57	+3.37
W3A3	28.35	31.43	35.30	35.98	+6.95	+4.55

Appendix

Table 31: Precision sweep of the local recalibration branch on ImageNet-C (3-seed mean top-1 % over seeds 0/1/2; accuracy columns from Table 1 ). All variants use source-initialized cumulative running target moments, lagged order,

\lambda{=}0.5

. Local Gain

=

Local-only

-

No Adapt; Marg. Local Gain

=

QuAR

-

Global-only.

Placement	W3A3	W4A4	W6A6	W8A8
No Adapt	$28.35_{\,\pm 0.00}$	$47.30_{\,\pm 0.01}$	$47.68_{\,\pm 0.00}$	$54.36_{\,\pm 0.00}$
Before the quantizer, input side (QuAR)	$\mathbf{35.30}_{\,\pm 0.07}$	$\mathbf{52.87}_{\,\pm 0.03}$	$\mathbf{53.42}_{\,\pm 0.05}$	$\mathbf{59.11}_{\,\pm 0.02}$
After the quantizer, output side	$27.98_{\,\pm 0.03}$	$47.01_{\,\pm 0.07}$	$51.90_{\,\pm 0.01}$	$57.96_{\,\pm 0.04}$
Block output, residual stream	$32.18_{\,\pm 0.05}$	$50.32_{\,\pm 0.12}$	$51.14_{\,\pm 0.05}$	$57.29_{\,\pm 0.02}$
Global-only (Table 1 )	$31.43_{\,\pm 0.02}$	$50.21_{\,\pm 0.02}$	$51.20_{\,\pm 0.03}$	$58.10_{\,\pm 0.01}$

Appendix

Table 32: Placement ablation (mean top-1 %, 3-seed mean

\pm

std). Every variant uses the deployed local configuration (blocks 8–11,

\lambda{=}0.5

, cumulative estimator, no global branch); only where the same per-channel affine is applied differs: before the activation quantizer, on the tensor it is about to round (our default, the Local-only row of Table 1 ); after it, on the dequantized tensor of the same interface; or at the block output, the residual stream that no quantizer reads and the site at which normalization-statistics adaptation acts. The Global-only branch of Table 1 , which recalibrates only the head input, is repeated for reference. Best in bold .

Method	#FP	GFLOPs/img	Peak mem (MB)	Latency (ms/img)
No Adapt	1	35.2	1601	6.0
T3A	1	35.2	1627	8.2 ∗
FOA	2	71.4	1619	13.0
ZOA	2	70.3	1606	12.3
NEO	1	35.2	1602	6.0
FOZO	2	71.4	1619	12.3

Appendix

Table 33: Measured test-time cost on W6A6 ViT-B (batch size 64, ImageNet-C). #FP: forward passes per batch. GFLOPs/image profiled over the full per-batch adaptation. Peak memory: max_memory_allocated (per-process), median over 5 reps. Latency: per-image forward (protocol above). ∗ T3A latency is early-stream (grows with its support set).

	TITAN RTX	Orin Nano
Method	batch $64$	batch $64$	batch $16$
No Adapt	$1.00\times$	$1.000\times$	$1.000\times$
NEO	$1.00\times$	$1.002\times$	$1.000\times$
QuAR	$\mathbf{1.10\times}$	$\mathbf{1.071\times}$	$\mathbf{1.084\times}$
T3A	$1.36\times$	$1.144\times$	$1.465\times$
ZOA	$2.04\times$	$2.001\times$	$1.998\times$

Appendix

Table 34: Per-image latency as a ratio to No Adapt on the same device and batch (W6A6 ViT-B). Device values are medians over

15

batches at batch

64

and

25

at batch

16

, over

3

repetitions. Batch

16

is the largest batch at which the backprop-based methods fit this board.

Batch	Device	No Adapt	Buffered	In place	Residual
$64$	TITAN RTX	$1600.91$	$1748.83$	$\mathbf{1601.08}$	$+0.17$
$64$	Orin Nano	$1603.16$	$1751.33$	$\mathbf{1603.33}$	$+0.17$
$16$	Orin Nano	$\phantom{0}909.47$	$\phantom{0}946.58$	$\phantom{0}\mathbf{909.64}$	$+0.17$

Appendix

Table 35: Peak memory of the local correction, buffered against applied in place ( --quar_local_chunk 2 ), at W6A6. Residual is over No Adapt on the same device and batch. The two forms are bit-identical, so no accuracy figure depends on the choice.

Method	Backprop	Updates params	Passes/batch	Frozen-quant deployable
TENT / EATA / SAR	yes	LN affine	1F + 1B	no
ActMAD	yes	all params	1F + 1B	no
TTT++	yes	feature extractor	1F + 1B	no
No Adapt	no	none	1F	yes
T3A / NEO	no	none	1F	yes
FOA / FOZO	no	prompt (grad-free)	2F	yes

Appendix

Table 36: Two families of test-time adaptation. The backprop-based family updates parameters by gradient and cannot run on a frozen quantized inference graph; our comparison set (and QuAR) is backprop-free . F: forward, B: backward pass per batch.

	No Adapt	SAR	TENT	EATA	SPA	QuAR
Mean top-1	$47.68$	$49.77_{\,\pm 0.11}$	$50.05_{\,\pm 0.13}$	$52.79_{\,\pm 0.53}$	$59.14_{\,\pm 0.15}$	$\mathbf{54.35}_{\,\pm 0.02}$

Appendix

Table 37: Seed spread behind Table 7 , which reports the means alone (W6A6, ImageNet-C 15-corruption mean, 3 seeds,

\pm

sample std). No Adapt performs no adaptation and its spread across seeds is below

0.01

	$16$ local interfaces ( $21{,}504$ channels)					head ( $768$ )
	mean $b_{c}$	$b_{c}\!\geq\!0.5$	$b_{c}\!\geq\!1$	99th pct.	worst	mean $b_{c}$
No Adapt	$0.277$	$20.6\%$	$4.8\%$	$1.43$	$2.51$	$0.003$
T3A	$0.277$	$20.6\%$	$4.8\%$	$1.43$	$2.51$	$0.003$
FOA	$0.319$	$24.3\%$	$6.3\%$	$1.54$	$3.27$	$0.115$
FOZO	$0.386$	$28.7\%$	$10.6\%$	$1.83$	$3.71$	$0.221$
ZOA	$0.274$	$20.4\%$	$4.6\%$	$1.43$	$2.49$	$0.007$

Appendix

Table 38: Per-channel code resolution lost under shift (W6A6 ViT-B, ImageNet-C sev. 5, all 15 corruptions, 5k subset; all

17

recalibration interfaces). For each channel we report

b_{c}=\log_{2}(\sigma_{s,c}/\sigma_{t,c})

, the code levels it forfeits against its own calibration (Proposition 1 ); the mean counts a channel that expands as zero, since the proposition is silent on those. T3A, NEO and Global-only act at or after the classifier, downstream of these interfaces, and are identical to No Adapt to every decimal. FOA and FOZO perturb input-side prompts and make the starvation worse. ZOA writes directly upstream, adapting the LayerNorm affine parameters of blocks 1–8, and still removes

1\%

. Local-only coincides with QuAR because both apply the local branch here. The head column is included for scope rather than as a second restoration result: the head input is post-LayerNorm and starves under no method, with no configuration putting a single one of its

768

channels past half a bit, so differences within that column are between negligible quantities and are not restoration.

		No Adapt		QuAR
Interface group	channels	mean $b_{c}$	$b_{c}\!\geq\!0.5$	mean $b_{c}$	$b_{c}\!\geq\!0.5$
attn.proj , mlp.fc2 (no preceding norm)	$15{,}360$	$0.365$	$28.9\%$	$0.238$	$9.7\%$
attn.qkv , mlp.fc1 (post-LayerNorm)	$6{,}144$	$0.056$	$0.0\%$	$0.030$	$0.0\%$
head input (post-LayerNorm)	$768$	$0.003$	$0.0\%$	$0.000$	$0.0\%$

Appendix

Table 39: Code-level starvation is confined to the quantizers with no LayerNorm in front of them (W6A6, ImageNet-C sev. 5, all 15 corruptions). Groups partition the

16

local interfaces by whether a LayerNorm immediately precedes the quantizer; the head input is listed separately and is also post-LayerNorm.

b_{c}\!\geq\!0.5

is the fraction of channels forfeiting at least half a bit of code resolution.

Corruption	T3A	FOA	FOZO	ZOA	NEO	QuAR (cum.)
Gaussian noise	$0.0$	$-2.5{\pm}1.9$	$-14.4{\pm}1.2$	$-0.4{\pm}1.6$	$-4.3$	$\mathbf{+58.9{\pm}0.2}$
Shot noise	$0.0$	$-1.0{\pm}0.1$	$-12.7{\pm}1.8$	$+0.9{\pm}0.2$	$-3.6$	$\mathbf{+57.2{\pm}0.6}$
Impulse noise	$0.0$	$+1.2{\pm}2.4$	$-13.6{\pm}1.9$	$-2.0{\pm}1.2$	$-4.8$	$\mathbf{+58.8{\pm}0.4}$
Defocus blur	$0.0$	$+9.4{\pm}2.4$	$-19.3{\pm}6.4$	$+6.6{\pm}2.9$	$+9.0$	$\mathbf{+40.5{\pm}0.2}$
Glass blur	$0.0$	$+10.2{\pm}2.2$	$-9.3{\pm}5.4$	$+4.4{\pm}3.4$	$+10.0$	$\mathbf{+46.4}$
Motion blur	$0.0$	$+15.6{\pm}3.3$	$-10.0{\pm}4.5$	$+7.0{\pm}1.5$	$+15.9$	$\mathbf{+42.2{\pm}0.7}$

Appendix

Table 40: Per-corruption quantizer code-distribution restored (%), W6A6. For each method, the percentage of the corruption-induced code-distribution drift removed at the late pre-quant interfaces (blocks 8–11

\times

{qkv, proj, fc1}; higher is better,

0=

No Adapt). Restored %

=(d_{\text{NoAdapt}}-d_{\text{method}})/d_{\text{NoAdapt}}\times 100

on the

L_{1}

code-histogram distance to the source calibration. 3-seed mean

{}\pm{}

std. QuAR uses the cumulative estimator. Reference: No Adapt

=0

by definition; T3A and NEO coincide with No Adapt at these interfaces (they modify only the classifier prototypes / head input, never the blocks 8–11 pre-quant activations), hence

0

. The Mean row is the unweighted mean over the 15 corruptions; Figure 1 b reports instead the drift-weighted summary (ratio of the 15-corruption mean distances;

+41.3\pm 0.1\%

for QuAR across the three seeds), which up-weights high-drift corruptions such as contrast.

Figure 6: Downstream perturbation gain by depth (W6A6 ViT-B, clean ImageNet; empirical local proxy for

K_{>\ell}

, full-precision composition). A perturbation injected at each block input induces a logit change

\|\Delta z\|/\|\delta\|

that generally decreases with depth (early blocks 0–3 mean

0.26

vs. late 8–11 mean

0.059

{\sim}4.4\times

; near-monotone). By Corollary U.1 , late interfaces have the smallest gain, so an imperfect correction there perturbs the logits least.

Explore similar work

Jun 20, 2026cs.CV

ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

Vision Transformers have achieved remarkable success in many fields, yet their deployment on edge devices remains challenging due to their substantial computational demands. Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead. However, most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances. Given the substantial diversity of natural images, the activation distributions vary significantly across samples, making these methods inherently suboptimal. In this paper, we propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs. We first reveal a hidden correlation between the distribution range of shallow-layer activations and the optimal scales of deeper layers. Based on this, we develop a scale learning mechanism that integrates an efficient range extraction approach to capture robust range statistics at the shallow stage, which are then fed into a Taylor-motivated polynomial scale projection module to generate all quantization scales simultaneously. With the efficiency of polynomial approximation, ScalePredictor introduces insignificant computational overhead while avoiding costly just-in-time calibration. Extensive experiments on ImageNet demonstrate that ScalePredictor consistently outperforms prior PTQ methods, achieving a more favorable accuracy-efficiency trade-off. Code and additional results are shown in the supplementary materials.

Changjun Li, Runqing Jiang, Lian Xu +3

School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · Computer Science and Software Engineering, The University of Western Australia, Perth, Australia · University of Oxford, Oxford, United Kingdom

Vision Transformer Transformer Architectures Model Activations

Ask PDF

Sep 28, 2026cs.CV

GLF-Q: Global-Local Feature-based Quantization for Vision Transformers

Post-training quantization (PTQ) efficiently compresses Vision Transformers (ViTs) without retraining, yet suffers severe accuracy degradation at low bit-widths. Existing optimization-based PTQ methods guide block reconstruction via either soft logits or second-order Hessian proxies. Logit supervision is prone to overfitting on limited calibration data, while Hessian approximations incur structural truncation errors. To address these limitations, we propose \textbf{GLF-Q}, a novel PTQ framework guided by Global-Local Feature alignment. GLF-Q propagates quantized block outputs through downstream full-precision layers to align penultimate-layer representations under local output regularization, providing downstream feature supervision without explicitly approximating the Hessian or using a Taylor expansion. Furthermore, offline Hadamard transformations are introduced with zero runtime overhead to disperse activation outliers across channels, effectively contracting dynamic ranges and reducing quantization errors. Meanwhile, optimizing this loss via a Straight-Through Estimator (STE) achieves rapid convergence, bypassing continuous relaxation rounding formulations such as AdaRound. Extensive experiments across representative ViT architectures demonstrate that GLF-Q with standard uniform quantizers substantially outperforms state-of-the-art methods under 3-bit quantization on image classification. In addition, GLF-Q exhibits strong out-of-domain calibration robustness and achieves speedups under 8-bit GPU deployment.

Peilin Sun, Guang Liang, Jin Tong +1

State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China · Zhongguancun Academy, Beijing 100094, China

Self-Supervised Vision Transformers Vision Transformer

Ask PDF

Jun 13, 2025cs.CV

GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers

Vision Transformers (ViTs) are essential in computer vision but are computationally intensive, too. Model quantization, particularly to low bit-widths like 4-bit, aims to alleviate this difficulty, yet existing Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) methods exhibit significant limitations. PTQ often incurs substantial accuracy drop, while QAT achieves high accuracy but suffers from prohibitive computational costs, limited generalization to downstream tasks, training instability, and lacking of open-source codebase. To address these challenges, this paper introduces General, Practical, and Lightning Quantization (GPLQ), a novel framework designed for efficient and effective ViT quantization. GPLQ is founded on two key empirical insights: the paramount importance of activation quantization and the necessity of preserving the model's original optimization basin'' to maintain generalization. Consequently, GPLQ employs a sequential activation-first, weights-later'' strategy. Stage 1 keeps weights in FP32 while quantizing activations with a feature mimicking loss in only 1 epoch to keep it stay in the same ``basin'', thereby preserving generalization. Stage 2 quantizes weights using a PTQ method. As a result, GPLQ is 100x faster than existing QAT methods, lowers memory footprint to levels even below FP32 training, and achieves 4-bit model performance that is highly competitive with FP32 models in terms of both accuracy on ImageNet and generalization to diverse downstream tasks, including fine-grained visual classification and object detection. We will release an easy-to-use open-source toolkit supporting multiple vision tasks.

Guang Liang, Xinyao Liu, Jianxin Wu

State Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · University of Science and Technology of China, Hefei, China

Quantization-Aware Training Self-Supervised Vision Transformers Vision Transformer Imagenet Transformer Architectures

Ask PDF