HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models

Xinmeng Liu; Xianglong Yan; Kaicheng Yang; Weisheng Dong; Yulun Zhang

cs.CVOct 11, 2026

HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models

Authors: Xinmeng Liu, Xianglong Yan, Kaicheng Yang, Weisheng Dong, Yulun Zhang

Organizations: Shanghai Jiao Tong University · Xi’an University of Electronic Science and Technology

Abstract

Recently, AutoRegressive Visual Generation (ARVG) models have emerged as a competitive paradigm for high-quality image synthesis. However, their substantial computational and memory demands make them difficult to deploy on resource-constrained edge devices. Post-Training Quantization (PTQ) provides a promising solution to reduce the computational and memory costs of ARVG models, yet existing methods often suffer from substantial image quality degradation under low-bit quantization. In this paper, we propose a training-free PTQ framework for ARVG, consisting of Tail-Aware Bounded-init Grid Refinement (T-BGR) and Coarse-to-Fine Activation Precision Scheduling (CAPS). T-BGR introduces a tail-aware reconstruction objective for weight quantizer calibration, assigning greater importance to distribution-tail coefficients and preserving fine-grained information without introducing additional parameters or high-precision branches. CAPS performs coarse-to-fine activation precision scheduling by jointly considering the importance and computational cost of different generation stages, enabling efficient mixed-precision allocation under a given bit budget. Extensive experiments on multiple ARVG backbones demonstrate that our framework consistently improves robustness under aggressive low-bit quantization. Notably, on Infinity, our framework achieves near full-precision image generation quality at an effective W4A5.6 activation precision, while existing methods at W4A6 still exhibit noticeable quality degradation or inconsistent performance. Code is available at https://github.com/lxm-lc/HGA-Quant.

Figures & tables

Figure 1: Motivation for HGA-Quant. Left: most weights concentrate near the central mass, but a non-negligible fraction lies in the tails, motivating tail-aware calibration. Right: token counts and linear-layer FLOPs grow sharply in later stages, motivating coarse-to-fine precision scheduling.

Figure 2: Overview of HGA-Quant. T-BGR performs tail-aware calibration for robust low-bit quantization, while CAPS assigns stage-wise precision in a coarse-to-fine manner for high-quality and efficient ARVG generation.

Quantization	Methods	Infinity-2B			HART
Quantization	Methods	GenEval $\uparrow$	ImageReward $\uparrow$	DPG-Bench $\uparrow$	DPG-Bench $\uparrow$	MJHQ-30K FID $\downarrow$
FP	–	0.71896	0.95485	83.4741	74.835	6.178
W8A8	RTN	0.72706	0.90395	83.0017	75.754	12.476
	SmoothQuant	0.72192	0.92828	83.0177	75.354	7.185
	SVDQuant	0.71827	0.93168	82.8461	75.411	7.477
	QuaRot	0.72866	0.94160	83.2944	75.019	6.380

Table 1: Unified comparison on Infinity-2B and HART.

Quantization	Methods	VAR-d30		FlexVAR
Quantization	Methods	FID $\downarrow$	IS $\uparrow$	FID $\downarrow$	IS $\uparrow$
FP	–	1.9827	323.326	2.1598	336.262
W8A8	RTN	1.9742	312.803	2.2093	329.612
	SmoothQuant	2.0138	318.025	2.1797	331.019
	SVDQuant	1.9799	319.311	2.1680	329.660
	QuaRot	1.9708	322.788	2.1604	335.229

Table 2: Unified comparison on VAR-d30 and FlexVAR.

Figure 3: Effect of the tail-stretch factor in T-BGR.

	no BGR	$\gamma=0$	$\gamma=1$	$\gamma=2$	$\gamma=8$	$\gamma=16$
W4A5.6	0.81594	0.81001	0.81458	0.80069	0.83523	0.81491
W4A6	0.82125	0.80062	0.80668	0.79104	0.82882	0.79840
W4A4	0.40181	0.50311	0.48880	0.49401	0.44772	0.43064

Table 3: Ablation studies on Infinity-2B. Left: ablation of the T-BGR tail-stretch factor

\gamma

. Right: ablation of CAPS. W4A5.6 combines W4A4 and W4A8 at a FLOPs-weighted effective activation precision of 5.5711 bits; “no BGR” omits bounded grid refinement.

Figure 4: CAPS stage sensitivity on Infinity-2B. The table reports FLOPs-weighted effective activation bits and ImageReward when changing one consecutive stage pair from A8 to A4; the curve plots the same ImageReward values with the uniform W4A8 reference as a dashed horizontal line.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Compact illustration of coarse-to-fine generation at selected stages. The generation process gradually evolves from coarse layouts to detailed images, with early stages determining global structures and later stages refining object-level details.

Figure 6: Stage-wise visualization of coarse-to-fine generation. We compare full precision (FP), uniform W4A6, CAPS with W4A5.6, and uniform W4A4 over all 13 generation stages. CAPS preserves the coarse structure with its high-precision prefix and maintains coherent refinement after switching to lower precision in the final stages, whereas uniform W4A4 exhibits visible artifacts throughout the trajectory.

Allocation	A8 stages	Avg. act. bits	ImageReward $\uparrow$	$\Delta$ vs. early-high
early-high (CAPS)	1–11	5.5711	0.83523	0.00000
late-high	13	5.5543	0.44577	-0.38946
random_01	1,2,3,4,5,6,8,9,11,12	5.9587	0.81186	-0.02337
random_02	1,2,3,4,5,10,13	5.9930	0.77997	-0.05526
random_03	1,2,3,4,5,6,7,8,9,10,12	5.8381	0.82448	-0.01075
random_04	1,2,3,4,5,6,7,8,13	5.9095	0.80030	-0.03493

Appendix

Table 4: Budgeted CAPS allocation comparison on Infinity-2B. Stage labels are one-based. The reported average bit-width is the FLOPs-weighted effective activation precision realized by each allocation, and all rows use the common allocation-comparison evaluation entry.

Figure 7: Comparisons across different methods. In this figure, all images were obtained with the model undergoing W4A6 quantization.

Figure 8: In this figure, the images labeled ”ours*” were obtained using our coarse-to-fine W4A5.6 approach, while the images from other methods were generated using W4A6 quantization.

Post-Training Quantization

Post-Training Quantization compresses trained neural networks into low-bit weights, activations, or caches to reduce memory use and inference cost.

See all 360 papers

Shift-and-Sum Quantization for Visual Autoregressive Models
Jun 15, 2026Jaehyeon Moon, Bumsub HamVisual Autoregressive Models Autoregressive Image Generation
1Yonsei University · 2Articron
Q-ARVD: Quantizing Autoregressive Video Diffusion Models
May 20, 2026Siao Tang, Xinyin Ma, Gongfan Fang +2Video Diffusion Models Diffusion Model Quantization
National University of Singapore · The Hong Kong Polytechnic University
GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers
Jun 13, 2025Guang Liang, Xinyao Liu, Jianxin WuQuantization-Aware Training 4-Bit Quantization
State Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · University of Science and Technology of China, Hefei, China

Jun 15, 2026J/K move · Enter open · S save

Shift-and-Sum Quantization for Visual Autoregressive Models

Jaehyeon Moon, Bumsub Ham

1Yonsei University · 2Articron

Visual Autoregressive Models Autoregressive Image Generation Post-Training Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) enables efficient deployment of deep networks using a small set of data. Its application to visual autoregressive models (VAR), however, remains relatively unexplored. We identify two key challenges for applying PTQ to VAR: (i) large reconstruction errors in attention-value products, especially at coarse scales where high attention scores occur more frequently; and (ii) a discrepancy between the sampling frequencies of codebook entries and their predicted probabilities due to limited calibration data. To address these challenges, we propose a PTQ framework tailored for VAR. First, we introduce a shift-and-sum quantization method that reduces reconstruction errors by aggregating quantized results from symmetrically shifted duplicates of value tokens. Second, we present a resampling strategy for calibration data that aligns sampling frequencies of codebook entries with their predicted probabilities. Experiments on class-conditional image generation, inpainting, outpainting, and class-conditional editing show consistent improvements across VAR architectures, establishing a new state of the art in PTQ for VAR.

Abstract

Post-Training Quantization

Explore similar work

Shift-and-Sum Quantization for Visual Autoregressive Models

Q-ARVD: Quantizing Autoregressive Video Diffusion Models

GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers