Post-training Quantization for Hybrid Iterative Generative Models

Jing Gao; Junyi Wu; Wei Wang; Yan Yan; Yao Zhao

Fig. 1: Overview of iterative generative model paradigms and post-training quantization challenges. IGMs include diffusion, autoregressive, and hybrid pipelines that couple autoregressive context updates with diffusion-style refinement (e.g., MAR, where an autoregressive Transformer conditions a denoising MLP to iteratively sample new tokens and update the token sequence). (Left) Excessive Outliers create an irreconcilable trade-off between sacrificing normal values and sacrificing outliers. Our Hierarchical Cluster Decoupling addresses this by isolating outlier channels while maintaining quantization precision for normal channels. (Right) Amplified Anomalies arising unpredictably from minor quantization errors, exceed vanilla PTQ’s static calibration ranges during diffusion, thus causing truncation artifacts. Our Scaling Recalibration scales anomalies into valid ranges, avoiding model collapse.

Fig. 2: Activation distribution comparison in a linear layer between hybrid IGMs and a conventional ViT-B. Using the maximum activation in ViT-B as a reference threshold, we highlight channels in hybrid IGMs that exceed this range. Hybrid IGMs exhibit sparse but significantly more pronounced outliers, revealing a more severe imbalance across channels.

Fig. 3: Comparison of full-precision and differently quantized activations in a linear layer, each including Excessive Outliers (EOs) and Normal Values . Notably, EOs significantly exceed the range of normal values. (a) Full-Precision: Original distribution of activations; (b) Sacrifice Normal Values: Preserve all outliers while compressing most normal values; (c) Sacrifice Excessive Outliers: Preserve normal values while truncating outliers with MSE metric; (d) Ours: Our method isolates outlier channels to preserve the range of outliers and maintain quantization precision for normal values.

Fig. 4: Comparison of denoising-related variables between hybrid IGM and conventional diffusion models. Subfigures (a), (b), and (c) show

\sqrt{1-\bar{\alpha}_{t}}

,

\gamma_{t}

, and

\frac{1}{\sqrt{\alpha_{t}}}\gamma_{t}

, respectively. Subfigure (d) reports the relative magnitude of boundary-layer inputs under quantization versus full precision.

Fig. 5: Boundary-layer activation distributions at selected denoising timesteps under full-precision and quantized inference. The Gaussian Bound is estimated from the full-precision distribution (Sec. IV-B ); quantized activations beyond this bound are marked as Amplified Anomalies (AAs) .

Fig. 6: Overview of the proposed HyGenQ framework. Starting from the hybrid IGM architecture, we identify two quantization challenges: Excessive Outliers and Amplified Anomalies. To address them, HyGenQ introduces two complementary modules. Hierarchical Cluster Decoupling separates normal and outlier channels and assigns them independent quantization parameters, while Scaling Recalibration rescales amplified boundary-layer inputs before quantization. Together, they mitigate both EOs and AAs, enabling robust quantized inference for hybrid IGMs.

Fig. 7: Illustration of Hierarchical Cluster Decoupling (HCD). (a) The module adopts a three-stage hierarchical clustering process to progressively identify and separate outlier channels from normal ones. (b) The optimal clustering configuration is selected by minimizing a Quantization Error Metric based on the Sum of Squared Errors (SSE), yielding the cluster number

k^{*}

in Eq. ( 16 ).

Fig. 8: Illustration of Scaling Recalibration (SR). We apply Scaling Recalibration to the boundary-layer activations in the diffusion procedure of hybrid IGMs. Specifically, a Gaussian Bound is first defined from the activation distribution of the full-precision model, providing a stable reference range for quantization. During quantized inference, amplified quantization errors may cause boundary-layer activations to exceed the calibration range. The proposed strategy rescales these out-of-bound activations into the bound and applies this fixed range across timesteps, thereby stabilizing boundary-layer quantization.

Method	Model	Precision (W/A)	Iteration	Timestep	Size	$\mathbf{\circ}$ FID $\downarrow$	$\mathbf{\circ}$ IS $\uparrow$	$\bullet$ FID $\downarrow$	$\bullet$ IS $\uparrow$
Full-Precision	MAR-B	32/32	64	100	208M	2.30	279.44	2.30	279.44
UniformQuant	MAR-B	8/8	64	100	52M	237.59	2.51	–	–
RepQ-ViT [ 18 ]	MAR-B	8/8	64	100	52M	298.71	1.60	–	–
SmoothQuant [ 24 ]	MAR-B	8/8	64	100	52M	10.31	144.51	253.28	1.97
PTQ4DM [ 25 ]	MAR-B	8/8	64	100	52M	10.34	144.22	252.41	1.98
TFMQ-DM [ 41 ]	MAR-B	8/8	64	100	52M	9.22	150.36	256.17	1.90

TABLE I: Performance comparison with 64 iterations and 100 timesteps, where

\mathbf{\circ}

indicates that the boundary layers in the diffusion procedure remain in full precision, while

\bullet

denotes that all linear layers are quantized.

Figure 10

Method	Model	Iteration	FID $\downarrow$	IS $\uparrow$
Full-Precision	MAR-B	32	2.46	274.23
$\mathbf{\circ}$ SmoothQuant [ 24 ]	MAR-B	32	11.93	139.61
$\mathbf{\circ}$ PTQ4DM [ 25 ]	MAR-B	32	11.88	139.66
$\mathbf{\circ}$ HyGenQ (Ours)	MAR-B	32	4.36	240.05
$\bullet$ SmoothQuant [ 24 ]	MAR-B	32	202.44	2.99
$\bullet$ PTQ4DM [ 25 ]	MAR-B	32	207.28	2.88

TABLE III: Performance comparison with 32 iterations and 100 timesteps. Marker

\mathbf{\circ}

indicates that the boundary layers in the diffusion procedure remain in full precision, while marker

\bullet

denotes that all linear layers are quantized.

Fig. 10: Visualization results with 64 iterations and 100 timesteps in MAR models. Marker

\mathbf{\circ}

indicates that the boundary layers in diffusion remain in full precision, while

\bullet

denotes that all linear layers are quantized. HyGenQ outperforms all baselines in both settings. Please zoom in for details.

Fig. 11: Visualization results under lower-bit quantization. W4A8 and W4A4 cause severe generation collapse for most methods.

Model	Method	$\mathbf{\circ}$ FID $\downarrow$	$\mathbf{\circ}$ IS $\uparrow$	$\bullet$ FID $\downarrow$	$\bullet$ IS $\uparrow$
MAR-B	QuaRot [ 71 ]	7.28	172.28	221.84	1.95
MAR-B	HadaNorm [ 72 ]	6.95	173.74	214.37	2.11
MAR-B	HyGenQ (Ours)	7.14	176.70	7.45	162.28
MAR-L	QuaRot [ 71 ]	5.96	187.90	216.92	2.08
MAR-L	HadaNorm [ 72 ]	5.95	187.68	208.56	2.24
MAR-L	HyGenQ (Ours)	6.06	193.94	6.54	190.17

TABLE IV: Additional comparison with outlier-suppression quantization methods on ImageNet

256\times 256

with 8,000 generated samples.

Method	Size	FID ↓	IS ↑
Full-Precision	208MB	2.30	279.44
$\mathbf{\circ}$ Baseline	52MB	10.34	144.22
$\mathbf{\circ}$ + HCD	52MB	3.50	247.33
$\bullet$ Baseline	52MB	252.41	1.98
$\bullet$ + SR	52MB	11.34	139.77
$\bullet$ + HCD + SR (HyGenQ)	52MB	3.61	246.00

TABLE V: Ablation study on ImageNet 256×256 with 64 iterations and 100 timesteps in MAR-B.

Size	Method	W/A	$\mathbf{\circ}$ Spd.	$\mathbf{\circ}$ T./Img.	$\mathbf{\circ}$ Param.	$\mathbf{\circ}$ Calib.	$\bullet$ Spd.	$\bullet$ T./Img.	$\bullet$ Param.	$\bullet$ Calib.
B	FP	32/32	1.00 $\times$	0.979	–	–	1.00 $\times$	0.979	–	–
B	PTQ4DM	8/8	1.49 $\times$	0.659	0.0175	15	1.47 $\times$	0.666	0.0191	15
B	HyGenQ	8/8	1.47 $\times$	0.667	0.0175	22	1.44 $\times$	0.678	0.0176	22
L	FP	32/32	1.00 $\times$	1.618	–	–	1.00 $\times$	1.618	–	–
L	PTQ4DM	8/8	1.73 $\times$	0.936	0.0224	20	1.72 $\times$	0.939	0.0239	20
L	HyGenQ	8/8	1.77 $\times$	0.915	0.0224	22	1.70 $\times$	0.953	0.0224	23

TABLE VI: End-to-end inference efficiency on MAR models with batch size 32. Spd. denotes latency speedup over full precision, T./Img. denotes the average generation time per image, Param. denotes the storage cost (MB) of the quantization parameters for a single autoregressive iteration, and Calib. denotes the offline calibration time (min).

Method	Model	Precision (W/A)	FID $\downarrow$	IS $\uparrow$
$\mathbf{\circ}$ SmoothQuant [ 24 ]	MAR-B	8/6	156.01	6.77
$\mathbf{\circ}$ PTQ4DM [ 25 ]	MAR-B	8/6	299.86	1.92
$\mathbf{\circ}$ HyGenQ (Ours)	MAR-B	8/6	144.12	8.07
$\mathbf{\circ}$ SmoothQuant [ 24 ]	MAR-L	8/6	352.84	1.72
$\mathbf{\circ}$ PTQ4DM [ 25 ]	MAR-L	8/6	287.09	1.75
$\mathbf{\circ}$ HyGenQ (Ours)	MAR-L	8/6	251.34	2.01

TABLE VII: W8A6 comparison on MAR with 8,000 generated images under the

\mathbf{\circ}

setting.

Method	Calib.	Samples	Seed	FID $\downarrow$	IS $\uparrow$
HyGenQ	Global Max	32	1	8.29	169.34
HyGenQ	PTQ4DM-style [ 25 ]	32	1	8.51	171.24
HyGenQ	PQD-style [ 74 ]	32	1	7.93	173.63
HyGenQ	Per-step	32	1	6.06	193.94
HyGenQ	Per-step	128	1	6.36	191.32
HyGenQ	Per-step	64	1	6.23	193.07

TABLE VIII: Analysis of calibration protocols on MAR-L. The upper block compares global calibration strategies with the adopted per-step setting, while the lower block evaluates the robustness of per-step calibration.

Method	Model	Clustering strategy	FID $\downarrow$	IS $\uparrow$
HyGenQ	MAR-B	$k$ -means++	8.17	151.45
HyGenQ	MAR-B	threshold ( $P_{99}$ )	6.96	175.72
HyGenQ	MAR-B	genetic	7.14	178.74
HyGenQ	MAR-L	$k$ -means++	6.26	187.84
HyGenQ	MAR-L	threshold ( $P_{99}$ )	6.11	188.66
HyGenQ	MAR-L	genetic	6.06	193.94

TABLE IX: Comparison of HCD grouping strategies under the 8,000-image setting on ImageNet

256\times 256

.

Method	Range(k)	FID $\downarrow$	IS $\uparrow$
FP	-	5.1166	207.2319
PTQ4DM	-	9.2551	132.0545
HyGenQ	(2,10)	6.0746	192.5079
HyGenQ	(2,15)	6.0746	192.5079
HyGenQ (Ours)	(2,20)	6.0596	193.9367
HyGenQ	(2,30)	6.0746	192.5079

TABLE X: Sensitivity of cluster number ranges in HCD.

Method	$\omega$	FID $\downarrow$	IS $\uparrow$
FP	-	5.1166	207.2319
PTQ4DM	-	9.2551	132.0545
HyGenQ	0.0001	6.0976	194.4706
HyGenQ (Ours)	0.001	6.0596	193.9367
HyGenQ	0.005	6.0746	192.5079
HyGenQ	0.01	6.0746	192.5079

TABLE XI: Sensitivity of

\omega

in HCD.

Method	$N_{pop}$	T	$P_{c}$	$P_{m}$	FID $\downarrow$	IS $\uparrow$
FP	—				5.1166	207.2319
PTQ4DM	—				9.2551	132.0545
HyGenQ	50	20	0.5	0.2	6.0596	193.9367
HyGenQ	20	20	0.5	0.2	6.4661	192.1524
HyGenQ	60	20	0.5	0.2	6.4661	192.1524
HyGenQ	50	10	0.5	0.2	6.6079	182.7686

TABLE XII: Robustness to GA hyperparameters in HCD.

Metric	Scope	Min	Max	Mean
${\mathrm{IoU}}(S_{a},S_{b})$	Random seeds	1.000	1.000	1.000
${\mathrm{Ret}}(S_{\mathrm{core}},S_{i})$	Input classes	1.000	1.000	1.000

TABLE XIII: Stability of HCD-detected outlier channels on MAR-B. Here,

S_{a},S_{b}

are detected sets under different seeds;

S_{i}

is the detected set under the

i

-th input-class split; and

S_{\mathrm{core}}

is the core anomalous-channel set.

Method	G	FID $\downarrow$	IS $\uparrow$
FP	-	5.1166	207.2319
PTQ4DM	-	9.2551	132.0545
HyGenQ	3	6.6878	183.5942
HyGenQ	4	6.6113	182.9316
HyGenQ	5	6.6691	183.7552
HyGenQ (Ours)	5.545	6.6079	182.7686

TABLE XIV: Sensitivity of Gaussian Bound (G) in SR.

Post-training Quantization for Hybrid Iterative Generative Models

Abstract

Figures & tables

Post-Training Quantization

HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models

PhaseQuant: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

Abstract

Post-Training Quantization

Explore similar work

HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models

PhaseQuant: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

G2^22PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation