MPQ: A Message-Passing View of Post-Training Quantization

Yanshuo Chen; Chenxi Liu; Reza Shirkavand; Heng Huang

cs.AIOct 23, 2026

MPQ: A Message-Passing View of Post-Training Quantization

Authors: Yanshuo Chen, Chenxi Liu, Reza Shirkavand, Heng Huang

Organizations: Department of Computer Science, University of Maryland, College Park, MD 20742

Abstract

Post-training quantization (PTQ) is a key technique for efficiently deploying large language models, as it compresses weights into low-bit representations without retraining. Existing PTQ solvers, including GPTQ, LDLQ, and block coordinate descent, commit each weight block to a single codeword before its neighbors are considered, leaving near-optimal alternatives unexplored. In this work, we cast row-wise PTQ as posterior inference over discrete codebooks, in which existing solvers correspond to a zero-temperature, hard-decision limit retaining only the top codeword per block. We propose Message-Passing Quantization (MPQ), which preserves soft posterior information across blocks during inference and commits to hard codewords only at the output stage. MPQ first uses vector approximate message passing to compute approximate blockwise posteriors, then refines a hard-decision initialization by exact block coordinate descent. Across 2-bit lattice and 3- and 4-bit scalar quantization on dense language models from Llama, Qwen, and Mistral families, MPQ improves perplexity and zero-shot accuracy over strong hard-decision baselines, and composes favorably with preprocessing techniques such as activation-aware scaling and incoherence processing. Our implementation is available at https://github.com/poseidonchan/MPQ.

Figures & tables

Figure 1: Greedy quantization and MPQ. Rows represent weight blocks

b

. Colors identify codewords

c\in\mathcal{C}_{b}

, and bar heights show probabilities

q_{b}^{t}(c)

(Eq. ( 4 )). (a) Greedy solvers fix blocks sequentially and pass error to remaining blocks. (b) MPQ alternates denoising and Gaussian linear updates across all blocks, then applies marginal MAP projection and exact block CD.

Figure 2: Solver-time scaling across Llama model sizes.

Figure 3: Solver-side speed-quality tradeoff on Meta-Llama-3-8B.

Figure 4: Mechanism diagnostics on Meta-Llama-3-8B.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Pilot sweeps over VAMP iterations, damping, anchoring multiplier

\tau

, and calibration budget for Llama-3-8B 3-bit scalar quantization.

Figure 6: MPQ recipe variants on WikiText-2 for the 8B/7B models. a Performance comparison of 3-bit scalar preprocessing and CD choices. b Performance comparison of 2-bit E8P12 VAMP and CD-polished recipes. AWQ preprocessing intentionally manipulates the per-channel activation magnitude which would disrupt the lattice codebook’s block-level isotropy and make quantization fail. Therefore, AWQ is not inlcuded in the 2-bit E8P12 setting for preprocessing ablation study.

Model	Method	WikiText-2	C4
Llama-3.2-1B	GPTQ+CD3	$10.625\pm 0.028$	$13.856\pm 0.017$
	QuantEase	$10.474\pm 0.014$	$13.827\pm 0.033$
	OJBKQ	$10.456\pm 0.023$	$13.672\pm 0.022$
	MPQ	$10.392\pm 0.010$	$13.629\pm 0.017$
Llama-3.2-3B	GPTQ+CD3	$8.264\pm 0.013$	$11.109\pm 0.010$
	QuantEase	$8.202\pm 0.006$	$11.038\pm 0.014$

Appendix

Table 9: Perplexity of 4-bit scalar quantization over three calibration draws (mean

\pm

sample SD).

Model	Method	WikiText-2	C4
Llama-3.2-1B	GPTQ+CD3	$15.352\pm 0.202$	$18.950\pm 0.191$
	QuantEase	$14.909\pm 0.174$	$18.759\pm 0.178$
	OJBKQ	$14.471\pm 0.260$	$18.346\pm 0.395$
	MPQ	$13.776\pm 0.091$	$17.502\pm 0.036$
Llama-3.2-3B	GPTQ+CD3	$10.575\pm 0.042$	$13.767\pm 0.029$
	QuantEase	$10.098\pm 0.135$	$13.467\pm 0.079$

Appendix

Table 10: Perplexity of 3-bit scalar quantization over three calibration draws (mean

\pm

sample SD).

Initialization	CD0	CD1	CD3
LDLQ	359.869170	347.919460	340.203384
OJBKQ ( $K=5$ )	357.7222	348.1638	340.3826
MPQ	371.615070	335.443011	324.342445

Appendix

Table 11: Calibration objective under matched D4 refinement. CD0 is the initial hard assignment.

Method	WikiText-2	C4
LDLQ	20.2967	24.1372
LDLQ + CD3	14.6389	17.6817
OJBKQ ( $K=5$ ) + CD3	15.0872	17.8354
MPQ + CD3	12.9658	15.7847

Appendix

Table 12: Held-out perplexity with the 2-bit D4 codebook.

Method	Time (s)	WikiText-2
LDLQ + CD3	2076	12.756
MPQ ( $T_{\mathrm{vamp}}=8$ ) + CD3	2134	12.370
LDLQ + CD6	3057	12.728
MPQ ( $T_{\mathrm{vamp}}=16$ ) + CD3	2951	11.630

Appendix

Table 13: Solver time and WikiText-2 perplexity for two offline compute budgets.

Initialization	CD sweeps	WikiText-2	C4
LDLQ	3	12.756	16.034
MPQ	3	11.753	14.818
LDLQ	20	12.651	15.943
MPQ	20	11.669	14.728

Appendix

Table 14: Perplexity after extended CD refinement on Llama-3-8B with E8P12.

Quantization

See all 52 papers

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
Sep 25, 2026Ruikang Liu, Haoli Bai, Yuxuan Sun +8LLM Quantization Post-Training Quantization
ZTE Corporation · The Chinese University of Hong Kong · Northwestern Polytechnical University +2
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
Aug 7, 2026Yongge Ma, Guoan Wang, Feiyu Wang +5LLM Quantization Post-Training Quantization
School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University +2
SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models
Apr 11, 2026Han Liu, Haotian Gao, Xiaotong Zhang +5LLM Quantization Post-Training Quantization
Dalian University of Technology, Dalian, China · Peking University, Beijing, China · Shenzhen MSU-BIT University, Shenzhen, China +1

Sep 25, 2026J/K move · Enter open · S save

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

Ruikang Liu, Haoli Bai, Yuxuan Sun, Qian Zhang+7

ZTE Corporation · The Chinese University of Hong Kong · Northwestern Polytechnical University+2

LLM Quantization Post-Training Quantization Low-Bit Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations. Methods with local, layer-wise objectives lack global supervision; while methods with global objectives fix their Hessian estimates at the start and ignore first-order gradients, so their guidance grows stale as quantization proceeds. This paper presents G

^2

PTQ, a unified PTQ framework with Generalized Gradient Compensation that integrates both first- and second-order information under a globally supervised, block-wise optimization objective. By refreshing gradient and Hessian estimates before quantizing each Transformer block, G

^2

PTQ avoids the staleness of prior global methods. Furthermore, to stabilize the exact first-order compensation, we introduce a trust-region scaling mechanism that dynamically bounds the gradient step to prevent exploding weight updates. Finally, we derive efficient implementations for block-wise Hessian approximation and exact gradient compensation. Experimental results on various model families and bit-widths demonstrate that G

^2

PTQ enables better alignment with the full-precision model, outperforming state-of-the-art baselines. Code is available at: https://github.com/G2PTQ/G2PTQ.

Abstract

Quantization

Explore similar work

G2^22PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation