Activation Sensitivity as a Unifying Principle for Post-Training Quantization

Bruce Changlong Xu

cs.LGJan 15, 2026

Activation Sensitivity as a Unifying Principle for Post-Training Quantization

Authors: Bruce Changlong Xu

Abstract

Post-training quantization (PTQ) methods for large language models compress pretrained weights to low bit-widths without retraining by optimizing layer-local proxy objectives. Activation-aware scaling methods such as AWQ protect input channels with large activation norms, whereas second-order error-compensation methods such as GPTQ redistribute rounding error using the inverse of the unweighted input covariance matrix. We show that both paradigms -- along with diagonal Fisher quantization, block reconstruction, and Hadamard rotation methods -- are special cases of minimizing a single loss-derived quadratic form governed by gradient-weighted activation sensitivity. From a Gauss-Newton expansion of the task loss at a linear layer $Y = X W^\top$ , we prove that under isotropic output curvature the expected excess loss from weight perturbation $ΔW$ reduces to $\frac{1}{2 d_{\mathrm{out}}}\operatorname{Tr}(ΔW H_D ΔW^\top)$ , where $H_D = \frac{1}{n} X^\top D X$ weights each calibration token's outer product by its squared output-gradient norm $D_{tt} = \|\nabla_{y_t}\mathcal{L}\|_2^2$ . Within this framework, AWQ corresponds to a diagonal approximation of $H_D$ with uniform token weights ( $D = I_n$ ) optimized via channel scaling, while GPTQ corresponds to Cholesky error compensation on $H_D$ under $D = I_n$ . We derive closed-form optimal scaling exponents for diagonal sensitivity protection, formulate sensitivity-weighted Cholesky quantization (SAQ-GPTQ) on $H_D$ , and prove bounds on token-gradient heterogeneity bias, cross-layer error amplification across residual blocks, and finite-sample calibration concentration. Controlled numerical simulations confirm that under token-gradient heterogeneity, replacing $X^\top X$ with $X^\top D X$ preserves $>99\%$ top-channel saliency recall and reduces INT3/INT4 excess loss by roughly $3\times$ over unweighted GPTQ.

Figures & tables

Panel of Figure 1 — Figure 1: Hierarchy of post-training quantization proxy objectives derived from the Kronecker-factored Gauss–Newton Hessian $\mathcal{H}_{W}$ (Theorem 2.1 ). Each arrow labels the exact simplifying assumption connecting a parent sensitivity metric to a child proxy.

Method Family	Sensitivity Metric	Token $D_{tt}$	Off-Diag	Mechanism
RTN (Baseline)	$I_{d_{\mathrm{in}}}$ (Uniform)	None ( $0$ )	Ignored	Per-group rounding
AWQ [ 3 ]	$\operatorname{diag}(\frac{1}{n}X^{\top}X)$	Uniform ( $1$ )	Ignored	Scaling $s_{j}=(\alpha_{j}^{\mathrm{act}})^{\gamma}$
SmoothQuant [ 8 ]	$\max_{t}\|X_{t,j}\|^{\gamma}\max_{i}\|W_{ij}\|^{1-\gamma}$	Uniform ( $1$ )	Ignored	W/X scale migration
OBC / GPTQ [ 14 , 2 ]	$\frac{1}{n}X^{\top}X+\lambda I$	Uniform ( $1$ )	Cholesky	Sequential OBS update
QuIP / QuIP# [ 16 , 17 ]	$U^{\top}(\frac{1}{n}X^{\top}X)U$	Uniform ( $1$ )	Cholesky	Rotation + lattice/OBS
BRECQ [ 24 , 26 ]	$J_{\mathrm{blk}}^{\top}J_{\mathrm{blk}}$	Block-local	Intra-block	Block reconstruction

Table 1: Unification of representative post-training quantization paradigms within the activation sensitivity framework of Theorem 2.1 and Proposition 3.1 .

Panel of Figure 2 — Figure 2: Channel saliency prediction accuracy as a function of token-gradient heterogeneity $\sigma_{g}\in[0,2.0]$ . Left: Spearman rank correlation $\rho_{S}$ with true channel excess loss $\Delta\mathcal{L}_{j}$ . Right: Recall of the top 10% most loss-sensitive channels. At $\sigma_{g}=0$ (uniform token gradients), AWQ and SAQ coincide; as $\sigma_{g}$ increases, unweighted activation magnitude suffers severe rank bias (Proposition 4.1 ), whereas gradient-weighted activation sensitivity $(H_{D})_{jj}$ maintains $\rho_{S}\geq 0.9996$ and recall $\geq 99.4\%$ .

		INT4 ( $b=4$ )		INT3 ( $b=3$ )
Method	Sensitivity Proxy	Excess Loss $\downarrow$	Recon. MSE	Excess Loss $\downarrow$	Recon. MSE
RTN	None ( $I_{d_{\mathrm{in}}}$ )	$1.0000$	$1.0000$	$1.0000$	$1.0000$
AWQ [ 3 ]	$\operatorname{diag}(\frac{1}{n}X^{\top}X)$	$0.8114$	$0.8109$	$0.8192$	$0.7780$
SAQ-Scale (Ours)	$\operatorname{diag}(\frac{1}{n}X^{\top}DX)$	$0.6650$	$1.2728$	$0.6610$	$1.2268$
GPTQ [ 2 ]	$\frac{1}{n}X^{\top}X+\lambda I$	$0.1714$	$\mathbf{0.1698}$	$0.1943$	$\mathbf{0.1971}$
SAQ-GPTQ (Ours)	$\frac{1}{n}X^{\top}DX+\lambda I$	$\mathbf{0.0535}$	$0.7120$	$\mathbf{0.0694}$	$0.6877$

Table 2: Relative excess loss

\operatorname{Tr}(\Delta WH_{D}\Delta W^{\top})

and unweighted layer output reconstruction MSE

\frac{1}{n}\|X\Delta W^{\top}\|_{F}^{2}

(both normalized to RTN

=1.0000

, averaged over

20

trials) for INT4 and INT3 quantization.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 381 papers

Sep 29, 2026

Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape

Post-training quantization (PTQ) methods in the GPTQ family minimize a layer-wise reconstruction error on a uniform grid whose scale must be chosen; the common max-based choice degrades sharply at low bit-widths. We study how sensitive this objective is to the scale. For a layer with i.i.d. Gaussian weights and calibration activations of sufficiently large effective rank, we prove that, as the width grows, the normalized round-to-nearest loss converges with high probability, uniformly over all scales, to the mean-squared error of a uniform quantizer applied to a standard Gaussian; we verify the effective-rank condition for wide, randomly initialized MLPs with odd Lipschitz activations and isotropic Gaussian calibration data. The limiting objective has a unique nondegenerate minimizer, whose scale decreases strictly with the number of levels and whose curvature with respect to relative scale errors decays approximately exponentially with the bit-width. GPTQ experiments on five LLMs show the same trend: the scale rule changes perplexity substantially at 2--3 bits and negligibly from 6 bits on, and a local measure of GPTQ scale sensitivity decreases with bit-width in line with the Gaussian curvature. The Gaussian-optimal scale fails on raw weights; after Hadamard incoherence processing it matches the best searched rule at 3 bits and above without any search, but remains clearly worse at 2 bits.

Jonas von Berg, Massimiliano Datres, Carlo Kneißl +1

Ludwig-Maximilians-Universität München Munich Center for Machine Learning (MCML) Konrad Zuse School of Excellence in Reliable AI (DAAD) · University of Tromso, DLR-German Aerospace Center

Post-Training Quantization Neural Network Quantization LLM Quantization Quantization Sensitivity Analysis Low-Bit Quantization

Ask PDF

Jul 23, 2026

C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

Multimodal large language models (MLLMs) require huge memory and computational costs, which limits their practical deployment. Post-training quantization (PTQ) techniques offer an efficient solution for model compression and inference acceleration. Yet, the quantized model faces performance degradation due to outlier channels, which are highly sensitive to quantization and substantially impair activation fidelity and task accuracy. To protect these salient channels during quantization, existing PTQ methods leverage modality- or token-level metrics to guide channel-wise scaling (CWS) of LLM decoders. However, these orthogonal measurements fail to capture channel-wise impacts on task-specific loss, and the misalignment between importance and scaling factors ultimately leads to suboptimal performance. To address this issue, we propose C-PTQ, a unified channel-wise PTQ method that harmonizes task-specific loss perturbation and quantization error. Motivated by second-order derivatives, we design a Fisher-weighted objective as a tractable Hessian approximation, seamlessly injecting task sensitivity into the scaling process. Notably, we achieve state-of-the-art performance without auxiliary modules like LoRA, thereby maintaining high efficiency. Experiments on Qwen2.5VL, InternVL2 and LLaVA-OV across 8 benchmarks demonstrate our effectiveness in both weight-only and weight-activation settings.

Jiameng Li, Han Zhou, Matthew B. Blaschko

1KU Leuven · 2Tiangong University

VLM Quantization Post-Training Quantization Neural Network Quantization

Ask PDF

Jun 9, 2026

Optimal Post-Training Quantization Scales and Where to Find Them

Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations. While much work has focused on how weights are rounded onto the quantization grid, the scaling factor that defines the grid is typically chosen with simple, data-free heuristics. We observe that, under round-to-nearest (RTN) quantization, the grid assignments are piecewise constant in the scale, making the output reconstruction error piecewise quadratic over finitely many intervals, each admitting a closed-form optimization. Building on this insight, we present PiSO (Piecewise Scale Optimization), an algorithm that leverages calibration data to efficiently select channel-wise scales to achieve globally optimal output reconstruction error to arbitrary accuracy. We extend PiSO to group-wise quantization via tractable objectives. We also propose strategies for integrating it with error-correction methods such as GPTQ and Qronos, which greedily adjust weight assignments to compensate for the rounding errors of previously quantized weights. Experiments on Llama and Qwen models across multiple sizes and bit-widths show consistent improvements in perplexity and zero-shot accuracy, both standalone and combined with error correction, with gains increasing as the bit-width narrows. For instance, on Llama-3, PiSO reduces the 2-bit perplexity of Qronos by up to 7x, and at 3 bits, PiSO without error correction outperforms GPTQ with absmax scales.

Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2

AMD

LLM Quantization Weight-Only Quantization Post-Training Quantization Low-Bit Quantization Neural Network Quantization

Ask PDF

Abstract

LLM Quantization

Explore similar work

Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape

C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

Optimal Post-Training Quantization Scales and Where to Find Them