A Unified Hessian-Guided Framework for Post-Training Quantization

Mads Toftrup; Marco Federici; Boris van Breugel; Paul Whatmough; Markus Nagel

cs.LGOct 29, 2026

A Unified Hessian-Guided Framework for Post-Training Quantization

Authors: Mads Toftrup, Marco Federici, Boris van Breugel, Paul Whatmough, Markus Nagel

Organizations: Aarhus University · Qualcomm AI Research

Abstract

Low-bit quantization enables efficient deployment of large language models by reducing memory footprint and computational cost. Achieving high accuracy at low precision requires selecting quantization ranges, linear transformations, and weight rounding decisions. Existing approaches typically optimize these components independently using local reconstruction objectives, which fail to account for the varying impact of quantization errors on downstream predictions. Direct optimization of task-level metrics would provide a more faithful objective, but is challenging due to the discontinuous nature of quantization operations and the high computational cost of evaluating their downstream effects. In this work, we derive a Hessian-guided approximation of the output change induced by joint weight and activation quantization. Based on a tractable Fisher approximation, we formulate a unified post-training quantization framework that enables the joint optimization of quantization ranges, linear transformations, and weight rounding under a common output-sensitivity objective. The resulting method remains practical for modern large language models and consistently outperforms state-of-the-art transformation and rounding methods across both channel-wise and block-wise quantization settings.

Figures & tables

Figure 1: Relationship between the KL divergence between the original and quantized model outputs and several layer-local error proxies across linear layers of Llama 3.2 1B Instruct under W4A4 quantization. Conventional reconstruction metrics, such as the output Frobenius norm, exhibit weak correlation with downstream output degradation. Fisher-weighted weight error fails to capture the effect of activation quantization. Output Fisher-weighted error correlates more strongly with the KL divergence, and our proposed Kronecker factorization of the Output Fisher retains this correlation.

Estimate	$\overbrace{{\bm{F}}_{s({\bm{Z}})}\otimes{\bm{F}}_{f({\bm{Z}})}}^{\text{Output Fisher: }{\bm{F}}_{\otimes{\bm{Z}}}}$		$\overbrace{{\bm{F}}_{i({\bm{W}})}\otimes{\bm{F}}_{o({\bm{W}})}}^{\text{Weight Fisher: }{\bm{F}}_{\otimes{\bm{W}}}}$		$\overbrace{{\bm{F}}_{s({\bm{X}})}\otimes{\bm{F}}_{f({\bm{X}})}}^{\text{Input Fisher: }{\bm{F}}_{\otimes{\bm{X}}}}$		Assumptions
Isotropic	${\bm{I}}_{s}$ $\;\otimes\;$	${\bm{I}}_{o}$	${\bm{X}}^{T}{\bm{X}}$ $\;\otimes\;$	${\bm{I}}_{o}$	${\bm{I}}_{s}$ $\;\otimes\;$	${\bm{W}}{\bm{W}}^{T}$	Isotropic ${\bm{F}}_{\bm{Z}}$
K-FAC	${\bm{I}}_{s}$ $\;\otimes\;$	${\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}$	${\bm{X}}^{T}{\bm{X}}$ $\;\otimes\;$	${\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}$	${\bm{I}}_{s}$ $\;\otimes\;$	${\bm{G}}_{{\bm{X}}}^{T}{\bm{G}}_{{\bm{X}}}$	Isotropic ${\bm{F}}_{s({\bm{Z}})}$
YAQA	${\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}$ $\;\otimes\;$	${\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}$	${\bm{G}}_{{\bm{W}}}{\bm{G}}_{{\bm{W}}}^{T}$ $\;\otimes\;$	${\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}$	${\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}$ $\;\otimes\;$	${\bm{W}}{\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}{\bm{W}}^{T}$	Rank-1 ${\bm{G}}_{\bm{W}}$
Ours	${\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}$ $\;\otimes\;$	${\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}$	${\bm{G}}_{{\bm{W}}}{\bm{G}}_{{\bm{W}}}^{T}$ $\;\otimes\;$	${\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}$	${\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}$ $\;\otimes\;$	${\bm{G}}_{{\bm{X}}}^{T}{\bm{G}}_{{\bm{X}}}$	Rank-1 ${\bm{G}}_{\bm{Z}}$

Table 1: Comparison of Fisher Information Metric estimates for linear layer output, weights and input implied by different approaches in literature. Normalization factors are omitted for brevity.

	Llama v3.2 1B inst.		Llama v3 8B		Ministral 8B inst.		Qwen v3 8B
Method	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )
FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
RTN	878.89	30.02	317.60	32.40	39.73	40.62	10441.69	29.23
QuaRot ${}^{+\text{GPTQ}}$	${20.03}^{\pm 0.33}$	${48.93}^{\pm 0.56}$	${8.34}^{\pm 0.04}$	${63.39}^{\pm 0.15}$	${8.18}^{\pm 0.00}$	${67.77}^{\pm 0.09}$	${34.43}^{\pm 9.39}$	${51.64}^{\pm 3.05}$
CAT ${}^{+\text{GPTQ}}$	${17.68}^{\pm 0.09}$	${52.19}^{\pm 0.46}$	${7.53}^{\pm 0.02}$	${65.38}^{\pm 0.13}$	${7.90}^{\pm 0.02}$	${68.06}^{\pm 0.21}$	${10.67}^{\pm 0.02}$	${64.60}^{\pm 0.34}$
FlatQuant ${}^{+\text{GPTQ}}$	${17.23}^{\pm 0.11}$	${52.95}^{\pm 0.40}$	${7.19}^{\pm 0.01}$	$\bf{67.32}^{\pm 0.55}$	${7.82}^{\pm 0.01}$	${68.88}^{\pm 0.33}$	${10.60}^{\pm 0.03}$	${65.26}^{\pm 0.11}$

Table 2: W4A4KV4 channel-wise quantization results. The effectiveness of

{\bm{H}}_{\bm{Z}}

-PTQ demonstrates the advantage of the joint optimization under a unified Hessian-guided objective.

		Llama v3.2 1B inst.		Llama v3 8B		Ministral 8B inst.		Qwen v3 8B
	Method	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )
	FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
W4-g32	RTN	$15.47$	$55.27$	$6.65$	$68.67$	$7.28$	$70.88$	$9.99$	$65.95$
	Had. ${}^{+\text{GPTQ}}$	${13.73}^{\pm 0.02}$	${56.83}^{\pm 0.18}$	${6.43}^{\pm 0.01}$	${68.97}^{\pm 0.19}$	${7.13}^{\pm 0.01}$	${71.01}^{\pm 0.12}$	${9.99}^{\pm 0.05}$	${66.80}^{\pm 0.15}$
	CAT ${}^{+\text{GPTQ}}$	${13.77}^{\pm 0.08}$	${56.77}^{\pm 0.17}$	${6.44}^{\pm 0.01}$	${69.64}^{\pm 0.18}$	${7.14}^{\pm 0.00}$	$\bf{70.99}^{\pm 0.03}$	${9.92}^{\pm 0.04}$	${67.14}^{\pm 0.28}$
	WUSH ${}^{+\text{GPTQ}}$	${13.85}^{\pm 0.08}$	${56.81}^{\pm 0.14}$	${6.44}^{\pm 0.01}$	${69.43}^{\pm 0.33}$	${7.13}^{\pm 0.01}$	${70.83}^{\pm 0.05}$	${9.88}^{\pm 0.08}$	${67.06}^{\pm 0.32}$

Table 3: For block-wise quantization,

{\bm{H}}_{\bm{Z}}

-PTQ consistently achieves the lowest perplexity across both W4-g32 and W4A4-g32 settings, demonstrating that the proposed Hessian-guided objective remains effective even at finer quantization granularities.

	Llama v3.2 1B inst.		Llama v3 8B		Ministral 8B inst.		Qwen v3 8B
Method	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )
${\bm{H}}_{\bm{Z}}$ -PTQ	${16.05}^{\pm 0.02}$	${54.11}^{\pm 0.19}$	${7.06}^{\pm 0.01}$	${67.25}^{\pm 0.23}$	${7.69}^{\pm 0.01}$	${69.30}^{\pm 0.26}$	${10.43}^{\pm 0.03}$	${65.56}^{\pm 0.46}$
$-$ Round	${16.52}^{\pm 0.07}$	${53.13}^{\pm 0.57}$	${7.13}^{\pm 0.00}$	${66.54}^{\pm 0.04}$	${7.74}^{\pm 0.02}$	${69.10}^{\pm 0.15}$	${10.56}^{\pm 0.04}$	${65.14}^{\pm 0.42}$
$-$ A.clip	${17.05}^{\pm 0.03}$	${52.35}^{\pm 0.36}$	${7.41}^{\pm 0.01}$	${65.72}^{\pm 0.14}$	${7.89}^{\pm 0.01}$	${68.35}^{\pm 0.62}$	${10.63}^{\pm 0.05}$	${64.08}^{\pm 0.05}$
$-$ Tran.	${18.64}^{\pm 0.13}$	${51.68}^{\pm 0.86}$	${7.61}^{\pm 0.01}$	${66.06}^{\pm 0.39}$	${8.13}^{\pm 0.01}$	${67.98}^{\pm 0.30}$	${25.19}^{\pm 0.90}$	${55.80}^{\pm 0.48}$
$-$ Hess.	${16.83}^{\pm 0.10}$	${52.80}^{\pm 0.64}$	${7.37}^{\pm 0.02}$	${65.75}^{\pm 0.11}$	${7.83}^{\pm 0.03}$	${68.90}^{\pm 0.34}$	${10.42}^{\pm 0.04}$	${65.77}^{\pm 0.03}$

Table 4: Ablation for the components for

{\bm{H}}_{\bm{Z}}

-PTQ on W4A4KV4.

-

Round refers to a version with round-to-nearest and a standard

L_{p}

weight range-setting.

-

Trans. refers to the use of Hadamard transform instead of our gradient alignment transforms.

-

Hess. indicates the use of a GPTQ-style hessian approximation instead of our output Kronecker approximation.

Figure 2: WikiText perplexity obtained by several weight scale-setting objectives on Llama 3.2 1B inst. under W4 per-channel quantization, with and without Hadamard transformations. Weighting the quantization error using the proposed Fisher estimate consistently exceeds the best reconstruction objective without requiring hyperparameter tuning.

		W4 Per-channel			W4 Per-block (32)
Tran.	Hessian Estimate	GPTQ	YAQA	ours	GPTQ	YAQA	ours
Id.	${\bf X}^{T}{\bf X}\otimes{\bf I}$	${30.96}^{\pm 4.58}$	${30.56}^{\pm 0.00}$	${17.65}^{\pm 0.09}$	${14.35}^{\pm 0.04}$	${69.59}^{\pm 10.14}$	${14.11}^{\pm 0.07}$
	${\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{W}^{T}{\bf G}_{W}$	${18.42}^{\pm 0.48}$	${18.81}^{\pm 0.10}$	${15.34}^{\pm 0.03}$	${14.28}^{\pm 0.11}$	${13.75}^{\pm 0.05}$	${\bf 13.64}^{\pm 0.05}$
	${\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{Z}^{T}{\bf G}_{Z}$	${19.66}^{\pm 3.89}$	${19.49}^{\pm 0.42}$	${\bf 15.23}^{\pm 0.06}$	${14.26}^{\pm 0.25}$	${13.92}^{\pm 0.08}$	${\bf 13.64}^{\pm 0.02}$
Had.	${\bf X}^{T}{\bf X}\otimes{\bf I}$	${14.96}^{\pm 0.08}$	${20.68}^{\pm 0.00}$	${14.50}^{\pm 0.01}$	${13.68}^{\pm 0.03}$	${13.90}^{\pm 0.07}$	${13.69}^{\pm 0.05}$
	${\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{W}^{T}{\bf G}_{W}$	${14.50}^{\pm 0.06}$	${19.12}^{\pm 0.29}$	${14.26}^{\pm 0.05}$	${13.61}^{\pm 0.01}$	${13.70}^{\pm 0.05}$	${\bf 13.56}^{\pm 0.02}$

Table 5: Comparison of rounding algorithms for different hessian estimates on the Llama v3.2 1B instruct architecture in two quantization settings. The bottom rows report the effect of applying a Hadamard transform prior to quantization.

{\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{Z}^{T}{\bf G}_{Z}

refers to our approximation.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Method	W. Scale	W. Round	Act. Clip	Transforms
RTN	$2\max_{i}\|{\bm{w}}_{i}\|$	None	$\gamma=1$	None
QuaRot +GPTQ	$\min_{s_{w}}\\|{\bm{w}}-Q({\bm{w}};s_{w})\\|^{2.4}$	GPTQ	$\gamma=1$	(Fused) ${\bm{H}}$
CAT +GPTQ	$\min_{s_{w}}\\|{\bm{w}}-Q({\bm{w}};s_{w})\\|^{2.4}$	GPTQ	$\gamma=1$	$B(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{H}}$
FlatQuant +GPTQ	Learned $s_{w}$	GPTQ	Learned $\gamma$	Learned ${\bm{T}}_{i}\otimes{\bm{T}}_{o}$
${\bm{H}}_{\bm{Z}}$ -PTQ	Algorithm in Sec. 3.5		Sec. 3.6	$B(\bm{\Sigma}_{{\bm{G}}_{\bm{W}}}^{-1}\#\bm{\Sigma}_{{\bm{G}}_{\bm{X}}})^{1/2}{\bm{H}}$

Appendix

Table 6: Details on the setting used to produce Table 2 . The block size for the transforms used for CAT and our method is 128.

Method	Weight Scale	Weight Round	Act. Clip	Block Transforms
RTN	$2\max_{i}\|{\bm{w}}_{i}\|$	None	$\gamma=1$	None
Had. +GPTQ	$2\max_{i}\|{\bm{w}}_{i}\|$	GPTQ	$\gamma=1$	Hadamard
Had. +YAQA	$2\max_{i}\|{\bm{w}}_{i}\|$	YAQA	$\gamma=1$	Hadamard
CAT +GPTQ	$2\max_{i}\|{\bm{w}}_{i}\|$	GPTQ	$\gamma=1$	$(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{H}}$
WUSH +GPTQ	$2\max_{i}\|{\bm{w}}_{i}\|$	GPTQ	$\gamma=1$	$(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{U}}{\bm{H}}$
${\bm{H}}_{\bm{Z}}$ -PTQ	Algorithm in Sec. 3.5		$\gamma=1$	$(\bm{\Sigma}_{{\bm{G}}_{\bm{W}}}^{-1}\#\bm{\Sigma}_{{\bm{G}}_{\bm{X}}})^{1/2}{\bm{H}}$

Appendix

Table 7: Details on the setting used to produce Table 3 . The transformation in this table already refer to a single block of size 32.

Figure 3: Wikitext perplexity obtained by different activation clipping objectives on Llama 3.2 1B Instruct under 4-bit per token quantization, with and without Hadamard transformations.

	Llama v3.2 1B inst.		Llama v3 8B		Ministral 8B inst.		Qwen v3 8B
Method	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )	wiki ( $\downarrow$ )	csr ( $\uparrow$ )
FP	13.16	57.61	6.14	70.12	6.97	71.29	9.73	67.46
QuaRot	29.77	46.23	10.99	57.07	9.14	64.55	13.68	57.33
CAT	${18.20}^{\pm 0.10}$	${51.41}^{\pm 0.28}$	${7.67}^{\pm 0.01}$	${63.90}^{\pm 0.24}$	${7.99}^{\pm 0.01}$	${67.72}^{\pm 0.20}$	${11.01}^{\pm 0.04}$	${63.66}^{\pm 0.05}$
FlatQuant	${17.09}^{\pm 0.04}$	${52.60}^{\pm 0.42}$	${7.50}^{\pm 0.52}$	${66.28}^{\pm 2.04}$	${7.84}^{\pm 0.05}$	${68.71}^{\pm 0.14}$	${10.55}^{\pm 0.13}$	${65.34}^{\pm 0.32}$
${\bm{H}}_{\bm{Z}}$ -PTQ -a.clip	${17.78}^{\pm 0.12}$	${51.42}^{\pm 0.16}$	${7.51}^{\pm 0.01}$	${64.55}^{\pm 0.56}$	${7.95}^{\pm 0.02}$	${68.23}^{\pm 0.25}$	${10.78}^{\pm 0.04}$	${63.21}^{\pm 0.00}$

Appendix

Table 8: A4W4KV4 Quantization results without any weight rounding optimization and standard

L_{p}

-based ranges.

Quantization

See all 54 papers

May 11, 2026

Theory-optimal Quantization Based on Flatness

Post-training quantization has emerged as a widely adopted technique for compressing and accelerating the inference of Large Language Models (LLMs). The primary challenges in LLMs quantization stem from activation outliers, which significantly degrade model performance especially at lower bit precision. While recent approaches attempt to mitigate outliers through linear transformations across feature dimensions, our analysis reveals that the transformed weights and activations still exhibit persistent outlier patterns with concentrated magnitude distributions. In this paper, we first model the mathematical relationship between quantization error and outliers, and then introduce a new metric Flatness to quantify the distribution of outliers. Based on this, we derive the theoretical optimal solution with respect to Flatness. Building on these insights, we propose Bidirectional Diagonal Quantization (BDQ), a novel post-training quantization framework that effectively disperses outlier patterns through optimized matrix transformations. BDQ strategically distributes outlier magnitudes across matrix dimensions via learned diagonal operations. Extensive experiments demonstrate that BDQ establishes a new quantization benchmark. It achieves less than 1% accuracy drop in W4A4 quantization on the LLaMA-3-8B model. In the more challenging W2A4KV16 experiment, compared to state-of-the-art approaches, BDQ reduces the performance gap by 39.1% on the DeepSeek-R1-Distill-LLaMA-70B model.

Xiusheng Huang, Zhe Li, Xuanwu Yin +5

The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2

LLM Quantization Activation Quantization Post-Training Quantization Low-Bit Quantization

Ask PDF

Sep 10, 2026

Why Does Post-Training Quantization Work?

Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization work? Comparing full-precision and quantized forward passes, we identify two mechanisms that characterize pretrained quantization robustness. First, the error a layer newly introduces tends to oppose the error it inherits from the layer's input. The two cancel partially such that the discrepancy between full-precision and quantized passes grows slowly. This counteracting residual interaction develops during pretraining. Our quantitative analysis identifies it as a major factor slowing hidden-error growth. Second, LM-head geometry preferentially preserves the scores and probabilities of high-ranked tokens, which typically represent the model's most confident predictions. Together, these mechanisms explain why quantization error that passes through numerous layers can still produce only small output changes, and we verify the findings across models and quantization settings.

Yuxiang Chen, Michael Beyer, Jun Zhu +1

Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University · College of AI, Tsinghua University · Bosch AI Research, Renningen, Germany

LLM Quantization Quantization Robustness Post-Training Quantization Neural Network Quantization

Ask PDF

Apr 15, 2026

Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

Large Language Models (LLMs) are widely used across many domains, but their scale makes deployment challenging. Post-Training Quantization (PTQ) reduces memory footprint without retraining by leveraging a small calibration set. Recent Hessian-based PTQ methods compensate quantization error via cross-channel dependencies, but such approaches degrade at low bit-widths due to noisy curvature estimates from limited calibration data. We propose DASH-Q, a robust PTQ framework using diagonal Hessian approximation and iterative weighted least squares. By discarding noise-prone dependencies, DASH-Q filters sampling noise while prioritizing the preservation of salient feature power. We outperform other PTQ baselines in ultra low-bit regime, improving zero-shot accuracy by 7.01% on average and up to 14.01% over the strongest baselines across five baseline LLM models, while showing robust and stable performance with very small calibration data.

Jaemin Kim, Sungkyun Kim, Junyeol Lee +1

Seoul National University · Hanyang University

LLM Quantization Language Model Calibration Post-Training Quantization Low-Bit Quantization Neural Network Quantization

Ask PDF

Abstract

Quantization

Explore similar work

Theory-optimal Quantization Based on Flatness

Why Does Post-Training Quantization Work?

Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate