cs.LGOct 29, 2026

A Unified Hessian-Guided Framework for Post-Training Quantization

Authors: Mads Toftrup, Marco Federici, Boris van Breugel, Paul Whatmough, Markus Nagel

Organizations: Aarhus University · Qualcomm AI Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Low-bit quantization enables efficient deployment of large language models by reducing memory footprint and computational cost. Achieving high accuracy at low precision requires selecting quantization ranges, linear transformations, and weight rounding decisions. Existing approaches typically optimize these components independently using local reconstruction objectives, which fail to account for the varying impact of quantization errors on downstream predictions. Direct optimization of task-level metrics would provide a more faithful objective, but is challenging due to the discontinuous nature of quantization operations and the high computational cost of evaluating their downstream effects. In this work, we derive a Hessian-guided approximation of the output change induced by joint weight and activation quantization. Based on a tractable Fisher approximation, we formulate a unified post-training quantization framework that enables the joint optimization of quantization ranges, linear transformations, and weight rounding under a common output-sensitivity objective. The resulting method remains practical for modern large language models and consistently outperforms state-of-the-art transformation and rounding methods across both channel-wise and block-wise quantization settings.

Figures & tables

Figure 1: Relationship between the KL divergence between the original and quantized model outputs and several layer-local error proxies across linear layers of Llama 3.2 1B Instruct under W4A4 quantization. Conventional reconstruction metrics, such as the output Frobenius norm, exhibit weak correlation with downstream output degradation. Fisher-weighted weight error fails to capture the effect of activation quantization. Output Fisher-weighted error correlates more strongly with the KL divergence, and our proposed Kronecker factorization of the Output Fisher retains this correlation.
EstimateFs(Z)⊗Ff(Z)⏞Output Fisher: F⊗Z\overbrace{{\bm{F}}_{s({\bm{Z}})}\otimes{\bm{F}}_{f({\bm{Z}})}}^{\text{Output Fisher: }{\bm{F}}_{\otimes{\bm{Z}}}}Fi(W)⊗Fo(W)⏞Weight Fisher: F⊗W\overbrace{{\bm{F}}_{i({\bm{W}})}\otimes{\bm{F}}_{o({\bm{W}})}}^{\text{Weight Fisher: }{\bm{F}}_{\otimes{\bm{W}}}}Fs(X)⊗Ff(X)⏞Input Fisher: F⊗X\overbrace{{\bm{F}}_{s({\bm{X}})}\otimes{\bm{F}}_{f({\bm{X}})}}^{\text{Input Fisher: }{\bm{F}}_{\otimes{\bm{X}}}}Assumptions
IsotropicIs{\bm{I}}_{s}   ⊗  \;\otimes\;Io{\bm{I}}_{o}XTX{\bm{X}}^{T}{\bm{X}}   ⊗  \;\otimes\;Io{\bm{I}}_{o}Is{\bm{I}}_{s}   ⊗  \;\otimes\;WWT{\bm{W}}{\bm{W}}^{T}Isotropic FZ{\bm{F}}_{\bm{Z}}
K-FACIs{\bm{I}}_{s}   ⊗  \;\otimes\;GZTGZ{\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}XTX{\bm{X}}^{T}{\bm{X}}   ⊗  \;\otimes\;GZTGZ{\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}Is{\bm{I}}_{s}   ⊗  \;\otimes\;GXTGX{\bm{G}}_{{\bm{X}}}^{T}{\bm{G}}_{{\bm{X}}}Isotropic Fs(Z){\bm{F}}_{s({\bm{Z}})}
YAQAGZGZT{\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}   ⊗  \;\otimes\;GWTGW{\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}GWGWT{\bm{G}}_{{\bm{W}}}{\bm{G}}_{{\bm{W}}}^{T}   ⊗  \;\otimes\;GWTGW{\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}GZGZT{\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}   ⊗  \;\otimes\;WGWTGWWT{\bm{W}}{\bm{G}}_{{\bm{W}}}^{T}{\bm{G}}_{{\bm{W}}}{\bm{W}}^{T}Rank-1 GW{\bm{G}}_{\bm{W}}
OursGZGZT{\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}   ⊗  \;\otimes\;GZTGZ{\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}GWGWT{\bm{G}}_{{\bm{W}}}{\bm{G}}_{{\bm{W}}}^{T}   ⊗  \;\otimes\;GZTGZ{\bm{G}}_{{\bm{Z}}}^{T}{\bm{G}}_{{\bm{Z}}}GZGZT{\bm{G}}_{\bm{Z}}{\bm{G}}_{\bm{Z}}^{T}   ⊗  \;\otimes\;GXTGX{\bm{G}}_{{\bm{X}}}^{T}{\bm{G}}_{{\bm{X}}}Rank-1 GZ{\bm{G}}_{\bm{Z}}
Table 1: Comparison of Fisher Information Metric estimates for linear layer output, weights and input implied by different approaches in literature. Normalization factors are omitted for brevity.
 Llama v3.2 1B inst.Llama v3 8BMinistral 8B inst.Qwen v3 8B
Methodwiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )
FP13.1657.616.1470.126.9771.299.7367.46
RTN878.8930.02317.6032.4039.7340.6210441.6929.23
QuaRot +GPTQ{}^{+\text{GPTQ}}20.03±0.33{20.03}^{\pm 0.33}48.93±0.56{48.93}^{\pm 0.56}8.34±0.04{8.34}^{\pm 0.04}63.39±0.15{63.39}^{\pm 0.15}8.18±0.00{8.18}^{\pm 0.00}67.77±0.09{67.77}^{\pm 0.09}34.43±9.39{34.43}^{\pm 9.39}51.64±3.05{51.64}^{\pm 3.05}
CAT +GPTQ{}^{+\text{GPTQ}}17.68±0.09{17.68}^{\pm 0.09}52.19±0.46{52.19}^{\pm 0.46}7.53±0.02{7.53}^{\pm 0.02}65.38±0.13{65.38}^{\pm 0.13}7.90±0.02{7.90}^{\pm 0.02}68.06±0.21{68.06}^{\pm 0.21}10.67±0.02{10.67}^{\pm 0.02}64.60±0.34{64.60}^{\pm 0.34}
FlatQuant +GPTQ{}^{+\text{GPTQ}}17.23±0.11{17.23}^{\pm 0.11}52.95±0.40{52.95}^{\pm 0.40}7.19±0.01{7.19}^{\pm 0.01}67.32±0.55\bf{67.32}^{\pm 0.55}7.82±0.01{7.82}^{\pm 0.01}68.88±0.33{68.88}^{\pm 0.33}10.60±0.03{10.60}^{\pm 0.03}65.26±0.11{65.26}^{\pm 0.11}
Table 2: W4A4KV4 channel-wise quantization results. The effectiveness of HZ{\bm{H}}_{\bm{Z}} -PTQ demonstrates the advantage of the joint optimization under a unified Hessian-guided objective.
  Llama v3.2 1B inst.Llama v3 8BMinistral 8B inst.Qwen v3 8B
 Methodwiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )
 FP13.1657.616.1470.126.9771.299.7367.46
W4-g32RTN15.4715.4755.2755.276.656.6568.6768.677.287.2870.8870.889.999.9965.9565.95
Had. +GPTQ{}^{+\text{GPTQ}}13.73±0.02{13.73}^{\pm 0.02}56.83±0.18{56.83}^{\pm 0.18}6.43±0.01{6.43}^{\pm 0.01}68.97±0.19{68.97}^{\pm 0.19}7.13±0.01{7.13}^{\pm 0.01}71.01±0.12{71.01}^{\pm 0.12}9.99±0.05{9.99}^{\pm 0.05}66.80±0.15{66.80}^{\pm 0.15}
CAT +GPTQ{}^{+\text{GPTQ}}13.77±0.08{13.77}^{\pm 0.08}56.77±0.17{56.77}^{\pm 0.17}6.44±0.01{6.44}^{\pm 0.01}69.64±0.18{69.64}^{\pm 0.18}7.14±0.00{7.14}^{\pm 0.00}70.99±0.03\bf{70.99}^{\pm 0.03}9.92±0.04{9.92}^{\pm 0.04}67.14±0.28{67.14}^{\pm 0.28}
WUSH +GPTQ{}^{+\text{GPTQ}}13.85±0.08{13.85}^{\pm 0.08}56.81±0.14{56.81}^{\pm 0.14}6.44±0.01{6.44}^{\pm 0.01}69.43±0.33{69.43}^{\pm 0.33}7.13±0.01{7.13}^{\pm 0.01}70.83±0.05{70.83}^{\pm 0.05}9.88±0.08{9.88}^{\pm 0.08}67.06±0.32{67.06}^{\pm 0.32}
Table 3: For block-wise quantization, HZ{\bm{H}}_{\bm{Z}} -PTQ consistently achieves the lowest perplexity across both W4-g32 and W4A4-g32 settings, demonstrating that the proposed Hessian-guided objective remains effective even at finer quantization granularities.
 Llama v3.2 1B inst.Llama v3 8BMinistral 8B inst.Qwen v3 8B
Methodwiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )
HZ{\bm{H}}_{\bm{Z}} -PTQ16.05±0.02{16.05}^{\pm 0.02}54.11±0.19{54.11}^{\pm 0.19}7.06±0.01{7.06}^{\pm 0.01}67.25±0.23{67.25}^{\pm 0.23}7.69±0.01{7.69}^{\pm 0.01}69.30±0.26{69.30}^{\pm 0.26}10.43±0.03{10.43}^{\pm 0.03}65.56±0.46{65.56}^{\pm 0.46}
−- Round16.52±0.07{16.52}^{\pm 0.07}53.13±0.57{53.13}^{\pm 0.57}7.13±0.00{7.13}^{\pm 0.00}66.54±0.04{66.54}^{\pm 0.04}7.74±0.02{7.74}^{\pm 0.02}69.10±0.15{69.10}^{\pm 0.15}10.56±0.04{10.56}^{\pm 0.04}65.14±0.42{65.14}^{\pm 0.42}
−- A.clip17.05±0.03{17.05}^{\pm 0.03}52.35±0.36{52.35}^{\pm 0.36}7.41±0.01{7.41}^{\pm 0.01}65.72±0.14{65.72}^{\pm 0.14}7.89±0.01{7.89}^{\pm 0.01}68.35±0.62{68.35}^{\pm 0.62}10.63±0.05{10.63}^{\pm 0.05}64.08±0.05{64.08}^{\pm 0.05}
−- Tran.18.64±0.13{18.64}^{\pm 0.13}51.68±0.86{51.68}^{\pm 0.86}7.61±0.01{7.61}^{\pm 0.01}66.06±0.39{66.06}^{\pm 0.39}8.13±0.01{8.13}^{\pm 0.01}67.98±0.30{67.98}^{\pm 0.30}25.19±0.90{25.19}^{\pm 0.90}55.80±0.48{55.80}^{\pm 0.48}
−- Hess.16.83±0.10{16.83}^{\pm 0.10}52.80±0.64{52.80}^{\pm 0.64}7.37±0.02{7.37}^{\pm 0.02}65.75±0.11{65.75}^{\pm 0.11}7.83±0.03{7.83}^{\pm 0.03}68.90±0.34{68.90}^{\pm 0.34}10.42±0.04{10.42}^{\pm 0.04}65.77±0.03{65.77}^{\pm 0.03}
Table 4: Ablation for the components for HZ{\bm{H}}_{\bm{Z}} -PTQ on W4A4KV4. −- Round refers to a version with round-to-nearest and a standard LpL_{p} weight range-setting. −- Trans. refers to the use of Hadamard transform instead of our gradient alignment transforms. −- Hess. indicates the use of a GPTQ-style hessian approximation instead of our output Kronecker approximation.
Figure 2: WikiText perplexity obtained by several weight scale-setting objectives on Llama 3.2 1B inst. under W4 per-channel quantization, with and without Hadamard transformations. Weighting the quantization error using the proposed Fisher estimate consistently exceeds the best reconstruction objective without requiring hyperparameter tuning.
  W4 Per-channelW4 Per-block (32)
Tran.Hessian EstimateGPTQYAQAoursGPTQYAQAours
Id.XTX⊗I{\bf X}^{T}{\bf X}\otimes{\bf I}30.96±4.58{30.96}^{\pm 4.58}30.56±0.00{30.56}^{\pm 0.00}17.65±0.09{17.65}^{\pm 0.09}14.35±0.04{14.35}^{\pm 0.04}69.59±10.14{69.59}^{\pm 10.14}14.11±0.07{14.11}^{\pm 0.07}
GWGWT⊗GWTGW{\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{W}^{T}{\bf G}_{W}18.42±0.48{18.42}^{\pm 0.48}18.81±0.10{18.81}^{\pm 0.10}15.34±0.03{15.34}^{\pm 0.03}14.28±0.11{14.28}^{\pm 0.11}13.75±0.05{13.75}^{\pm 0.05}13.64±0.05{\bf 13.64}^{\pm 0.05}
GWGWT⊗GZTGZ{\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{Z}^{T}{\bf G}_{Z}19.66±3.89{19.66}^{\pm 3.89}19.49±0.42{19.49}^{\pm 0.42}15.23±0.06{\bf 15.23}^{\pm 0.06}14.26±0.25{14.26}^{\pm 0.25}13.92±0.08{13.92}^{\pm 0.08}13.64±0.02{\bf 13.64}^{\pm 0.02}
Had.XTX⊗I{\bf X}^{T}{\bf X}\otimes{\bf I}14.96±0.08{14.96}^{\pm 0.08}20.68±0.00{20.68}^{\pm 0.00}14.50±0.01{14.50}^{\pm 0.01}13.68±0.03{13.68}^{\pm 0.03}13.90±0.07{13.90}^{\pm 0.07}13.69±0.05{13.69}^{\pm 0.05}
GWGWT⊗GWTGW{\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{W}^{T}{\bf G}_{W}14.50±0.06{14.50}^{\pm 0.06}19.12±0.29{19.12}^{\pm 0.29}14.26±0.05{14.26}^{\pm 0.05}13.61±0.01{13.61}^{\pm 0.01}13.70±0.05{13.70}^{\pm 0.05}13.56±0.02{\bf 13.56}^{\pm 0.02}
Table 5: Comparison of rounding algorithms for different hessian estimates on the Llama v3.2 1B instruct architecture in two quantization settings. The bottom rows report the effect of applying a Hadamard transform prior to quantization. GWGWT⊗GZTGZ{\bf G}_{W}{\bf G}_{W}^{T}\otimes{\bf G}_{Z}^{T}{\bf G}_{Z} refers to our approximation.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

MethodW. ScaleW. RoundAct. ClipTransforms
RTN2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|Noneγ=1\gamma=1None
QuaRot +GPTQmin⁡sw∥w−Q(w;sw)∥2.4\min_{s_{w}}\|{\bm{w}}-Q({\bm{w}};s_{w})\|^{2.4}GPTQγ=1\gamma=1(Fused) H{\bm{H}}
CAT +GPTQmin⁡sw∥w−Q(w;sw)∥2.4\min_{s_{w}}\|{\bm{w}}-Q({\bm{w}};s_{w})\|^{2.4}GPTQγ=1\gamma=1B(ΣX−1#ΣW)1/2HB(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{H}}
FlatQuant +GPTQLearned sws_{w}GPTQLearned γ\gammaLearned Ti⊗To{\bm{T}}_{i}\otimes{\bm{T}}_{o}
HZ{\bm{H}}_{\bm{Z}} -PTQAlgorithm in Sec. 3.5Sec. 3.6B(ΣGW−1#ΣGX)1/2HB(\bm{\Sigma}_{{\bm{G}}_{\bm{W}}}^{-1}\#\bm{\Sigma}_{{\bm{G}}_{\bm{X}}})^{1/2}{\bm{H}}
Appendix
Table 6: Details on the setting used to produce Table 2 . The block size for the transforms used for CAT and our method is 128.
MethodWeight ScaleWeight RoundAct. ClipBlock Transforms
RTN2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|Noneγ=1\gamma=1None
Had. +GPTQ2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|GPTQγ=1\gamma=1Hadamard
Had. +YAQA2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|YAQAγ=1\gamma=1Hadamard
CAT +GPTQ2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|GPTQγ=1\gamma=1(ΣX−1#ΣW)1/2H(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{H}}
WUSH +GPTQ2max⁡i∣wi∣2\max_{i}|{\bm{w}}_{i}|GPTQγ=1\gamma=1(ΣX−1#ΣW)1/2UH(\bm{\Sigma}_{\bm{X}}^{-1}\#\bm{\Sigma}_{\bm{W}})^{1/2}{\bm{U}}{\bm{H}}
HZ{\bm{H}}_{\bm{Z}} -PTQAlgorithm in Sec. 3.5γ=1\gamma=1(ΣGW−1#ΣGX)1/2H(\bm{\Sigma}_{{\bm{G}}_{\bm{W}}}^{-1}\#\bm{\Sigma}_{{\bm{G}}_{\bm{X}}})^{1/2}{\bm{H}}
Appendix
Table 7: Details on the setting used to produce Table 3 . The transformation in this table already refer to a single block of size 32.
Figure 3: Wikitext perplexity obtained by different activation clipping objectives on Llama 3.2 1B Instruct under 4-bit per token quantization, with and without Hadamard transformations.
 Llama v3.2 1B inst.Llama v3 8BMinistral 8B inst.Qwen v3 8B
Methodwiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )wiki ( ↓\downarrow )csr ( ↑\uparrow )
FP13.1657.616.1470.126.9771.299.7367.46
QuaRot29.7746.2310.9957.079.1464.5513.6857.33
CAT18.20±0.10{18.20}^{\pm 0.10}51.41±0.28{51.41}^{\pm 0.28}7.67±0.01{7.67}^{\pm 0.01}63.90±0.24{63.90}^{\pm 0.24}7.99±0.01{7.99}^{\pm 0.01}67.72±0.20{67.72}^{\pm 0.20}11.01±0.04{11.01}^{\pm 0.04}63.66±0.05{63.66}^{\pm 0.05}
FlatQuant17.09±0.04{17.09}^{\pm 0.04}52.60±0.42{52.60}^{\pm 0.42}7.50±0.52{7.50}^{\pm 0.52}66.28±2.04{66.28}^{\pm 2.04}7.84±0.05{7.84}^{\pm 0.05}68.71±0.14{68.71}^{\pm 0.14}10.55±0.13{10.55}^{\pm 0.13}65.34±0.32{65.34}^{\pm 0.32}
HZ{\bm{H}}_{\bm{Z}} -PTQ -a.clip17.78±0.12{17.78}^{\pm 0.12}51.42±0.16{51.42}^{\pm 0.16}7.51±0.01{7.51}^{\pm 0.01}64.55±0.56{64.55}^{\pm 0.56}7.95±0.02{7.95}^{\pm 0.02}68.23±0.25{68.23}^{\pm 0.25}10.78±0.04{10.78}^{\pm 0.04}63.21±0.00{63.21}^{\pm 0.00}
Appendix
Table 8: A4W4KV4 Quantization results without any weight rounding optimization and standard LpL_{p} -based ranges.
More in this topic

Quantization

See all 54 papers

Explore similar work

CardsList

  1. Theory-optimal Quantization Based on Flatness

    May 11, 2026Xiusheng Huang, Zhe Li, Xuanwu Yin +5LLM QuantizationActivation Quantization

    The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2

  2. Why Does Post-Training Quantization Work?

    Sep 10, 2026Yuxiang Chen, Michael Beyer, Jun Zhu +1LLM QuantizationQuantization Robustness

    Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University · College of AI, Tsinghua University · Bosch AI Research, Renningen, Germany

  3. Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

    Apr 15, 2026Jaemin Kim, Sungkyun Kim, Junyeol Lee +1LLM QuantizationLanguage Model Calibration

    Seoul National University · Hanyang University