PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

Yanlong Chen; Yining Chen; Song Zhang; Amirhossein Habibian; Yawei Li

cs.AISep 26, 2026

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

Authors: Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, Yawei Li

Organizations: Nanyang Technological University, Singapore · Nanjing University of Posts and Telecommunications, Nanjing, China · Nanjing University, Nanjing, China · Qualcomm AI Research, Amsterdam, Netherlands

Abstract

Smaller activation outliers do not necessarily imply better low-bit quantization: their alignment with the quantizer matters. We introduce PrismQuant, a quantizer-aware rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4. The affine offsets represent the energy in this subspace without widening the range within the group. We formulate rotation design as a Ky Fan trace maximization and derive a closed-form solution that is provably optimal for this alignment objective. Compact Householder transformations and their compact-WY representation enable gradient-free construction and efficient application at both foldable and online sites. A predictive range law further connects unaligned activation energy and group size to quantization-relevant variation. Experiments on Llama, Qwen, and Mistral span dense models up to 70B parameters and a 30B mixture-of-experts model. Under W4A4KV4, PrismQuant sets the state of the art on Llama-3.2-3B among the compared methods in both perplexity and accuracy. On Llama-3.1-70B, it attains 3.85 perplexity and 72.46% average zero-shot accuracy, only 0.22 percentage points below full precision. In the deployment study on Llama-3.1-8B, our optimized implementation achieves 1.51x prefill and 1.22x CUDA Graph decode speedups over matched FP16 baselines, with 56.34% lower decode peak memory and only 2.35% additional Graph decode latency over Hadamard. Code and models are available at https://github.com/ForeverBlue816/PrismQuant and https://huggingface.co/collections/ForeverBlue/prsimquant.

Figures & tables

Figure 1: Aligning activations with quantizer geometry. Llama-3.2-3B layer 27 down-projection inputs, $g{=}128$ . (a–d) PrismQuant reduces the mean within-group range from $1.76$ to $0.84$ relative to Hadamard under matched asymmetric INT4 quantization. (e–g) Token/group traces illustrate the mechanism; the dashed line marks the stored affine offset.

Figure 2: Quantizer-aware alignment predicts INT4 error. All 28 Llama-3.2-3B down-projection inputs,

g{=}128

, evaluated on identical tokens. (a) Energy captured by the group-constant subspace. (b,c) Within-group range and activation NMSE under matched asymmetric INT4; PrismQuant reduces them by

25\%

and

40\%

on average relative to Hadamard.

Figure 3: Overview of PrismQuant. Calibration takes the leading eigenvectors $V_{k}$ of the second moment $\Sigma$ and builds a rotation that turns them into per-group shared levels, which the affine offset $z$ represents outside the INT4 range. $R_{1}$ is folded into the embedding, head, and all residual-facing weights; $R_{2}$ is applied after $W_{V}$ and its transpose folded into $W_{O}$ so the V cache is quantized in the aligned basis; at the down-projection input only the rank- $k$ correction $I-W^{\prime}Y^{\prime\top}$ and the block Hadamard $H_{128}$ run online (clock), with $R_{4}^{\top}$ folded into $W_{D}$ . The schematic uses right-acting row-vector rotations.

		Zero-shot accuracy $\uparrow$
Method	PPL $\downarrow$	ARC-c	ARC-e	BoolQ	HellaS	OBQA	PIQA	SIQA	WinoG	Avg. $\uparrow$
Llama-3.2-3B (bf16)	7.80	46.20	71.70	73.10	73.70	43.40	77.40	47.20	69.10	62.73
QuaRot (NeurIPS ’24)	10.10	38.60	59.00	65.90	66.50	35.80	74.40	43.10	65.20	56.06
SpinQuant (ICLR ’25)	9.20	38.90	64.80	68.00	69.10	39.40	74.90	45.10	62.90	57.89
KurTail (EMNLP-F ’25)	9.00	42.20	66.70	69.80	68.80	39.80	75.60	44.80	64.60	59.04
ResQ (ICML ’25)	8.80	43.10	65.60	68.80	70.50	38.40	75.10	45.60	64.80	58.99

Table 1: W4A4KV4 quantization across the Llama family. WikiText-2 perplexity (

\downarrow

) and zero-shot accuracy (%,

\uparrow

); Avg. averages the eight displayed tasks. Published rows and the 3B/8B bf16 references follow Wang et al. (2026) (3B) and He et al. (2025) (8B/70B); the 70B reference, Hadamard, and PrismQuant rows are measured in our pipeline. Bold marks the best quantized result per column within each model; PrismQuant rows are shaded light for

k=8

and darker for

k=\max


	PPL $\downarrow$		Zero-shot accuracy $\uparrow$
Method	WT2	C4	ARC-c	ARC-e	BoolQ	HellaS	OBQA	PIQA	SIQA	WinoG	Avg. $\uparrow$
Qwen3-30B-A3B (bf16)	6.11	10.70	57.68	79.55	81.41	81.40	44.00	80.96	52.15	72.53	68.71
Hadamard	6.68	11.47	55.38	79.04	81.41	79.24	44.80	79.33	50.26	70.32	67.47
PrismQuant ( $k=8$ )	6.45	11.19	57.25	79.29	81.90	80.08	45.40	80.30	51.28	72.61	68.52
PrismQuant ( $k=\max$ )	6.42	11.18	55.63	79.84	83.55	79.60	44.40	79.60	51.94	71.27	68.23

Table 2: W4A4KV4 quantization of Qwen3-30B-A3B-Base. WikiText-2 and C4 perplexity (

\downarrow

) and zero-shot accuracy (%,

\uparrow

) on the eight-task suite of Table 1 . All rows are evaluated in our pipeline with three seeds; the router is kept in bf16, and every expert receives its own

R_{4}

(

k=6

, its full slot count), so the two PrismQuant rows differ only in the rank of

R_{1}

. Bold marks the best quantized result per column.

Configuration	Llama - 3.2 - 3B	Llama - 3.1 - 8B	Qwen3 - 8B
Hadamard gap, $P_{\mathrm{Had}}-P_{\mathrm{bf16}}$ (PPL)	$\mathbf{+0.154}$	$\mathbf{+0.142}$	$\mathbf{+0.442}$

PrismQuant ( $k=8$ )	38.59	35.53	59.12
PrismQuant ( $k=16$ )	38.30	35.74	51.97
PrismQuant ( $k=32$ )	41.88	40.23	85.57
PrismQuant ( $k=64$ )	40.83	41.84	99.85

Table 3: Alignment rank. Activation-only recovery (

\%

\uparrow

) of the Hadamard-to-bf16 WikiText-2 perplexity gap by PrismQuant at rank

k

; the first row lists the gap itself. Weights and KV states remain in bf16; ranks are capped per site by the number of group slots. Bold marks the largest recovery; row shading darkens with

k

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix


	Perplexity $\downarrow$		Task score $\uparrow$
Method	WT2	C4	MMLU	MMLU-R	GSM8K	ARC-e
Qwen3-0.6B (bf16)	12.66	19.38	52.52	55.83	60.96	57.95
Hadamard	16.50	25.36	43.46	43.49	28.96	56.69
PrismQuant ( $k=8$ )	15.28	23.92	44.22	46.23	31.54	53.62
PrismQuant ( $k=\max$ )	15.38	23.92	44.69	45.27	28.66	56.36

Appendix

Table 4: W4A4KV4 quantization across the Qwen3 dense family. All models are Base checkpoints evaluated in our pipeline with three seeds. WT2 and C4 report perplexity (

\downarrow

); task scores are percentages (

\uparrow

). MMLU and MMLU-R (Redux) are 5-shot; GSM8K is 4-shot CoT; ARC-e is zero-shot ( Raffel et al., 2020 ; Cobbe et al., 2021 ; Wang et al., 2024 ; Gema et al., 2025 ) . Bold marks the best quantized value per model and column, including rounded ties.

	PPL $\downarrow$		Zero-shot accuracy $\uparrow$								Mean $\uparrow$
Method	WT2	C4	ARC-c	ARC-e	BoolQ	HellaS	OBQA	PIQA	SIQA	WinoG	Avg. 5	Avg. 8
Mistral-7B-v0.3 (bf16)	5.45	8.32	54.35	80.13	83.64	80.67	47.40	81.99	47.75	74.19	74.27	68.77
QuaRot (NeurIPS ’24)	6.41	10.01	45.22	72.26	—	75.14	—	79.22	—	68.82	68.13	—
SmoothRot (SMC ’25)	6.19	9.78	46.59	73.19	—	75.97	—	79.11	—	68.11	68.59	—
SpinQuant (ICLR ’25)	5.80	—	52.50	77.30	80.20	79.20	58.40	80.30	48.90	72.30	72.32	68.60
PrefixQuant (TPAMI ’26)	5.76	—	50.00	77.74	—	78.39	—	80.41	—	70.88	71.48	—

Appendix

Table 5: W4A4KV4 quantization on Mistral-7B-v0.3. WT2/C4 perplexity (

\downarrow

) and zero-shot accuracy (%,

\uparrow

). Avg. 5 averages ARC-c, ARC-e, HellaS, PIQA, and WinoG; Avg. 8 covers all eight tasks. The bf16 reference is ours; QuaRot and SmoothRot rows are their reported RTN results, and cross-paper protocols differ. A dash denotes an unreported result. Bold marks the best displayed quantized result per column.

(a) Activation metadata allocation: WikiText-2 PPL
Configuration	Llama-3.2-3B		Llama-3.1-8B
$(g,m)$ ; bits/value	H	PQ	H	PQ
$(256,1)$ ; 4.13	7.79	7.74 ( $-0.05$ )	6.37	6.31 ( $-0.06$ )
$(256,2)$ ; 4.19	7.79	7.73 ( $-0.06$ )	6.38	6.30 ( $-0.08$ )
$(256,3)$ ; 4.25	7.79	7.73 ( $-0.06$ )	6.37	6.30 ( $-0.07$ )
$(128,1)$ ; 4.25 default	7.77	7.71 ( $-0.06$ )	6.35	6.29 ( $-0.06$ )

Appendix

Table 6: Ablations of metadata allocation and expert calibration. (a) Activation-only WikiText-2 PPL (

\downarrow

) under matched representation budgets; H/PQ denote Hadamard/PrismQuant at

k=\max

, and green parentheses give

\mathrm{PQ}-\mathrm{H}

. (b) W4A4KV4 WikiText-2 PPL (

\downarrow

) for the MoE calibration variants at

k=\max

;

\Delta

PPL is relative to shrinkage, from unrounded values. Bold denotes the lower PPL within each H/PQ pair in (a) and the lowest PPL in (b); mint rows mark the configuration adopted in the main tables.

(a) Both sites quantized: WikiText-2 PPL
Subspace estimator	Llama-3.2-3B		Llama-3.1-8B
	PPL	$\Delta$ [90% CI]	PPL	$\Delta$ [90% CI]
All tokens (PrismQuant)	7.705	$0$	6.285	$0$
Top-1 token / seq. (OffQ-style)	7.723	$+0.017$ [ $-$ 0.002, $+$ 0.036]	6.301	$+0.016$ [ $+$ 0.012, $+$ 0.020]
BOS excluded	7.800	$+0.095$ [ $+$ 0.092, $+$ 0.098]	6.342	$+0.057$ [ $+$ 0.049, $+$ 0.065]
Global top 1% of tokens	7.711	$+0.006$ [ $-$ 0.012, $+$ 0.024]	6.293	$+0.008$ [ $+$ 0.002, $+$ 0.014]

Appendix

Table 7: Which tokens define the subspace. Activation-only WikiText-2 PPL with bf16 weights and KV states, three paired seeds,

k=\max

. (a) Both sites quantized;

\Delta

is the paired difference to the all-token estimator with its 90% Student-

t

interval. (b)

\Delta

when only one site is quantized (BOS included); bold marks intervals that exclude zero. (c) Diagnostics at the down-projection input: captured group-constant energy

f

and per-group range relative to Hadamard. Mint rows mark our estimator.

(a) Calibration set: size and corpus
Calibration	WT2 PPL	$\Delta$ [90% CI]	C4 PPL	$\Delta$ [90% CI]
Hadamard (no calibration)	7.877	$0$	13.160	$0$

WT2, 64 seq.	7.690	$-0.187$ [ $-$ 0.199, $-$ 0.174]	12.851	$-0.309$ [ $-$ 0.330, $-$ 0.289]
WT2, 128 seq. default	7.694	$-0.183$ [ $-$ 0.194, $-$ 0.171]	12.844	$-0.316$ [ $-$ 0.337, $-$ 0.295]
WT2, 256 seq.	7.703	$-0.174$ [ $-$ 0.185, $-$ 0.163]	12.852	$-0.309$ [ $-$ 0.330, $-$ 0.287]

Appendix

Table 8: Calibration and estimation robustness on Qwen3-4B-Base. Activation-only WikiText-2 (WT2) and C4 PPL, three paired seeds,

k=\max

;

\Delta

is the paired difference to Hadamard with its 90% interval. (a) Calibration set size and corpus. (b) Randomized eigensolver passes versus exact fp64 eigendecomposition;

f

is the captured energy at the down-projection input,

\Delta

is relative to the exact solver. Mint rows mark the default.

(a) Quantizer format
Quantizer	Qwen3-4B-Base			Llama-3.2-3B
	Had	PQ	$\Delta$	Had	PQ	$\Delta$
Asymmetric, $g=128$ default	7.877	7.697	$-0.179$	7.767	7.705	$-0.061$
Symmetric, $g=128$	7.958	7.764	$-0.194$	7.819	7.759	$-0.060$
Asymmetric, $g=d$ ( $k=1$ )	8.075	7.857	$-0.218$	7.926	7.886	$-0.041$
Symmetric, $g=d$ ( $k=1$ )	8.178	8.041	$-0.137$	8.008	7.995	$-0.013$

Appendix

Table 9: Quantizer format and target column. Activation-only WikiText-2 PPL, three paired seeds, both sites quantized. (a) The same PrismQuant rotation (

k=\max

;

k=1

for

g=d

) under four quantizers;

\Delta

is PQ minus Hadamard, and every 90% interval excludes zero. (b) Anchors moved from the constant Walsh column to a nonconstant one at

g=128

k=\max

;

\Delta

is relative to the constant column under the asymmetric and symmetric formats, and

s

is the offset-specific share

\Delta_{\rm asym}/(\mathrm{PPL}_{\rm Had}-\mathrm{PPL}_{\rm PQ})

. Mint rows mark the configuration of the main tables.

Variant	$k=8$		$k=\max$
	PPL	$\Delta$ [90% CI]	PPL	$\Delta$ [90% CI]
Balanced $\Pi$ , low-energy fillers default	7.707	$0$	7.697	$0$
Identity $\Pi$	7.710	+0.002 [ $-$ 0.004, $+$ 0.009]	7.697	$-$ 0.001 [ $-$ 0.007, $+$ 0.006]
Random $\Pi$	7.708	+0.001 [ $-$ 0.005, $+$ 0.007]	7.694	$-$ 0.003 [ $-$ 0.011, $+$ 0.004]
High-energy fillers	7.698	$-0.009$ [ $-$ 0.015, $-$ 0.003]	7.697	$0$ (identical)
Fixed signs $D=+1$	7.724	$+0.017$ [ $+$ 0.011, $+$ 0.023]	7.697	$-$ 0.001 [ $-$ 0.007, $+$ 0.005]

Appendix

Table 10: Anchor placement, fillers, and signs on Qwen3-4B-Base. Activation-only WikiText-2 PPL, three paired seeds,

G

identical across rows;

\Delta

is relative to the default at the same rank with its 90% interval. Hadamard:

7.877

	Qwen3-4B-Base	Llama-3.2-3B
$\|z\|/\mathrm{range}$ , median / P99 / max (worst layer)	0.51 / 10.0 / 21.5	0.52 / 5.8 / 10.2
Offset error / step, median / P99 (worst layer)	0.0012 / 0.025	0.0012 / 0.017
Groups with error $>0.5$ step, overall	$8\times 10^{-5}\%$	$5\times 10^{-5}\%$
worst layer/site	$0.003\%$	$0.0008\%$
Median (offset error)/(unaligned cost)	0.0085	0.0093

Appendix

Table 11: fp16 offset rounding at fixed codes. PrismQuant at

g=128

k=\max

, activation-only, three seeds. Offset error is

|z_{j}-\mathrm{fp16}(z_{j})|

in units of the group’s step; the unaligned cost is

2|c|/15

for the same level

c

\Delta

PPL rows re-store only the offset at the stated precision.

Figure 4: From null-space geometry to a shared range law. (a) Standardized token projections at the Llama-3.2-3B layer-27 down input; arrows are the group-constant direction pulled back through each rotation. (b) Cumulative energy in the leading directions at seven layers. (c) Group range relative to paired Hadamard versus $\sqrt{1-f}$ for 2,520 activation, 280 V-cache, and 112 multi-slot observations under one pooled fit; dashed is the identity. (d) The same fitted line on 5,342 Qwen3-30B-A3B experts held out of the fit: bin medians (Q1–Q3 bars) follow the dense reference line, and the scatter is widest for experts with fewer than 2,048 tokens.

Model	Site	$k=8$ median $\|e\|$	$k=\max$ median $\|e\|$	$k=\max$ P90 $\|e\|$	$k=\max$ median $e$	$k=\max$ $\kappa_{\rm PQ}/\kappa_{\rm H}$
Llama-3.2-3B	q/k/v	1.7%	1.7%	8.9%	$+1.7\%$	0.98
Llama-3.2-3B	down	11.1%	8.9%	15.0%	$+8.4\%$	$0.92$

Llama-3.1-8B	q/k/v	2.0%	1.3%	6.4%	$+1.3\%$	0.99
Llama-3.1-8B	down	12.2%	8.8%	15.8%	$+8.6\%$	$0.92$

Appendix

Table 12: Per-site error of the range law at

g=128

. Relative error

e=(\widehat{s}-s_{\rm meas})/s_{\rm meas}

of the predicted step, activation-only protocol, three paired seeds; medians and 90th percentiles of

|e|

over layers, and the median signed error, at the default operating point

k=8

and at

k=\max

. Positive signed error means the law over-predicts the step; the last column is the crest-factor ratio of Equation 30 ,

1/(1+\mathrm{median}\,e)

Figure 5: Predicted versus measured quantization step. Each point is one layer and site of the activation-only studies at

g=128

; whiskers are 90% intervals over three paired seeds and the dashed line is the identity. Points above the line are over-predicted. The q/k/v input follows the identity closely at every rank; the down-projection input sits below it, more so on Qwen3-4B-Base; the ratio of measured to predicted step there is the crest-factor ratio of Equation 30 .

Figure 6: Activation design and local transform cost. (a) Activation representation budget versus WikiText-2 PPL on Llama-3.2-3B;

(g,m)

denotes group size and the number of represented directions. (b) Activation-only PPL recovery versus rank. The highlighted

k=8

category is the deployment operating point; the actual rank is capped separately at each quantization site. Llama results use three seeds and 64 chunks per seed; Qwen results use three seeds and 146 chunks, with bf16 weights and KV states. (c) Original local cost measurements at

T=2048

on an NVIDIA RTX PRO 6000 Blackwell Server Edition:

100t_{\mathrm{transform}}/(t_{\mathrm{decoder\ layer}}+t_{\mathrm{transform}})

. These separately benchmarked durations are not whole-model decode overhead; end-to-end deployment is evaluated separately.

Figure 7: Practical efficiency of the compact PrismQuant implementation. The upper strip shows the two-kernel $R_{4}$ computation followed by the shared INT4 quantizer and GEMM; box widths do not represent time. (a) Eager prefill throughput relative to matched FP16, with 2048 input tokens per sequence. (b,c) Matched CUDA Graph decode latency and peak allocated memory on A40, using batch size 1 and a 2048-token prefix. (d) Local implementation ablations: pre-bound versus generic launches at $T=1$ , and complete $R_{4}$ paths using Tensor Core versus shuffle-based Kernel B at $T=2048$ . Each ablation has its own reference; these are not whole-model speedups. Timing centers are pooled medians; ablation centers are medians of paired-session ratios. Whiskers show session ranges, not confidence intervals. All performance models use random weights.

	Prefill $\uparrow$		Decode $\downarrow$		Graph deployment
Method	B1	B16	Eager	Graph	Peak GB $\downarrow$	Speedup $\uparrow$
Llama-3.2-3B (FP16)	10,163.48	13,985.13	21.86	15.04	8.74	$1.00\times$
QuaRot–Hadamard	12,819.74	17,894.66	42.83	16.33	4.37	$\mathbf{0.92}\times$
PrismQuant ( $k=8$ )	12,674.40	17,630.27	43.08	16.65	4.39	$0.90\times$
Llama-3.1-8B (FP16)	5,241.70	6,418.43	30.22	28.83	18.77	$1.00\times$
QuaRot–Hadamard	7,607.40	9,556.00	52.31	23.10	8.15	$\mathbf{1.25}\times$

Appendix

Table 13: End-to-end performance on a single NVIDIA A40. Prefill reports input tokens/s (

\uparrow

), and decode reports ms/token (

\downarrow

). Both decode columns use the matched current-stream backend; Graph speedup and peak memory use CUDA Graph execution. Speedup is relative to the FP16 row of the same model and mode. All rows use random weights; bold marks the best quantized entry per model and metric.

Model	$T=1$	$T=2048$	$T=32768$
Llama-3.2-3B	$\mathbf{1.25}\times$	$0.93\times$	$0.85\times$
Llama-3.1-8B	$\mathbf{1.73}\times$	$\mathbf{1.17}\times$	$\mathbf{1.12}\times$

Appendix

Table 14: Rotation-plus-quantization frontend speedup over QuaRot. Each value is the reciprocal of the median of three paired-session PrismQuant/Hadamard elapsed-time ratios. Values above

1.00\times

indicate faster execution and are bold. These are directly measured local frontend chains, not end-to-end model speedups;

T

is the number of input activation rows.

Figure 8: Local end-to-end activation magnitudes at q_proj on Qwen3-8B-Base. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( $k=8$ ), and PrismQuant ( $k=\max$ ). The window is sample 0, tokens 0–127, and channels 0–511, with $g=128$ . Heights represent absolute input activations before quantization. Linear heights and square-root colors share limits across all four rows within each column; the pale plane marks $z=0$ . The rotated rows include upstream quantization effects.

Figure 9: Local end-to-end activation magnitudes at down_proj on Qwen3-8B-Base. The rotated rows are observed after the online $R_{4}$ transform and before activation quantization. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( $k=8$ ), and PrismQuant ( $k=\max$ ). We retain sample 0, tokens 0–127, and channels 0–511, corresponding to four groups with $g=128$ . Heights are absolute activations, not centered residuals. Each column shares linear height limits and square-root color mapping across its four rows, with a pale $z=0$ plane.

Figure 10: Paired-local activation magnitudes at q_proj on Qwen3-8B-Base. All methods transform the same captured input tensor within each block, isolating the local effect of rotation. Columns show Blocks 1, 13, 24, and 36; rows show the unrotated reference, Hadamard, PrismQuant ( $k=8$ ), and PrismQuant ( $k=\max$ ). The displayed window contains sample 0, tokens 0–127, and channels 0–511, with $g=128$ and no pooling. Heights are linear and colors use square-root mapping, with identical local limits across the four rows of each column; the pale plane marks $z=0$ .

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 386 papers

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
Oct 19, 2026Desheng Xu, Hanglin Li, Tianchen Song +2LLM Quantization Post-Training Quantization
Li Auto Inc.
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Jul 22, 2026Miguel P. Bento, João F. SeabraLLM Quantization Rotation-Based Quantization
Independent Researcher · Lisbon, Portugal
SpectrumQuant: Spectrum Alignment for LLM Quantization
Oct 14, 2026Dwaipayan Haldar, Indranil Patra, Azhar Yousuf +1LLM Quantization Rotation-Based Quantization
Indian Institute of Science Bengaluru-560012, Karnataka, India

Oct 19, 2026J/K move · Enter open · S save

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

Desheng Xu, Hanglin Li, Tianchen Song, Zhiyong Zhao+1

Li Auto Inc.

LLM Quantization Post-Training Quantization Low-Bit Quantization Rotation-Based Quantization Activation Quantization LLM Inference Acceleration

Open paper AskPDF arXiv

Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to

2.3\times

prefill speedup and

3.5\times

memory saving over FP16.

Abstract

LLM Quantization

Explore similar work

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries

SpectrumQuant: Spectrum Alignment for LLM Quantization