SpectrumQuant: Spectrum Alignment for LLM Quantization

Dwaipayan Haldar; Indranil Patra; Azhar Yousuf; Chandra Sekhar Seelamantula

cs.LGOct 14, 2026

SpectrumQuant: Spectrum Alignment for LLM Quantization

Authors: Dwaipayan Haldar, Indranil Patra, Azhar Yousuf, Chandra Sekhar Seelamantula

Organizations: Indian Institute of Science Bengaluru-560012, Karnataka, India

Abstract

Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an effective and efficient data-aware alternative for post-training quantization. Experiments on Llama and Mistral models show that SpectrumQuant is competitive with state-of-the-art rotation strategies while requiring less than one-third of the GPU memory and significantly less time than learned rotations approaches like SpinQuant and DartQuant.

Figures & tables

	Data-Agnostic Rotation (QuaRot, QuIP#)	Learned Rotation (SpinQuant, OSTQuant)	Calibration-Optimized Rotation (DartQuant, ButterflyQuant)	Non-Orthogonal Transform (WUSH, CAT)	SpectrumQuant (Ours)
Closed-form	✓	✗	✗	✓	✓
Data-aware	✗	✓	✓	✓	✓
Zero inference overhead	✓	✓	✓	✗	✓

Table 1: Qualitative characterization of different LLM quantization approaches.

Figure 1: Left to right: The top row shows distribution in original space, Hadamard-transformed space, and proposed transform space. The bottom row shows corresponding marginals.

Panel of Figure 2 — Figure 2: Structure of the solution space of the mixing component ${\bm{V}}$ in our COAT $\bm{Q}=\bm{U}\bm{V}$ . By Theorem 4.1 , the data-independent mixing components are exactly the normalized Hadamard matrices. The Butterfly matrices with rotation angles chosen from $\{\pi/4,3\pi/4,5\pi/4,7\pi/4\}$ are a subset of solutions; this subset admits $\mathcal{O}(d\log d)$ operations and contains the class of Sylvester-Hadamard matrices.

Figure 3: MSE as a function of the number of bits for a synthetic multivariate Gaussian experiment. The left panel uses AbsMax scaling $A=\|\bm{y}\|_{\infty}$ , and the right panel uses variance-based scaling $A=3\sqrt{\max_{i}\widehat{\mathrm{Var}}(y_{i})}$ .

Panel of Figure 4 — Figure 4: Placement of the rotations $\bm{R}_{1}$ to $\bm{R}_{4}$ in a Transformer block. Operations outlined in black are in FP16 and those outlined in blue are in INT4. The dashed lines indicate read and write operations to the KV cache.

Cost	Method	7B	13B	70B
GPU hour (h)	SpinQuant	0.41	0.84	46.40
	DartQuant	0.32	0.57	3.74
	SpectrumQuant	0.26	0.35	2.25
Memory (GiB)	SpinQuant	18.23	32.21	218.72
	DartQuant	15.79	27.80	128.21
	SpectrumQuant	4.51	7.05	22.42

Figure 5: Comparison of calibration cost. SpectrumQuant requires less than one-tenth the memory of SpinQuant and less than one-sixth the memory of DartQuant at 70B scale, while also being the fastest to calibrate across all model sizes.

Method	Llama-2 7B		Llama-2 13B		Llama-3 8B		Mistral 7B		Llama-2 70B
Method	PPL $\downarrow$	zero-shot 8 $\uparrow$	PPL $\downarrow$	zero-shot 8 $\uparrow$	PPL $\downarrow$	zero-shot 8 $\uparrow$	PPL $\downarrow$	zero-shot 8 $\uparrow$	PPL $\downarrow$	zero-shot 8 $\uparrow$
FP16	16.88	63.66	20.85	65.99	8.92	66.54	17.04	67.74	11.09	70.07
QuaRot	25.57 $\pm$ 1.22	59.73 $\pm$ 0.36	24.64 $\pm$ 0.45	63.34 $\pm$ 0.38	12.10 $\pm$ 0.13	59.70 $\pm$ 0.61	20.11 $\pm$ 0.42	64.37 $\pm$ 0.21	11.68 $\pm$ 0.14	68.66 $\pm$ 0.36
SpinQuant	22.92 $\pm$ 0.67	59.23 $\pm$ 0.54	24.93 $\pm$ 0.88	63.19 $\pm$ 0.50	11.67 $\pm$ 0.09	61.36 $\pm$ 0.44	19.50 $\pm$ 0.26	65.18 $\pm$ 0.18	12.26 $\pm$ 0.14	68.58 $\pm$ 0.50
DartQuant	20.46 $\pm$ 0.20	60.99 $\pm$ 0.43	24.47 $\pm$ 0.60	64.30 $\pm$ 0.14	10.95 $\pm$ 0.05	62.74 $\pm$ 0.35	18.73 $\pm$ 0.30	65.34 $\pm$ 0.28	12.00 $\pm$ 0.08	68.87 $\pm$ 0.42
SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	21.50 $\pm$ 0.46	60.06 $\pm$ 0.32	24.53 $\pm$ 0.56	63.71 $\pm$ 0.58	10.99 $\pm$ 0.12	62.57 $\pm$ 0.50	18.77 $\pm$ 0.17	65.45 $\pm$ 0.34	12.00 $\pm$ 0.11	68.94 $\pm$ 0.54

Table 2: Comparison of average PPL across three datasets and average accuracy on eight zero-shot common-sense reasoning tasks (zero-shot 8 ), over 5 seeds, under W4-A4-KV4 quantization. SpectrumQuant matches or outperforms SpinQuant on all five models and shows no statistically significant difference from DartQuant on four out of five models, without requiring iterative optimization. Detailed results are given in Appendix B .

Model	Dataset	WikiText2	PTB	C4
Llama-2 7B	WikiText2	$6.28_{\pm 0.06}$	$49.86_{\pm 1.31}$	$8.37_{\pm 0.06}$
	PTB	$6.23_{\pm 0.029}$	$44.95_{\pm 1.42}$	$8.392_{\pm 0.066}$
	C4	$6.28_{\pm 0.041}$	$46.90_{\pm 0.65}$	$8.438_{\pm 0.060}$
Mistral 7B v0.3	WikiText2	$5.67_{\pm 0.01}$	$41.64_{\pm 0.50}$	$9.00_{\pm 0.00}$
	PTB	$5.74_{\pm 0.014}$	$40.51_{\pm 0.63}$	$9.016_{\pm 0.027}$
	C4	$5.70_{\pm 0.010}$	$42.56_{\pm 1.21}$	$8.983_{\pm 0.015}$

Table 3: Comparison of PPL of SpectrumQuant (with

{\bm{Q}}=\bm{UH}

) across various calibration datasets for 5 seeds. SpectrumQuant shows no statistically significant difference across calibration datasets for Mistral 7B v0.3, and remains competitive across calibration datasets for Llama-2 7B, suggesting the approach is generally robust to calibration data selection.

No. of samples	WikiText2	PTB	C4	Avg. PPL $\downarrow$
64	6.169 ±0.033	49.07 ±2.24	8.393 ±0.065	21.21 ±0.76
128	6.28 ±0.06	49.86 ±1.31	8.37 ±0.06	21.50 ±0.46
256	6.188 ±0.038	50.22 ±2.12	8.429 ±0.023	21.61 ±0.72

Table 4: PPL comparison of SpectrumQuant (with

{\bm{Q}}=\bm{UH}

) across sample sizes for Llama-2 7B. SpectrumQuant shows no statistically significant difference in performance from 64 to 256 calibration samples, suggesting that the dominant covariance structure is reliably captured even at small sample counts.

Figure 6: Comparison of sorted per-channel quantization MSE across hidden dimensions for different projection activations in Llama-2 7B at layers 10 and 25.

Model	WikiText2	PTB	C4	Avg. PPL $\downarrow$	zero-shot 8 $\uparrow$
Llama-2 7B	6.21 ±0.04	49.93 ±1.00	8.49 ±0.10	21.54 ±0.38	60.26 ±0.46
Llama-3 8B	7.74 ±0.06	13.34 ±0.07	12.57 ±0.09	11.22 ±0.07	62.85 ±0.74
Mistral 7B v0.3	5.66 ±0.02	43.03 ±1.20	9.01 ±0.01	19.23 ±0.41	64.56 ±0.30

Table 5: PPL and average accuracy for zero-shot common-sense reasoning task with calibrated

\bm{R}_{1},\bm{R}_{2}

\bm{UB}

and

{\bm{R}_{3}},{\bm{R}_{4}}

as random butterfly matrices. Replacing the standard Hadamard matrices in

{\bm{R}_{3}},{\bm{R}_{4}}

with random

45^{\circ}

butterfly matrices shows no statistically significant difference in results.

	Quant	SpinQuant	DartQuant	SpectrumQuant
Decode Tokens/sec $\uparrow$	18.330	18.337	18.246	18.397
Decode ms/Token $\downarrow$	54.6	54.5	54.8	54.4

Table 6: Real-time inference cost of Llama-3 8B on MacBook Air M4. The rotation-based approaches including SpectrumQuant show no significant difference in inference throughput compared to the no-transform baseline (Quant), confirming that the rotation matrices are fully absorbed into the weights with no additional inference cost. Quant corresponds to quantization without applying a rotation transform.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Bits	Method	Wiki	PTB	C4	Avg $\downarrow$
(W-A-KV)	Method	Wiki	PTB	C4	Avg $\downarrow$
16	Full Precision	5.47	37.91	7.26	16.88
4-4-4	QuaRot	$6.21_{\pm 0.01}$	$62.05_{\pm 3.68}$	$8.45_{\pm 0.01}$	$25.57_{\pm 1.22}$
	SpinQuant	$6.49_{\pm 0.04}$	$53.50_{\pm 1.92}$	$8.77_{\pm 0.10}$	$22.92_{\pm 0.67}$
	DartQuant	$6.10_{\pm 0.02}$	$47.17_{\pm 0.59}$	$8.11_{\pm 0.02}$	$20.46_{\pm 0.20}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$6.28_{\pm 0.06}$	$49.86_{\pm 1.31}$	$8.37_{\pm 0.06}$	$21.50_{\pm 0.46}$

Appendix

Table 7: Perplexity of each method on WikiText2, PTB and C4 for Llama-2 7B. The two SpectrumQuant variants (with

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) are statistically indistinguishable from each other on every dataset and lie between QuaRot and DartQuant, indicating that the choice of mixing component does not affect performance.

Bits	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
(W-A-KV)	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
16	Full Precision	69.06	46.16	79.05	44.20	73.90	76.02	74.54	46.33	63.66
4-4-4	QuaRot	$66.17_{\pm 0.28}$	$44.66_{\pm 0.62}$	$76.61_{\pm 0.44}$	$39.12_{\pm 0.98}$	$69.37_{\pm 0.60}$	$72.61_{\pm 0.37}$	$68.01_{\pm 1.17}$	$41.30_{\pm 1.32}$	$59.73_{\pm 0.36}$
	SpinQuant	$65.35_{\pm 1.27}$	$43.64_{\pm 0.33}$	$76.36_{\pm 0.71}$	$39.52_{\pm 2.05}$	$68.27_{\pm 0.85}$	$71.45_{\pm 0.90}$	$68.25_{\pm 1.05}$	$40.99_{\pm 1.35}$	$59.23_{\pm 0.54}$
	DartQuant	$67.44_{\pm 0.46}$	$45.12_{\pm 0.74}$	$77.20_{\pm 0.43}$	$41.04_{\pm 1.69}$	$71.17_{\pm 0.36}$	$72.98_{\pm 0.25}$	$70.09_{\pm 1.11}$	$42.92_{\pm 0.99}$	$60.99_{\pm 0.43}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$66.99_{\pm 0.71}$	$44.11_{\pm 1.07}$	$77.10_{\pm 0.64}$	$39.24_{\pm 1.00}$	$70.46_{\pm 0.60}$	$72.10_{\pm 0.43}$	$68.76_{\pm 1.34}$	$41.67_{\pm 1.30}$	$60.06_{\pm 0.32}$

Appendix

Table 8: Accuracy on each of the eight common-sense benchmarks for Llama-2 7B, evaluated without in-context examples. In average accuracy, both SpectrumQuant variants (

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) are comparable to DartQuant and ahead of SpinQuant.

Bits	Method	Wiki	PTB	C4	Avg $\downarrow$
(W-A-KV)	Method	Wiki	PTB	C4	Avg $\downarrow$
16	Full Precision	4.88	50.94	6.73	20.85
4-4-4	QuaRot	$5.49_{\pm 0.01}$	$60.79_{\pm 1.33}$	$7.64_{\pm 0.01}$	$24.64_{\pm 0.45}$
	SpinQuant	$5.61_{\pm 0.02}$	$61.39_{\pm 2.62}$	$7.78_{\pm 0.02}$	$24.93_{\pm 0.88}$
	DartQuant	$5.40_{\pm 0.01}$	$60.64_{\pm 1.79}$	$7.37_{\pm 0.01}$	$24.47_{\pm 0.60}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$5.54_{\pm 0.04}$	$60.48_{\pm 1.70}$	$7.58_{\pm 0.05}$	$24.53_{\pm 0.56}$

Appendix

Table 9: Perplexity of each method on WikiText2, PTB and C4 for Llama-2 13B. SpectrumQuant (

{\bm{Q}}=\bm{UB}

) attains the lowest average perplexity of all approaches, and both variants (

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) improve on QuaRot and SpinQuant.

Bits	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
(W-A-KV)	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
16	Full Precision	72.14	47.39	80.52	45.20	76.73	79.38	77.44	49.15	65.99
4-4-4	QuaRot	$70.02_{\pm 0.42}$	$45.77_{\pm 0.52}$	$78.71_{\pm 0.90}$	$43.92_{\pm 1.19}$	$72.97_{\pm 0.60}$	$76.29_{\pm 0.60}$	$73.28_{\pm 0.95}$	$45.74_{\pm 0.81}$	$63.34_{\pm 0.38}$
	SpinQuant	$69.36_{\pm 0.93}$	$45.65_{\pm 0.94}$	$78.70_{\pm 0.68}$	$43.92_{\pm 0.66}$	$72.98_{\pm 1.25}$	$76.05_{\pm 0.38}$	$72.97_{\pm 1.24}$	$45.90_{\pm 0.45}$	$63.19_{\pm 0.50}$
	DartQuant	$70.40_{\pm 1.09}$	$45.92_{\pm 0.21}$	$79.05_{\pm 0.56}$	$44.48_{\pm 1.58}$	$75.28_{\pm 0.59}$	$77.09_{\pm 0.24}$	$74.66_{\pm 1.23}$	$47.48_{\pm 1.20}$	$64.30_{\pm 0.14}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$70.48_{\pm 1.07}$	$45.05_{\pm 0.54}$	$79.09_{\pm 0.46}$	$43.64_{\pm 1.00}$	$74.77_{\pm 1.32}$	$76.46_{\pm 0.49}$	$74.16_{\pm 1.45}$	$46.01_{\pm 1.15}$	$63.71_{\pm 0.58}$

Appendix

Table 10: Accuracy on each of the eight common-sense benchmarks for Llama-2 13B, evaluated without in-context examples. SpectrumQuant (with

{\bm{Q}}=\bm{UB}

) matches DartQuant in average accuracy within statistical error, without any iterative optimization and at a small fraction of its calibration memory.

Bits	Method	Wiki	PTB	C4	Avg $\downarrow$
(W-A-KV)	Method	Wiki	PTB	C4	Avg $\downarrow$
16	Full Precision	6.14	11.18	9.45	8.92
4-4-4	QuaRot	$8.29_{\pm 0.09}$	$14.48_{\pm 0.21}$	$13.54_{\pm 0.10}$	$12.10_{\pm 0.13}$
	SpinQuant	$8.12_{\pm 0.10}$	$13.69_{\pm 0.09}$	$13.19_{\pm 0.14}$	$11.67_{\pm 0.09}$
	DartQuant	$7.52_{\pm 0.03}$	$13.17_{\pm 0.07}$	$12.16_{\pm 0.06}$	$10.95_{\pm 0.05}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$7.66_{\pm 0.06}$	$12.96_{\pm 0.15}$	$12.34_{\pm 0.14}$	$10.99_{\pm 0.12}$

Appendix

Table 11: Perplexity of each method on WikiText2, PTB and C4 for Llama-3 8B. Both SpectrumQuant variants (with

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) match DartQuant in average perplexity within statistical error and improve on QuaRot and SpinQuant.

Bits	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
(W-A-KV)	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
16	Full Precision	73.32	47.19	80.96	45.00	75.63	79.14	77.82	53.24	66.54
4-4-4	QuaRot	$67.01_{\pm 0.90}$	$43.88_{\pm 0.76}$	$74.94_{\pm 0.49}$	$40.12_{\pm 0.59}$	$66.11_{\pm 1.16}$	$73.03_{\pm 0.48}$	$68.86_{\pm 2.69}$	$43.60_{\pm 1.72}$	$59.70_{\pm 0.61}$
	SpinQuant	$68.59_{\pm 1.14}$	$44.65_{\pm 0.89}$	$75.88_{\pm 1.38}$	$41.04_{\pm 0.71}$	$68.57_{\pm 1.83}$	$73.59_{\pm 0.72}$	$72.41_{\pm 1.18}$	$46.18_{\pm 1.19}$	$61.36_{\pm 0.44}$
	DartQuant	$69.41_{\pm 0.98}$	$45.17_{\pm 0.57}$	$76.97_{\pm 0.42}$	$42.84_{\pm 1.27}$	$70.00_{\pm 0.62}$	$75.38_{\pm 0.42}$	$73.93_{\pm 0.58}$	$48.21_{\pm 0.34}$	$62.74_{\pm 0.35}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$70.10_{\pm 0.84}$	$45.54_{\pm 1.04}$	$76.22_{\pm 1.09}$	$41.68_{\pm 0.72}$	$71.21_{\pm 1.31}$	$75.26_{\pm 0.47}$	$73.07_{\pm 1.22}$	$47.44_{\pm 0.89}$	$62.57_{\pm 0.50}$

Appendix

Table 12: Accuracy on each of the eight common-sense benchmarks for Llama-3 8B, evaluated without in-context examples. Both SpectrumQuant variants (with

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) match DartQuant in average accuracy within statistical error and exceed SpinQuant on every individual benchmark, while requiring far less calibration memory.

Bits	Method	Wiki	PTB	C4	Avg $\downarrow$
(W-A-KV)	Method	Wiki	PTB	C4	Avg $\downarrow$
16	Full Precision	5.32	37.34	8.48	17.04
4-4-4	QuaRot	$5.82_{\pm 0.02}$	$45.27_{\pm 1.49}$	$9.25_{\pm 0.01}$	$20.11_{\pm 0.42}$
	SpinQuant	$5.76_{\pm 0.01}$	$43.53_{\pm 0.77}$	$9.20_{\pm 0.02}$	$19.50_{\pm 0.26}$
	DartQuant	$5.67_{\pm 0.01}$	$41.50_{\pm 0.90}$	$9.02_{\pm 0.00}$	$18.73_{\pm 0.30}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$5.67_{\pm 0.01}$	$41.64_{\pm 0.50}$	$9.00_{\pm 0.00}$	$18.77_{\pm 0.17}$

Appendix

Table 13: Perplexity of each method on WikiText2, PTB and C4 for Mistral 7B v0.3. SpectrumQuant (with

{\bm{Q}}=\bm{UH}

) matches DartQuant within statistical error on all three datasets, showing that the closed-form alignment carries over to a model family beyond Llama.

Bits	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
(W-A-KV)	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
16	Full Precision	74.74	47.65	82.05	47.60	75.00	80.67	80.01	54.18	67.74
4-4-4	QuaRot	$70.56_{\pm 0.56}$	$45.24_{\pm 0.59}$	$79.75_{\pm 0.50}$	$44.20_{\pm 0.76}$	$71.96_{\pm 0.32}$	$77.98_{\pm 0.34}$	$76.03_{\pm 0.40}$	$49.23_{\pm 0.92}$	$64.37_{\pm 0.21}$
	SpinQuant	$72.04_{\pm 0.73}$	$45.66_{\pm 1.02}$	$80.13_{\pm 0.45}$	$44.68_{\pm 1.12}$	$73.60_{\pm 0.36}$	$78.31_{\pm 0.42}$	$76.79_{\pm 0.67}$	$50.21_{\pm 1.25}$	$65.18_{\pm 0.18}$
	DartQuant	$71.63_{\pm 0.67}$	$45.64_{\pm 0.50}$	$80.60_{\pm 0.70}$	$45.88_{\pm 1.14}$	$72.90_{\pm 0.28}$	$78.76_{\pm 0.28}$	$77.08_{\pm 0.36}$	$50.26_{\pm 0.89}$	$65.34_{\pm 0.28}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$72.63_{\pm 0.88}$	$46.03_{\pm 0.20}$	$80.91_{\pm 0.37}$	$44.36_{\pm 1.28}$	$73.32_{\pm 1.11}$	$78.68_{\pm 0.44}$	$77.02_{\pm 0.54}$	$50.66_{\pm 0.80}$	$65.45_{\pm 0.34}$

Appendix

Table 14: Accuracy on each of the eight common-sense benchmarks for Mistral 7B v0.3, evaluated without in-context examples. SpectrumQuant (with

{\bm{Q}}=\bm{UH}

) achieves the highest average accuracy of all approaches, statistically tied with DartQuant, further supporting generalization across architectures.

Bits	Method	Wiki	PTB	C4	Avg $\downarrow$
(W-A-KV)	Method	Wiki	PTB	C4	Avg $\downarrow$
16	Full Precision	3.32	24.25	5.71	11.09
4-4-4	QuaRot	$3.802_{\pm 0.008}$	$25.10_{\pm 0.44}$	$6.126_{\pm 0.008}$	$11.68_{\pm 0.14}$
	SpinQuant	$3.739_{\pm 0.016}$	$26.92_{\pm 0.39}$	$6.107_{\pm 0.026}$	$12.26_{\pm 0.14}$
	DartQuant	$3.602_{\pm 0.004}$	$26.39_{\pm 0.24}$	$6.002_{\pm 0.007}$	$12.00_{\pm 0.08}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$3.639_{\pm 0.035}$	$26.34_{\pm 0.34}$	$6.019_{\pm 0.013}$	$12.00_{\pm 0.11}$

Appendix

Table 15: Perplexity of each method on WikiText2, PTB and C4 for Llama-2 70B. At this scale, both SpectrumQuant variants (with

{\bm{Q}}=\bm{UH}

and

{\bm{Q}}=\bm{UB}

) match or improve on DartQuant in average perplexity, while using less than one-sixth of its calibration memory and finishing calibration in under 43 minutes on one GPU with pre-saved activations.

Bits	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
(W-A-KV)	Method	WG	SIQA	PIQA	OBQA	LAMB	HS	ARC-E	ARC-C	Avg $\uparrow$
16	Full Precision	77.98	49.13	82.75	48.80	79.58	83.80	81.02	57.51	70.07
4-4-4	QuaRot	$77.38_{\pm 0.57}$	$47.94_{\pm 0.57}$	$81.70_{\pm 0.48}$	$47.76_{\pm 0.62}$	$78.65_{\pm 0.56}$	$82.30_{\pm 0.23}$	$78.88_{\pm 0.82}$	$54.66_{\pm 0.68}$	$68.66_{\pm 0.36}$
	SpinQuant	$77.58_{\pm 0.48}$	$48.55_{\pm 0.88}$	$82.01_{\pm 0.49}$	$46.47_{\pm 0.81}$	$78.70_{\pm 0.25}$	$82.54_{\pm 0.24}$	$78.87_{\pm 0.99}$	$53.90_{\pm 1.33}$	$68.58_{\pm 0.50}$
	DartQuant	$77.35_{\pm 0.81}$	$48.37_{\pm 0.46}$	$81.70_{\pm 0.42}$	$47.76_{\pm 0.93}$	$78.96_{\pm 0.53}$	$83.07_{\pm 0.12}$	$78.68_{\pm 0.86}$	$55.05_{\pm 1.17}$	$68.87_{\pm 0.42}$
	SpectrumQuant ( $\bm{Q}=\bm{UH}$ )	$78.15_{\pm 0.86}$	$48.43_{\pm 0.83}$	$81.65_{\pm 0.63}$	$47.04_{\pm 1.37}$	$78.91_{\pm 0.55}$	$82.64_{\pm 0.19}$	$79.36_{\pm 0.92}$	$55.32_{\pm 2.09}$	$68.94_{\pm 0.54}$

Appendix

Table 16: Accuracy on each of the eight common-sense benchmarks for Llama-2 70B, evaluated without in-context examples. SpectrumQuant (with

{\bm{Q}}=\bm{UH}

) obtains the highest average accuracy of all approaches, statistically tied with DartQuant, with calibration completed on a single GPU in under 43 minutes with pre-saved activations.

Panel of Figure 7 — Figure 7: Comparison of sorted per-channel MSE across hidden dimensions for different projection activations in Llama-2 7B at layers 2,5,8,15,21,27,30 from the top to the bottom.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 381 papers

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
Oct 19, 2026Desheng Xu, Hanglin Li, Tianchen Song +2LLM Quantization Post-Training Quantization
Li Auto Inc.
Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization
May 24, 2026Gorgi PavlovLLM Quantization Rotation-Based Quantization
Lehigh University
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
May 11, 2026Chayne Thrash, Ali Abbasi, Soheil KolouriRotation-Based Quantization LLM Inference Acceleration
Department of Computer Science Vanderbilt University

Oct 19, 2026J/K move · Enter open · S save

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

Desheng Xu, Hanglin Li, Tianchen Song, Zhiyong Zhao+1

Li Auto Inc.

LLM Quantization Post-Training Quantization Low-Bit Quantization Rotation-Based Quantization Activation Quantization LLM Inference Acceleration

Open paper AskPDF arXiv

Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to

2.3\times

prefill speedup and

3.5\times

memory saving over FP16.

Abstract

LLM Quantization

Explore similar work

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization

ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs