ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation

Suyoung Kim; Sunghyun Wee; Hyeonjin Kim; Kyomin Hwang; Hyunho Lee; Nojun Kwak

cs.CVApr 13, 2026

ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation

Authors: Suyoung Kim, Sunghyun Wee, Hyeonjin Kim, Kyomin Hwang, Hyunho Lee, Nojun Kwak

Organizations: Seoul National University, Seoul, Republic of Korea · LG Electronics, Seoul, Republic of Korea

Abstract

Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large Language Models (LLMs). Global rotation methods achieve inference efficiency by fusing activation rotations into attention and FFN blocks, but suffer from limited expressivity as they are constrained to use a single learnable rotation matrix across all layers. To tackle this, layer-wise transformation methods emerged, achieving superior accuracy through localized adaptation. However, layer-wise methods cannot fuse activation rotation matrices into weights, requiring online computations and causing significant overhead. In this paper, we propose ReSpinQuant, a quantization framework that resolves such overhead by leveraging offline activation rotation fusion and matching basis using efficient residual subspace rotation. This design reconciles the high expressivity of layer-wise adaptation with only negligible inference overhead. Extensive experiments on W4A4 and W3A3 quantization demonstrate that ReSpinQuant achieves state-of-the-art performance, outperforming global rotation methods and matching the accuracy of computationally expensive layer-wise methods with minimal overhead.

Figures & tables

Figure 1: Comparison of rotation paradigms. (a) Global Rotation ensures efficiency via offline weight merging but limits expressivity. (b) Layer-wise Transformation enhances expressivity but incurs high online overhead. (c) ReSpinQuant (Ours) reconciles this trade-off by merging layer-wise rotation into weights, and resolves the basis mismatch problem via subspace residual rotation approximation, achieving both high accuracy and efficiency.

Figure 2: Overview of the ReSpinQuant architecture. Unlike global rotation methods that enforce a shared basis, ReSpinQuant assigns distinct learnable orthogonal matrices (

\mathbf{R}_{1}^{i},\mathbf{R}_{2}^{i}

, etc.) to each layer

i

to maximize outlier suppression. (1) Offline Weight Fusion: To ensure offline weight integration, rotation matrices are mathematically absorbed into the weight matrices at inference phase ( dashed red boxes , “Merged and Quantized Weights”), leaving the main computation unchanged. (2) Subspace Residual Approximation: The resulting basis mismatch in residual connections is resolved using our proposed Subspace Residual Rotation Approximation ( dash-dot blue boxes , “Merge and Approximate”). This replaces the expensive dense basis transition with a low-rank correction, reducing alignment complexity from

\mathcal{O}(D^{2})

\mathcal{O}(D)

while maintaining high model expressivity.

Figure 3: Visualization of the trained rotation matrices optimized by Cayley optimizer. We display the sub-blocks [

{:}4,{:}4

] of two rotation matrices (

\mathbf{R}_{1},\mathbf{R}_{2}\in\mathbb{R}^{4096\times 4096}

) of Llama-3 8B layer 16, and their relative transformation (

\mathbf{R}_{1}^{\top}\mathbf{R}_{2}

) after training. Despite optimization, the matrices deviate minimally from the Hadamard initialization. Crucially, the relative transformation

\mathbf{R}_{1}^{\top}\mathbf{R}_{2}

remains sparse and diagonally dominant, motivating the use of our residual subspace approximation.

Figure 4: Optimization dynamics of rotation matrices on LLaMA-3 8B (Layers 2, 16). (a) The Frobenius norm of the deviation from the initialization, calculated as $\|\mathbf{R}-\mathbf{R}_{\text{init}}\|_{F}$ . The curve indicates stable convergence during training. (b) The cosine similarity between the optimized rotation $\mathbf{R}$ and the initialization $\mathbf{R}_{\text{init}}$ remains consistently high. Together, these plots confirm that while the learned rotations successfully converge to an optimal state, they do not deviate significantly from their initial values.

Figure 5: Illustration of the subspace residual rotation approximation. We approximate the dense rotation matrix (

\mathbf{T}\in\mathbb{R}^{D\times D}

) that aligns the residual bases into a low-dimensional subspace to minimize the computational overhead. During inference, the input features are projected into a subspace of rank

r

via

\mathbf{Q}

, rotated by the approximated matrix (

\hat{\mathbf{R}}_{\text{sub}}\in\mathbb{R}^{r\times r}

), and re-projected via

\mathbf{Q}^{\top}

. The orthogonal complement passes through the identity path (

\mathbf{I}-\mathbf{Q}\mathbf{Q}^{\top}

). The alignment complexity is reduced from quadratic

\mathcal{O}(D^{2})

to linear

\mathcal{O}(D)

Bits	Method	LLaMA-2 7B		LLaMA-2 13B		LLaMA-3 8B		LLaMA-3.2 1B		LLaMA-3.2 3B
Bits	Method	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$
16-16	FP16	5.47	65.08	4.88	67.77	6.13	68.05	9.76	55.75	7.81	63.44
W4A4	RTN	1753.70	33.69	3575.44	31.24	219.82	36.74	330.79	33.52	266.80	35.15
	GPTQ	9787.38	34.89	2841.03	31.46	187.49	35.74	223.08	33.81	168.41	36.51
	QuaRot	6.12	62.03	5.37	65.25	7.82	62.90	14.59	46.81	9.99	57.22
	SpinQuant	5.98	61.60	5.30	65.58	7.50	64.53	13.52	48.35	9.46	56.78

Table 1: Main results on LLaMA-2, LLaMA-3, and LLaMA-3.2 families. We report Perplexity (PPL) on WikiText-2 and Zero-shot Accuracy (Avg.). ReSpinQuant consistently outperforms the global rotation baseline (SpinQuant) and matches or exceeds the layer-wise baseline across diverse model scales.

Method	Transform Scope	Learnable Parameters			Additional Computation
Method	Transform Scope	Parameter Space	Trainable Params	Online Params	Complexity	MACs
SpinQuant ( Liu et al., 2025 )	Global	$\mathcal{O}(D^{2})$	17.3M	$-$	${\mathcal{O}(D\log D)}$	15.4M
OSTQuant ( Hu et al., 2025 )	Layer-wise	$\mathcal{O}(D^{2}+L\cdot D)$	21.7M	$-$	${\mathcal{O}(D\log D)}$	15.4M
FlatQuant ( Sun et al., 2025 )	Layer-wise	$\mathcal{O}(L\cdot D)$	5.8M	5.8M	${\mathcal{O}(D^{1.5}})$	198.1M
ReSpinQuant (Ours)	Layer-wise	${\mathcal{O}(L\cdot D^{2})}$	1091.0M	8.4M	${\mathcal{O}(D\log D+rD)}$	32.3M

Table 2: Comparison of learnable parameters (training vs. online) and multiply-accumulate operations (MACs) for rotation-based quantization methods on Llama3-8B model. ReSpinQuant maximizes learnable parameters during training (

\mathcal{O}(L\cdot D^{2})

) for high expressivity, but most are fused into weights, leaving only low-rank subspace residual rotation parameters for online inference.

Batch Size	SpinQuant		ReSpinQuant
Batch Size	TTFT (ms)	TTIT (ms)	TTFT (ms)	TTIT (ms)
1	389.20	120.38	394.75	123.45
2	757.18	122.29	766.03	126.90
4	1,453.42	126.42	1,473.54	129.53
8	2,774.59	141.23	2,886.14	144.80
16	5,589.04	160.95	5,661.14	163.81

Table 3: End-to-end latency comparison on LLaMA-3 8B (W4A4). Measured on an NVIDIA H100 GPU with a sequence length of

T=2048

. We report Time to First Token (TTFT) which measures the prefill latency, and Time to Iterative Token (TTIT) representing the decoding latency. ReSpinQuant (rank

r=32

) demonstrates comparable inference speed to the globally-rotated SpinQuant baseline across all batch sizes.

Method	Training Time (Minute)
Method	LLaMA-2 7B	LLaMA-2 13B	LLaMA-3 8B
SpinQuant	17m	23m	17m
OSTQuant	15m	26m	12m
FlatQuant	45m	74m	45m
ReSpinQuant	43m	61m	42m

Table 4: Comparison of calibration overhead measured on a single NVIDIA H100 GPU. Although a larger parameter space leads to increased training time, ReSpinQuant still has a practical calibration time, finishing within an hour for most models.

Rank $r$	PPL $\downarrow$	Zero-shot Avg. $\uparrow$
0 (Identity)	20.03	46.94
8	14.20	49.77
16	13.59	50.14
32	13.09	50.74
64	12.84	51.29
128	12.80	50.80

Table 5: Ablation study on approximation rank

r

using LLaMA-3-8B (W3A3). Performance consistently improves as the rank increases. We select

r=32

as the default setting, as it achieves a favorable trade-off between accuracy and parameter efficiency.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Rank $r$	0	8	16	32	64	128	4096
Cumulative energy of $\Delta T$ (%)	0.00	51.07	53.45	55.77	58.55	62.33	100.00
Retained Frobenius magnitude (%)	0.00	70.25	72.09	73.80	75.76	78.34	100.00

Appendix

Table 6: Spectral concentration of the residual transition deviation

\Delta T=T-I

on LLaMA-2 7B under W3A3 quantization.

Setting	Method	LLaMA-2 7B		LLaMA-2 13B		LLaMA-3 8B		LLaMA-3.2 1B		LLaMA-3.2 3B
		PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$
FP16	Full precision	5.47	65.08	4.88	67.77	6.13	68.05	9.76	55.75	7.81	63.44
W4A4KV16	SpinQuant	5.93	61.98	5.26	65.43	7.38	64.43	13.03	48.98	9.15	57.55
W4A4KV16	ReSpinQuant	5.69	62.49	5.08	65.63	7.16	65.41	12.36	49.44	8.89	59.49
W4A8KV8	SpinQuant	5.62	64.07	5.00	66.85	6.53	67.28	10.50	53.39	8.24	62.24
W4A8KV8	ReSpinQuant	5.60	64.24	4.98	67.10	6.53	67.32	10.51	53.28	8.22	62.46

Appendix

Table 7: Additional bit-width results on LLaMA-family models. We report PPL and zero-shot accuracy averaged over nine tasks.

Method	LLaMA-2 7B		LLaMA-2 13B		LLaMA-3 8B		LLaMA-3.2 1B		LLaMA-3.2 3B
	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$	PPL $\downarrow$	0-shot 9 $\uparrow$
SpinQuant	8.07	61.67	7.39	65.46	11.88	63.79	20.33	49.11	14.39	58.01
ReSpinQuant	7.70	62.40	7.29	66.06	11.50	64.67	19.82	49.48	13.99	58.64

Appendix

Table 8: Calibration dataset sensitivity under W4A4KV4. We use C4 as the calibration corpus and report PPL and nine-task zero-shot average accuracy.

Model	Method	BoolQ	PIQA	SIQA	HellaS.	WinoG.	ARC-e	ARC-c	OBQA	LAMB	Avg. ( $\uparrow$ )	PPL ( $\downarrow$ )
llama2-7b	Full precision	77.77	79.05	45.96	76.00	68.51	74.54	46.16	44.00	73.70	65.08	5.47
	RTN	55.63	51.80	34.54	30.01	48.70	27.40	26.11	25.00	3.98	33.69	1753.70
	GPTQ	54.95	54.13	35.88	31.60	49.88	32.66	25.26	25.60	4.08	34.89	9787.38
	QuaRot	74.13	77.09	43.50	73.27	66.85	70.24	43.52	39.60	70.10	62.03	6.12
	Spinquant	73.03	76.44	42.99	72.87	64.40	70.29	42.58	41.40	70.39	61.60	5.98
	OSTQuant	74.31	76.88	44.27	73.78	66.85	69.23	41.72	42.80	71.05	62.32	5.95

Appendix

Table 9: Detailed W4A4 quantization results across benchmarks.

Model	Method	BoolQ	PIQA	SIQA	HellaS.	WinoG.	ARC-e	ARC-c	OBQA	LAMB	Avg. ( $\uparrow$ )	PPL ( $\downarrow$ )
llama2-7b	Full precision	77.77	79.05	45.96	76.00	68.51	74.54	46.16	44.00	73.70	65.08	5.47
	RTN	54.74	49.78	34.75	26.39	47.12	25.59	29.10	25.60	0.00	32.56	19519.63
	GPTQ	61.87	48.26	34.08	26.10	50.51	25.55	30.29	27.40	0.00	33.78	99691.84
	QuaRot	56.36	61.37	35.36	41.73	48.38	43.22	27.13	26.80	28.08	40.94	19.26
	Spinquant	62.60	67.79	38.43	56.99	54.78	50.84	32.51	32.00	46.42	49.15	9.69
	OSTQuant	60.12	68.88	38.54	56.14	55.56	52.23	29.95	30.80	43.78	48.44	9.53

Appendix

Table 10: Detailed W3A3 quantization results across benchmarks.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 362 papers

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models
Oct 19, 2026Desheng Xu, Hanglin Li, Tianchen Song +2LLM Quantization Post-Training Quantization
Li Auto Inc.
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
May 11, 2026Chayne Thrash, Ali Abbasi, Soheil KolouriRotation-Based Quantization LLM Inference Acceleration
Department of Computer Science Vanderbilt University
ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs
Sep 28, 2026Mehdi Makni, Ryan Lucas, Rahul MazumderLLM Quantization Rotation-Based Quantization
Operations Research Center Massachusetts Institute of Technology

Oct 19, 2026J/K move · Enter open · S save

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

Desheng Xu, Hanglin Li, Tianchen Song, Zhiyong Zhao+1

Li Auto Inc.

LLM Quantization Post-Training Quantization Low-Bit Quantization Rotation-Based Quantization Activation Quantization LLM Inference Acceleration

Open paper AskPDF arXiv

Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant), a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to

2.3\times

prefill speedup and

3.5\times

memory saving over FP16.

Abstract

LLM Quantization

Explore similar work

LIRQuant: An Efficient Locally Isolated Rotation Quantization Method for Large Language Models

ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs

ThinQuant: Scalable Rotation Learning for Weight and Activation Quantization of LLMs