SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

Gunjun Lee; Sehwan Son; Younjoo Lee; Byungjun Kim; Jung Ho Ahn

cs.LGAug 16, 2026

SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

Authors: Gunjun Lee, Sehwan Son, Younjoo Lee, Byungjun Kim, Jung Ho Ahn

Organizations: Seoul National University Seoul, South Korea

Abstract

Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backpropagation-free PTQ optimizers have two limitations: group decisions ignore the correction that the remaining continuous suffix can absorb, and discrete refinements typically keep the affine quantization grid fixed. We introduce SCHUROPT, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature. It then alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes. With the GPTQ objective fixed, SCHUROPT improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points (pp). At higher precision, however, tighter reconstruction does not consistently improve end-model metrics. SCHURQUANT therefore combines SCHUROPT with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and teacher-decision token weighting. Across eight Llama and Qwen models, SCHURQUANT achieves the highest mean zero-shot accuracy among the evaluated backpropagation free PTQ baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.

Figures & tables

		2 bits				3 bits				4 bits
Model	FP16	SchQ	GPTQ	FOEM	QEP	SchQ	GPTQ	FOEM	QEP	SchQ	GPTQ	FOEM	QEP
Llama2-7B	5.46	10.54	37.63	68.73	29.84	6.07	6.32	6.38	6.39	5.59	5.62	5.64	5.71
Llama2-13B	4.88	8.28	16.69	15.42	13.50	5.33	5.42	5.43	5.43	4.99	4.98	4.97	4.98
Llama3.2-1B	9.76	79.19	3432.88	6034.37	1248.04	13.25	15.41	68.22	15.26	10.42	10.63	43.25	10.70
Llama3.2-3B	7.82	29.39	10549.22	4331.26	378.70	9.39	12.07	253.99	18.05	8.12	8.24	17.33	8.51
Llama3-8B	6.14	36.69	486.51	554.52	221.76	7.55	8.31	15.72	9.79	6.46	6.57	11.04	6.64

Table 1: WikiText-2 test perplexity (lower is better). FP16 is the full-precision reference and SchQ denotes SchurQuant . For each model and bit-width, the best quantized result is bold.

		2 bits				3 bits				4 bits
Model	FP16	SchQ	GPTQ	FOEM	QEP	SchQ	GPTQ	FOEM	QEP	SchQ	GPTQ	FOEM	QEP
Llama2-7B	70.45	56.58	41.38	39.47	43.73	68.13	67.03	67.04	67.53	69.54	70.07	69.89	69.93
Llama2-13B	73.29	62.55	48.33	40.35	50.14	71.59	71.84	71.08	71.63	73.11	73.06	73.04	73.09
Llama3.2-1B	60.35	45.05	37.60	36.45	37.07	55.49	53.94	52.07	53.51	59.28	59.19	57.88	58.56
Llama3.2-3B	69.04	47.90	36.72	36.90	41.11	65.98	63.05	60.54	60.56	68.04	68.11	66.52	67.81
Llama3-8B	74.61	48.53	38.04	37.20	37.73	72.20	66.15	66.72	68.22	74.27	72.94	73.03	73.27

Table 2: Mean zero-shot accuracy (%; higher is better) across the six evaluated tasks. For each model and bit-width, the best quantized result is bold.

Curvature	Grid	$\Delta\mathcal{L}$ (%) $\downarrow$	PPL $\downarrow$
Raw	fixed	1.645	2344.82
Raw	refit	0.872	152.95
Schur	fixed	1.165	324.81
Schur	refit	0.550	86.64

Table 3: Controlled optimizer ablation on Qwen3-4B at 2 bits.

\Delta\mathcal{L}

is the normalized layer-0 Q/K/V local-loss increase in percent.

Optimizer and objective	PPL $\downarrow$	Mean $\uparrow$	PPL $\downarrow$	Mean $\uparrow$	PPL $\downarrow$	Mean $\uparrow$
	2 bits		3 bits		4 bits
GPTQ optimizer + GPTQ objective	258.96	36.77	15.79	65.53	14.16	69.50
SchurOpt + GPTQ objective	81.95	48.65	21.26	65.41	15.52	69.58
SchurOpt + teacher, $\lambda=1$	75.01	42.97	16.23	65.13	14.29	69.06
SchurOpt + teacher + weights, $\lambda=0$	64.81	43.62	18.92	63.77	15.69	69.02
SchurOpt + teacher + weights, $\lambda=1$	67.70	46.20	16.07	66.83	14.31	70.00

Table 4: Objective ablation on Qwen3-4B. Mean is six-task zero-shot accuracy (%).

$R$	0	1	2	4	8	16	32
PPL $\downarrow$	70.46	37.93	28.75	20.54	21.41	21.19	21.21

Table 5: WikiText-2 perplexity versus refinement count

R

for 2-bit Llama2-7B under the fixed GPTQ objective.

R=0

uses only min–max initialization.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Method	2 bit	3 bit	4 bit
Qwen3-4B	GPTQ	5.5	6.9	5.5
	FOEM	6.4	7.6	6.6
	QEP	55.6	71.2	69.4
	SchurQuant	253.7	254.0	282.5
Llama2-7B	GPTQ	7.1	8.4	7.1
	FOEM	8.2	9.1	8.0

Appendix

Table 6: End-to-end quantization time in minutes on one NVIDIA H100 PCIe GPU.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 386 papers

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
Aug 7, 2026Yongge Ma, Guoan Wang, Feiyu Wang +5LLM Quantization Post-Training Quantization
School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University +2
SliderQuant: Accurate Post-Training Quantization for LLMs
Mar 26, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +3LLM Quantization Post-Training Quantization
Intel Labs China · BUPT
G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
Sep 25, 2026Ruikang Liu, Haoli Bai, Yuxuan Sun +8LLM Quantization Post-Training Quantization
ZTE Corporation · The Chinese University of Hong Kong · Northwestern Polytechnical University +2

Aug 7, 2026J/K move · Enter open · S save

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

Yongge Ma, Guoan Wang, Feiyu Wang, Yaoming Li+4

School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University+2

LLM Quantization Post-Training Quantization Neural Network Quantization Quantization

Open paper AskPDF arXiv

Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models. Existing PTQ methods typically obtain an initial quantized model through heuristic rules or greedy optimization, and once quantization is completed the resulting integer assignments are usually treated as final. This observation motivates a complementary optimization stage within PTQ that keeps quantized weights improvable after an executable quantized model has been produced, while preserving the quantized format. We introduce ReQuant, a backpropagation-free fixed-grid refinement procedure for this stage. Agnostic to the PTQ initializer, ReQuant takes an existing quantized model as a feasible starting point and iteratively revisits its discrete weight assignments on the fixed quantization grid. Accepted updates strictly reduce the mean squared reconstruction error and remain on the original grid. In this way, ReQuant turns the initially fixed PTQ output into an iteratively optimizable discrete solution and serves as a plug-and-play post-processing stage for existing PTQ pipelines. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers, with especially large gains on simple initializers and lower bit-widths. Notably, ReQuant can refine a simple round-to-nearest initialization across multiple sweeps until it approaches or surpasses GPTAQ under the same quantization format. These results establish ReQuant as a practical complementary stage for further improving existing PTQ pipelines.

Abstract

LLM Quantization

Explore similar work

ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

SliderQuant: Accurate Post-Training Quantization for LLMs

G2^22PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation

G $^2$ PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation