Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization

Ian W. Kennedy; Nafise Sadat Moosavi

cs.CLApr 9, 2026

Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization

Authors: Ian W. Kennedy, Nafise Sadat Moosavi

Organizations: Department of Computer Science, University of Sheffield, Sheffield, UK

Abstract

Additive quantization enables extreme LLM compression with O(1) lookup-table dequantization, making it attractive for edge deployment. Yet at 2-bit precision, it often fails catastrophically, even with extensive search and finetuning. We show that the dominant bottleneck is codebook initialisation. Greedy sequential initialisation frequently places the model in poor optimisation regions that subsequent beam search and PV-tuning struggle to overcome. We analyse this behaviour through the representational ratio \r{ho} = N/KM, which characterises the relationship between weight groups and codebook capacity, and propose OA-EM, an output-aware EM initialisation method using Hessian-weighted Mahalanobis distance. Across compression rates, search budgets, and three architectures (Llama 3.2 3B, Llama 3.1 8B, Qwen 2.5 3B), OA-EM consistently produces better solutions after PV-tuning and dominates the quality-compute frontier. The severity of the bottleneck scales with \r{ho}: moderate at 3 bpp but extreme at 2 bpp, where poor initialisation can degrade perplexity by orders of magnitude. More broadly, our results highlight the importance of optimisation geometry in compressed model spaces, where initialisation can dominate subsequent search and fine-tuning.

Figures & tables

Rate	$M$	$K^{M}$	$\rho$	Regime
3 bpp	3	$16.8\text{M}$	${\approx}\,0.07$	Overcomplete
2 bpp	2	$65\text{K}$	${\approx}\,18$	Undercomplete

Table 1

	Wiki-2	C4	Time
FP16	$7.28$	$11.04$	—
Greedy initialisation
$b{=}4,\ e{=}100$	$352.39$	$20.66$	6.1h
$b{=}8,\ e{=}100$	$60.61$	$18.64$	9.9h
$b{=}16,\ e{=}100$	$46.01$	$19.00$	16.9h
$b{=}8,\ e{=}5$	$85.72$	$47.04$	6.8h

Table 1: Pre-PV-tuning results at 2 bpp on Llama 3.2 3B. Beam width

b

, maximum epochs

e

. Quantization time on one A100.

Init	Config	Pre-PV	Post-PV	$\Delta$	Q-time
Narrow beam ( $b{=}4$ , $e{=}100$ )
Greedy		352.39	12.66	$-$ 339.73	6.1h
OA-EM		16.82	11.53	$-$ 5.29	6.1h
Standard beam ( $b{=}8$ , $e{=}100$ )
Greedy		60.61	11.76	$-$ 48.85	9.9h
OA-EM		17.39	11.53	$-$ 5.86	9.2h

Table 2: WikiText-2 perplexity before and after PV-tuning at 2 bpp on Llama 3.2 3B.

		Avg Acc $\uparrow$
Model	Config	Grdy	OA-EM
Llama 3B	$b{=}4$	.573	.589
Llama 3B	$b{=}8$	.585	.591
Llama 3B	$b{=}16$	.585	.591
Llama 3B	$b{=}8,e{=}5$	.563	.572
Llama 8B	$b{=}8$	.649	.656

Table 3: Post-PV-tuning downstream summary at 2 bpp. Avg = mean of 6 zero-shot accuracy tasks. Full per-task tables in Appendix B .

		Pre-PV		Post-PV
Model	Rate	Grdy	OA-EM	Grdy	OA-EM
Llama 3B	3 bpp	9.52	8.87	8.66	8.54
Llama 3B	2 bpp	60.61	17.39	11.76	11.53
Llama 8B	2 bpp	18.86	16.38	9.39	9.25
Qwen 3B	2 bpp	12.50	12.30	10.93	10.73

Table 4: Basin persistence across compression rates and architectures (

b{=}8

e{=}100

). WikiText-2 perplexity before and after PV-tuning.

Benchmark	Domain	Bsln	OA-EM	Ratio
C4	In-domain	18.64	18.00	1.04 $\times$
LAMBADA	Near-OOD	12.28	8.85	1.39 $\times$
WikiText-2	Far-OOD	60.61	17.39	3.49 $\times$

Table 5: Degradation scales with domain distance at 2 bpp (

b{=}8

, pre-PV-tuning). Ratio = baseline/OA-EM.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

	Wiki-2 $\downarrow$	C4 $\downarrow$
FP16	7.28	11.04
Baseline	9.52	13.39
+ OA-EM	8.87	13.51

Appendix

Table 6: Pre-PV-tuning perplexity at 3 bpp on Llama 3.2 3B (

b{=}8

e{=}100

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.418	.658	.704	.670	4.87	.760	.669	.647
OA-EM	.421	.693	.704	.685	4.64	.762	.657	.654

Appendix

Table 7: Post-PV-tuning downstream at 3 bpp on Llama 3.2 3B (

b{=}8

e{=}100

). Avg = mean of 6 accuracy tasks.

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.350	.560	.620	.577	7.65	.734	.594	.573
OA-EM	.359	.614	.625	.584	7.13	.734	.619	.589

Appendix

Table 8: Post-PV downstream, Llama 3.2 3B, 2 bpp,

b{=}4

e{=}100

. OA-EM wins or ties every metric.

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.356	.600	.623	.574	7.96	.741	.614	.585
OA-EM	.366	.601	.626	.604	6.95	.736	.611	.591

Appendix

Table 9: Post-PV downstream, Llama 3.2 3B, 2 bpp,

b{=}8

e{=}100

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.364	.619	.625	.579	7.52	.731	.589	.585
OA-EM	.357	.624	.627	.587	7.66	.726	.624	.591

Appendix

Table 10: Post-PV downstream, Llama 3.2 3B, 2 bpp,

b{=}16

e{=}100

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.332	.569	.611	.556	8.61	.716	.594	.563
OA-EM	.331	.561	.625	.574	7.73	.734	.609	.572

Appendix

Table 11: Post-PV downstream, Llama 3.2 3B, 2 bpp,

b{=}8

e{=}5

(early stopping).

	Before PV		After PV
Init	Wiki-2	C4	Wiki-2	C4
Greedy	18.86	15.01	9.39	12.02
OA-EM	16.38	14.50	9.25	11.89

Appendix

Table 12: WikiText-2 / C4 perplexity before and after PV-tuning at 2 bpp on Llama 3.1 8B (

b{=}8

e{=}100

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.432	.656	.707	.679	4.60	.759	.661	.649
OA-EM	.424	.677	.714	.675	4.59	.769	.679	.656

Appendix

Table 13: Post-PV downstream, Llama 3.1 8B, 2 bpp,

b{=}8

e{=}100

	Before PV		After PV
Init	Wiki-2	C4	Wiki-2	C4
Greedy	12.50	16.01	10.93	14.57
OA-EM	12.30	16.08	10.73	14.49

Appendix

Table 14: WikiText-2 / C4 perplexity before and after PV-tuning at 2 bpp on Qwen 2.5 3B (

b{=}8

e{=}100

	ARC-C	ARC-E	HeSw	LAM acc	LAM ppl $\downarrow$	PIQA	WiGr	Avg $\uparrow$
Greedy	.375	.662	.626	.600	7.29	.739	.634	.606
OA-EM	.366	.651	.630	.587	7.37	.737	.647	.603

Appendix

Table 15: Post-PV downstream, Qwen 2.5 3B, 2 bpp,

b{=}8

e{=}100

Init	Config	Wiki-2 $\downarrow$	Avg $\uparrow$	Q-time
Greedy	$b{=}4$ , $e{=}100$	12.66	.573	6.1h
OA-EM	$b{=}4$ , $e{=}100$	11.53	.589	6.1h
Greedy	$b{=}8$ , $e{=}100$	11.76	.585	9.9h
OA-EM	$b{=}8$ , $e{=}100$	11.53	.591	9.2h
Greedy	$b{=}16$ , $e{=}100$	12.01	.585	16.9h
OA-EM	$b{=}16$ , $e{=}100$	11.49	.591	15.5h

Appendix

Table 16: Quality-compute Pareto analysis at 2 bpp on Llama 3.2 3B. All metrics post-PV-tuning. Avg = mean of 6 accuracy tasks.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 381 papers

BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization
Oct 3, 2026Chenhang Cui, Xu Xie, Linrui Xu +4LLM Quantization Low-Bit Quantization
National University of Singapore · Independent Researcher · Central South University
Few Bits, One Law: Toward W2A4KV2
Oct 6, 2026Kai Yi, Tarek Elgamal, Sruthikesh Surineni +3Neural Network Quantization LLM Quantization
Meta AI
PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression
Mar 30, 2026Caio VicentinoLLM Quantization Neural Network Quantization

Oct 3, 2026J/K move · Enter open · S save

BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization

Chenhang Cui, Xu Xie, Linrui Xu, Xiaohao Liu+3

National University of Singapore · Independent Researcher · Central South University

LLM Quantization Low-Bit Quantization Vector Quantization Weight-Only Quantization

Open paper AskPDF arXiv

As large language models (LLMs) grow in parameter count, model storage and parameter memory traffic have become major bottlenecks to efficient deployment. Codebook-based weight quantization reduces these costs, but imbalanced nearest-codeword assignments during fitting can leave some codewords insufficiently updated, limiting effective codebook utilization. To address this limitation, we propose Balanced Assignment Refinement for Quantization (BARQ), which improves quantization quality through balanced fitting of existing codebooks. Specifically, we first compute joint soft assignments between weight blocks and codewords through entropically regularized optimal transport with uniform marginals and curvature-weighted reconstruction costs, ensuring equal positive fitting mass for every codeword in the exact solution. We then refine the codewords through an assignment-weighted barycentric update, which we prove minimizes the fitting objective for fixed assignments. For finite Sinkhorn iterations, the implemented update retains this optimality provided all codeword masses exceed the denominator floor. Finally, we discard the soft assignments and use the refined codebook for standard hard nearest-codeword encoding, with our analysis establishing sufficient conditions for reducing hard-quantization distortion and evaluation loss. Across multiple LLMs, BARQ achieves lower perplexity and higher mean zero-shot accuracy than the evaluated baselines at comparable bit budgets. The code is available at https://github.com/chenhangcuisg-code/BARQ.

Abstract

LLM Quantization

Explore similar work

BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization

Few Bits, One Law: Toward W2A4KV2

PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression