E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

Xinyu Lei; Hong Huang; Jiebin Zhang; Hongyu Li; Jinlong Song; Decheng Wu; Guanghua Yu; Jianchen Zhu

cs.AIOct 20, 2026

E8-QAT: Efficient 2-Bit Quantization-Aware Training with Implicit Lattice Codebooks

Authors: Xinyu Lei, Hong Huang, Jiebin Zhang, Hongyu Li, Jinlong Song, Decheng Wu, Guanghua Yu, Jianchen Zhu,

Organizations: Peking University · City University of Hong Kong · Tencent

Abstract

Low-bit quantization reduces the memory footprint of large language models, but preserving accuracy with efficient training remains challenging. Scalar quantization encodes weights independently, while explicit-codebook vector quantization introduces costly search during training. We propose E8-QAT, a 2-bit quantization-aware training framework that uses the $E_8$ lattice as an implicit eight-dimensional codebook. It jointly quantizes weight groups through arithmetic projection without storing or searching an explicit codebook during the training forward pass. A 2-bit core encoding and sparse extension preserve the selected lattice points exactly, adding less than 0.1% to core index storage. On Qwen3.6-35B-A3B, E8-QAT matches the performance of the full-precision model in terms of the average score across the four tasks. E8-QAT improves over the state-of-the-art baseline by 2.03% on IFEval and by 1.51% on LiveCodeBench, under the same data, schedule, and bit width. The 2-bit representation reduces peak memory by 80% and supports a $4\times$ larger batch. At each representation's largest feasible batch, E8-QAT reaches $1.97\times$ the decoding throughput of BF16.

Figures & tables

Figure 1: Training-time quantization. (a) Explicit-codebook vector quantization searches a stored codebook of

K=2^{kv}

entries. (b) LC-QAT rounds each coordinate and applies a fixed scale and shift,

\mathbf{q}=s\mathbf{z}+\mathbf{b}

. (c) E8-QAT projects eight normalized coordinates onto

E_{8}

by arithmetic, without a stored codebook.

Figure 2: Overview of E8-QAT. (a) PTQ initializes latent weights from a fixed

E_{8}

core. (b) Arithmetic lattice projection with an identity STE enables joint QAT without explicit codebook search or finite-core clipping. (c) Core indices and sparse escape records preserve the final lattice points. Hadamard transforms and reconstruction scales are omitted for clarity.

			Base (lm-eval)					Agent		IF
Type	Method	Bits	ARC-C	HelS	PIQA	WinG	Avg.	LCB	BFCL	IFEval
N/A	BF16	16	55.20	83.49	82.75	74.27	73.93	81.42	75.99	85.40
SQ	RTN	2	49.23	79.44	79.87	69.85	69.60	20.38	53.91	77.82
	GPTQ	2	52.78	79.17	80.20	69.76	70.48	20.38	55.87	78.18
	UniSVQ	2	56.83	78.31	80.58	71.59	71.83	31.28	72.32	80.41
VQ	QuIP#	2	56.66	77.25	80.20	72.77	71.72	47.77	67.31	81.70

Table 1: Results at 2 bits on Qwen3.6-35B-A3B. Bold marks the best quantized result in each column. Full results, including ARC-Easy, BoolQ, are given in Appendix C.3 .

Figure 4: Layer-wise stage-1 output distortion at matched bit width (lower is better).

Figure 5: Performance of PTQ and QAT model with the 750-step second stage. Dashed lines mark BF16.

Figure 6: Escape trajectories (a), peak-learning-rate control at 375 steps (b), and the index-and-extension payload

b=2+20\rho

(c).

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Check	Coverage	Result
Core decoding agreement	All 65,536 core codewords	Identical lattice coordinates across the tested decoding paths
Single-root membership	All 26,435 escaped groups in the 375-step artifact	Every residual belongs to $\mathcal{R}$

Appendix

Table 3: Encoding checks. Root packing holds for every escaped group in the 375-step artifact.

Setting	Value
Model	Qwen3.6-35B-A3B
Total parameters	$35{,}107{,}181{,}936$
Quantized parameters	$32{,}338{,}083{,}840$ expert FFN weights ( $92.1\%$ )
Unquantized weights	$2{,}769{,}098{,}096$ in BF16: attention projections, router, embeddings, norms
Rate, quantized weights	2.03 bits, including scales and codebooks
Latent weights	BF16

Appendix

Table 4: Recorded settings for the Qwen3.6-35B-A3B experiments.

Method	Mean distortion $\downarrow$	Best layers $\uparrow$
UniSVQ	0.0605	0/40
AQLM, layer-shared	0.0598	1/40
E8-PTQ	0.0482	39/40

Appendix

Table 5: Stage-1 reconstruction on Qwen3.6-35B-A3B. Bold marks the lowest distortion and the largest best-layer count.

Stage	Method	Bits	ARC-C	ARC-E	HellaSwag	PIQA	WinoGrande	BoolQ	Avg 6
Ref.	BF16	16	55.20	72.22	83.49	82.75	74.27	85.02	75.49
PTQ	UniSVQ	2.03	56.83	75.84	78.31	80.58	71.59	89.33	75.41
PTQ	AQLM, layer-shared	2.03	54.95	73.06	79.59	81.50	73.32	88.38	75.13
PTQ	E8-PTQ	2.03	56.74	78.41	79.38	80.47	72.61	88.78	76.06
QAT	LC-QAT	2.03	56.57	75.72	81.28	80.74	73.88	87.61	75.97
QAT	E8-QAT	2.03	57.85	79.34	82.29	81.56	74.03	86.64	76.95

Appendix

Table 6: Six-task lm_eval results on Qwen3.6-35B-A3B. Bold marks the best 2-bit score in each column.

Method	AIME25 pass@3 $\uparrow$
BF16	90.00
AQLM	83.33
LC-QAT	90.00
E8-QAT	90.00

Appendix

Table 7: AIME25 pass@3. Bold marks the highest score, including ties.

Stage	Method	ARC-C	ARC-E	HelS	PIQA	WinG	BoolQ	Avg 6	WikiText
Ref.	BF16	55.55	79.80	79.55	81.28	73.80	85.47	75.91	9.01
PTQ	UniSVQ	39.51	65.36	63.55	72.63	66.46	79.27	64.46	22.38
PTQ	E8-PTQ	43.94	70.50	66.16	74.76	68.90	81.47	67.62	20.79
QAT	LC-QAT	40.10	64.14	67.71	73.29	67.40	82.57	65.87	16.73
QAT	E8-QAT	44.62	69.49	70.03	76.17	69.61	77.43	67.89	13.77
QAT	E8-QAT, LR $10^{-4}$	35.84	52.36	58.39	69.48	59.04	72.29	57.90	19.46

Appendix

Table 8: Complete lm_eval results on Llama-3.1-8B. WikiText is perplexity (lower is better). Bold marks the best quantized result; the last row is excluded.

Training budget	Checkpoint step	Escape fraction $\rho$
375	100	$0$
375	200	$1.196\times 10^{-6}$
375	300	$5.672\times 10^{-6}$
375	375	$6.540\times 10^{-6}$
750	400	$1.618\times 10^{-5}$
750	500	$3.618\times 10^{-5}$

Appendix

Table 9: Escape fractions under the two annealing schedules. A zero is an exact zero.

Endpoint	$\rho$	$\Delta b$	$b_{\mathrm{payload}}$
375 steps, peak LR $10^{-4}$	$6.540\times 10^{-6}$	$1.308\times 10^{-4}$	$2.000131$
375 steps, peak LR $5\times 10^{-5}$	$1.608\times 10^{-7}$	$3.216\times 10^{-6}$	$2.000003$
750 steps	$8.556\times 10^{-5}$	$1.711\times 10^{-3}$	$2.001711$

Appendix

Table 10: Endpoint storage with 20-byte coordinate records, in bits per quantized weight.

LLM Quantization

Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.

See all 381 papers

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
Jun 9, 2026Haoyu Wang, Xingyu Yu, Haiyan Zhao +2LLM Quantization Quantization-Aware Training
Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · College of Computer Science and Technology, National University of Defense Technology, Hunan, China.
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
Jul 3, 2026Hamish OgilvyLLM Inference Acceleration Mixed-Precision Quantization
Algolia Research
ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning
Oct 26, 2026Rhea Sanjay Sukthanker, Pashmina CameronLLM Quantization Low-Bit Quantization
Microsoft Applied Sciences Group

Jun 9, 2026J/K move · Enter open · S save

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Haoyu Wang, Xingyu Yu, Haiyan Zhao, Fengxiang Wang+1

Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · College of Computer Science and Technology, National University of Defense Technology, Hunan, China.

LLM Quantization Quantization-Aware Training Vector Quantization Weight-Only Quantization Neural Network Quantization Data-Efficient Learning

Open paper AskPDF arXiv

Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). Current QAT methods are mainly based on scalar quantization (SQ), which enables efficient optimization but suffers from severe performance degradation at 2-bit precision. On the other hand, vector quantization (VQ) provides substantially higher representational capacity, but its discrete codebook lookup prevents end-to-end training. We propose LC-QAT, a 2-bit weight-only VQ-QAT framework that represents quantized weights via a learned affine mapping over discrete vectors, which yields a high-quality PTQ initialization and enables fully differentiable end-to-end optimization without explicit codebook lookup in the training forward pass. This strong post-training initialization makes LC-QAT highly data-efficient. Experiments across diverse LLMs demonstrate that LC-QAT consistently outperforms state-of-the-art QAT methods while using only 0.1%--10% of the training data. Our results establish LC-QAT as a practical and scalable solution for extreme low-bit model deployment. Codes are publicly available at https://github.com/AI9Stars/UniSVQ.

Abstract

LLM Quantization

Explore similar work

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models

ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning