All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs

Zhixiong Zhao; Zukang Xu; Guangyu Sun; Lifeng Liu; Dawei Yang

cs.LGSep 5, 2026

All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs

Authors: Zhixiong Zhao, Zukang Xu, Guangyu Sun, Lifeng Liu, Dawei Yang

Organizations: Houmo AI · School of Integrated Circuits, Peking University

Abstract

Large language models (LLMs) have achieved remarkable progress, yet their massive storage and memory-bandwidth demands still hinder efficient deployment. Weight binarization is a promising solution, but existing binarization-based post-training quantization (PTQ) methods usually far exceed the nominal 1-bit storage target due to hidden overhead. To address this gap, we propose All for 1-Bit (AF1), a genuine 1-bit PTQ framework for LLMs. AF1 comprises two complementary components: (1) Null-space-Aware Binary Factorization (NABF) for improving binary reconstruction through Hessian-aware surrogate reparameterization, null-space-aware binary factorization, and scale-only global reconstruction; and (2) Hierarchical Shapley Allocation (HiSA) for assigning structural capacity using hierarchical Shapley sensitivity. Together, they preserve model accuracy under a strict 1.0-BPW budget in the PTQ setting. Experiments on LLaMA, Qwen, and Gemma families show that AF1 consistently outperforms existing binarization-based PTQ methods in perplexity and zero-shot accuracy. Compared with BF16, AF1 achieves an average 2.5 times inference speedup and over 90% memory reduction across evaluated models, providing a practical path toward deployable genuine 1-bit compression for LLMs. The code for reproducibility is available at https://github.com/Kishon-zzx/AF1.

Figures & tables

Figure 1: Accuracy–memory trade-off across BF16 models and binarization-based PTQ methods, with L, G, and Q denoting LLaMA-3, Gemma-3, and Qwen3.

Figure 2: Overview of AF1. Null-space-Aware Binary Factorization (NABF) constructs Hessian-aware surrogate weights, suppresses SVID projection errors, and performs scale-only reconstruction. Hierarchical Shapley Allocation (HiSA) allocates structural budgets from coarse layer-level capacity to fine-grained module-level dimensions.

Method	#Bits (W)	LLaMA-3-8B		LLaMA-3-70B		Qwen3-8B		Qwen3-14B		Qwen3-32B		Gemma-3-4B-it		Gemma-3-12B-it
		0-shot 7	Wiki	0-shot 7	Wiki	0-shot 7	Wiki	0-shot 7	Wiki	0-shot 7	Wiki	0-shot 7	Wiki	0-shot 7	Wiki
		Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )	Avg.( $\uparrow$ )	( $\downarrow$ )
BF16	16	72.51	6.14	79.08	2.86	69.09	9.00	72.47	8.64	72.42	7.61	67.12	17.39	72.20	25.20
RTN	1.00	25.82	2.57e6	24.88	3.77e4	25.28	7.81e6	25.55	1.37e8	25.41	3.82e6	25.28	2.12e7	25.30	7.29e7
BiLLM	2.88	35.38	48.18	28.32	137.64	33.70	96.21	53.53	28.66	61.23	16.79	32.33	134.42	38.41	381.19
STBLLM ${}_{\text{(4:8)}}$	3.50	29.43	196.03	26.55	963.92	28.02	291.76	40.88	48.80	55.86	17.31	29.31	202.54	31.46	4.69e5

Table 1: Comparison of perplexity on WikiText2 and averaged accuracy on seven zero-shot tasks. Bold and underline denote the best and second-best quantized results. Full results are in Appendix A.4.1 .

Model	Method	#Bits(W)	Model Size	MMLU-Pro	HumanEval	GSM8K
LLaMA-3.1 -8B-Instruct	BF16	16	16.07GB	46.64	67.07	83.24
	BiLLM	2.88	3.75GB	8.77	15.33	21.36
	STBLLM ${}_{\text{(4:8)}}$	3.50	4.32GB	13.74	10.15	10.83
	ARB-LLM ${}_{\text{RC}}$	2.51	3.36GB	10.16	20.12	30.38
	HBLLM ${}_{\text{row}}$	3.25	4.15GB	23.57	28.97	40.26
	AF1	1.00	1.56GB	20.12	36.71	44.78

Table 2: Results on instruction-tuned models across three challenging benchmarks. Bold and underline denote the best and second-best quantized results.

Figure 3: Hyperparameter sensitivity of HiSA on WikiText2 perplexity. (a) Layer-level coarse-grained allocation under different

\tau

and

M_{\mathrm{layer}}

. (b) Block-level fine-grained allocation under different

\alpha

and

M_{\mathrm{block}}

Model	NABF			HiSA		Wiki2	0-shot 7
Model	Step1	Step2	Step3	Layer	Module	PPL ( $\downarrow$ )	Avg. ( $\uparrow$ )
LLaMA- 3.2-3B	✗	✗	✗	✗		8.9e2	25.48
	✓	✗	✗			1.3e2	29.31
	✓	✓	✗			41.34	34.85
	✓	✓	✓			30.47	40.79
	✓			✓	✗	25.52	42.30

Table 3: Combined ablation on LLaMA-3.2-3B. Step1–3 denote surrogate reparameterization, null-space binary factorization, and scale-only reconstruction; Layer and Module denote the two stages of HiSA allocation.

Figure 4: Quantization time and WikiText2 perplexity comparison under near-1-bit compression.

Figure 5: Inference memory and decoding throughput comparison across BF16, GPTQ-W4, and AF1.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Method	#Bits(W)	AE $\uparrow$	AC $\uparrow$	HS $\uparrow$	LO $\uparrow$	LS $\uparrow$	PQ $\uparrow$	WG $\uparrow$	Avg. 7 $\uparrow$
LLaMA-3-8B	BF16	16	77.90	52.82	79.07	75.63	68.58	80.63	72.93	72.51
	RTN	1.00	24.71	25.94	26.63	0.00	0.00	51.85	51.62	25.82
	BiLLM	2.88	36.78	22.18	34.81	28.76	13.00	58.49	53.67	35.38
	STBLLM ${}_{\text{(4:8)}}$	3.50	31.10	21.76	31.29	11.84	5.88	54.52	49.64	29.43
	ARB-LLM ${}_{\text{RC}}$	2.51	48.78	28.33	43.55	50.65	29.83	65.61	56.75	46.21
	HBLLM ${}_{\text{row}}$	3.25	54.12	32.68	60.22	58.51	46.40	70.35	65.19	55.35

Appendix

Table 4: Detailed zero-shot accuracy on ARC-Easy (AE), ARC-Challenge (AC), HellaSwag (HS), LAMBADA-openai (LO), LAMBADA-standard (LS), PIQA (PQ), and WinoGrande (WG). The average score is computed over the seven zero-shot tasks. Bold and underline denote the best and second-best quantized results within each model.

$\eta$	LLaMA-3-8B	Qwen3-14B	Gemma-3-4B-it	Avg.
0.01	22.18	13.32	32.82	22.77
0.02	22.35	13.45	32.74	22.85
0.03	22.51	13.58	33.02	23.04
0.04	22.43	13.72	33.11	23.09
0.05	22.69	13.66	33.23	23.19
0.06	22.58	13.84	33.34	23.25

Appendix

Table 5: Ablation on the residual-energy threshold

\eta

used in the adaptive null-space cutoff. Results are reported as WikiText2 perplexity on LLaMA-3-8B, Qwen3-14B, and Gemma-3-4B-it. Bold and underline denote the best and second-best results, respectively.

Model	Avg. Bits(W)	General Allocation Baselines (PPL $\downarrow$ )					HiSA Variants (PPL $\downarrow$ )
Model	Avg. Bits(W)	Uniform	LIM	ZD	Act.	NLL	Only Dir.	Only Mag.	HiSA
LLaMA-3-8B	1.00	35.84	29.67	32.51	27.94	25.86	24.59	23.33	22.18
Qwen3-8B	1.00	28.92	23.38	25.71	21.45	19.37	18.15	17.59	16.26

Appendix

Table 6: Ablation on different allocation-sensitivity metrics and HiSA variants under the same 1.00-bit weight budget. We report WikiText2 perplexity (PPL; lower is better). Only Dir. and Only Mag. use only directional and magnitude distortion, respectively, while HiSA uses the full Soft-OR fusion of both distortion modes. Bold denotes the best result.

Figure 6: Additional hyperparameter sensitivity of HiSA on LLaMA-3-8B . We report WikiText2 perplexity under different values of the layer-level temperature coefficient

\tau

, the layer-level sampling number

M_{\mathrm{layer}}

, the block-level smoothing exponent

\alpha

, and the block-level sampling number

M_{\mathrm{block}}

Figure 7: Additional hyperparameter sensitivity of HiSA on Gemma-3-4B-it . We report WikiText2 perplexity under different values of the layer-level temperature coefficient

\tau

, the layer-level sampling number

M_{\mathrm{layer}}

, the block-level smoothing exponent

\alpha

, and the block-level sampling number

M_{\mathrm{block}}

Figure 8: Ablation on calibration data. We use C4 as the calibration corpus and report WikiText2 perplexity after quantization. (a) varying the number of calibration samples. (b) varying the random seed for calibration-data sampling.

Prompt	Method	Response	Assessment
Q1: I have 3 apples, buy 5 more, and give away 2. How many apples are left? Answer: 6	BF16	You start with 3 apples, buy 5 more, and give away 2. Final answer: 6 apples .	Correct
	BiLLM	The $\cdots$	Degenerated
	ARB-LLM	$3+5=8$ , and $8-2=6$ . 6 apples are left.	Correct
	HBLLM	$3+5=8$ apples, then $8-2=6$ apples.	Correct
	AF1	You start with 3 apples, buy 5 more, making 8 apples. Then you give away 2, so you have 6 apples left.	Correct
Q2: What are the capitals of Canada, Japan, and Germany? Answer: Ottawa, Tokyo, Berlin	BF16	Canada: Ottawa ; Japan: Tokyo ; Germany: Berlin .	Correct

Appendix

Table 7: Qualitative examples on Qwen3-14B-Instruct under full precision and different binary PTQ methods. All prompts have objective answers. Dark green marks correct key information, dark red marks wrong answers or degenerated repetitions, and dark orange indicates partially correct but unstable generations. Repeated continuations are abbreviated for readability.

Explore similar work

BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs
May 1, 2026Zhixiong Zhao, Zukang Xu, Dawei YangLLM Quantization LLM Inference Acceleration
1Houmo AI
LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation
Apr 21, 2026Siqing Song, Chuang Wang, Yong Lang +2LLM Quantization Activation Quantization
MAIS, Institute of Automation, Chinese Academy of Sciences, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, China · Central Media Technology Institute, Huawei
Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision Quantization LLM Quantization
Beihang University · Communication University of China · Beijing Jiaotong University

May 1, 2026J/K move · Enter open · S save

BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs

Zhixiong Zhao, Zukang Xu, Dawei Yang

1Houmo AI

LLM Quantization LLM Inference Acceleration Activation Quantization Post-Training Quantization Low-Bit Quantization

Open paper AskPDF arXiv

Large language models (LLMs) have driven major progress in NLP, yet their substantial memory and compute demands still hinder practical deployment. Binarization can compress weights to 1 bit, fundamentally lowering compute and bandwidth cost. However, existing methods cannot address activation heavy tails and thus must keep activations in high precision, preventing true end-to-end acceleration. To overcome this limitation, we propose BWLA (Binarized Weights and Low-bit Activations), the first post-training quantization framework that preserves high accuracy while achieving 1-bit weight quantization together with low-bit activations (e.g., 6 bits). The Orthogonal-Kronecker Transformation (OKT) learns an orthogonal mapping via EM minimization, converting unimodal weights into symmetric bimodal forms while suppressing activation tails and incoherence. The Proximal SVD Projection (PSP) then performs lightweight low-rank refinement through proximal SVD projection, further enhancing quantizability with minimal overhead. On Qwen3-32B, BWLA reaches a Wikitext2 perplexity of 11.92 under 6-bit activations (vs. 38 from SOTA), improves five zero-shot tasks by more than 70%, and delivers 3.26 times inference speedup, demonstrating strong potential for real-world LLM compression and acceleration.

Abstract

Explore similar work

BWLA: Breaking the Barrier of W1AX Post-Training Quantization for LLMs

LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation

Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method