Recent advances in post-training quantization for large language models (LLMs) have demonstrated that low-bit quantization methods can maintain most of the original model performance. Despite such successes, 1-bit quantization remains particularly challenging. A common strategy in 1-bit quantization is to determine binary weights by matching full-precision parameters, following a weight-driven criterion. However, this objective is not directly aligned with the quantized model's objective, which is to preserve the model's output behavior under the impact of quantization. A natural alternative is to adopt output-driven criteria that minimize discrepancies in model outputs using calibration data. Surprisingly, naive output-driven approaches often perform even worse in the 1-bit regime. In this paper, we show that this failure arises from two fundamental issues: error accumulation across layers and, more critically, anisotropic distortion of the representation space. Based on this observation, we introduce a geometry-preserving output-alignment framework for 1-bit LLM quantization. First, we formulate output alignment as a constrained optimization problem that improves reconstruction while explicitly controlling changes in token similarity, preventing reconstruction gains from being achieved at the expense of representation geometry. Second, we propose a geometry-aware salient-mask selection strategy that identifies quantization configurations according to their ability to preserve token relations, rather than relying solely on reconstruction error. These components enable effective output-driven optimization while retaining the structural information lost by naive alignment. Extensive experiments across multiple LLM families and downstream tasks demonstrate consistent improvements over existing 1-bit PTQ methods.
Figures & tables
Figure 1: Reconstruction error alone does not characterize the functional distortion introduced by extreme quantization. (a) Output-driven optimization allocates its reconstruction improvement unevenly across representation dimensions, disproportionately benefiting high-magnitude channels. (b) Despite reducing reconstruction error, it induces increasing distortion of the pairwise geometry between token representations. (c) We isolate this effect by perturbing the first-block representations with equal-magnitude errors while varying only their direction. Although all perturbations have identical reconstruction error, those that induce larger token-geometry distortion lead to higher final perplexity. Together, these observations suggest that preserving representation direction and token relationships captures information missed by reconstruction MSE.
Dataset
Method
Use
OPT
OPT
LLaMA
LLaMA
OA
Bits
1.3B
2.7B
6.7B
Bits
LLaMA-2 7B
LLaMA-2 13B
LLaMA-3 8B
C4
Full Precision
–
16
16.07
14.34
12.71
16
7.26
6.73
9.45
PB-LLM
{\color[rgb]{1,0,0}\times}
1.7
168.12
222.15
104.78
1.7
80.69
184.67
104.15
BiLLM
{\color[rgb]{1,0,0}\times}
1.11
64.14
44.77
42.13
1.08
39.38
25.87
61.04
ARB-X
{\color[rgb]{0,0.75,0.16}\checkmark}
1.11
47.60
34.97
22.54
1.08
28.02
19.82
41.86
OAC
{\color[rgb]{1,0,0}\times}
1.11
–
–
–
1.08
21.64
15.25
–
Table 1: Perplexity (↓) results of OPT and LLaMA models on C4, WikiText2, and PTB. Separate weight-bit columns are used for each model family. Use OA denotes methods that use an output-alignment objective for quantization.
Model
Method
Bits
Lambada
PIQA
OBQA
WinoGrande
ARC-E
ARC-C
HellaSwag
COPA
Avg. †
LLaMA-2-7B
ARB-RC
1.08
51.66
64.85
19.40
59.51
50.97
24.32
36.79
71.00
47.31
DBELL
1.08
41.63
64.85
17.60
56.91
50.38
23.98
35.65
63.00
44.25
Ours
1.08
58.53 (+6.87)
67.46 (+2.61)
20.80 (+1.40)
60.14 (+0.63)
53.41 (+2.44)
25.60 (+1.28)
38.91 (+2.12)
74.00 (+3.00)
49.86 (+2.55)
LLaMA-2-13B
ARB-RC
1.08
66.02
69.59
23.20
61.48
61.28
29.18
38.73
81.00
53.81
DBELL
1.08
55.75
68.44
20.80
61.64
60.10
28.16
39.08
78.00
51.50
Ours
1.08
69.28 (+3.26)
71.71 (+2.12)
25.00 (+1.80)
66.22 (+4.74)
64.44 (+3.16)
30.89 (+1.71)
43.09 (+4.36)
82.00 (+1.00)
56.58 (+2.77)
Table 2: Zero-shot accuracy (%) on QA reasoning benchmarks. Green values denote the improvement of our method over ARB-RC in percentage points. † The average is computed over all tasks.
Dataset
Metric
Method
Block Size
Qwen2
Qwen2.5
Qwen3
1.5B
7B
1.5B
7B
8B
C4
PPL ( ↓ )
ARB-RC
128
109.83
22.63
128.77
24.32
32.71
DBellQuant
128
607.06
29.03
1428.88
29.52
50.88
Ours
128
74.83 ( ↓ 35.00)
20.54 ( ↓ 2.09)
64.33 ( ↓ 64.44)
20.95 ( ↓ 3.37)
27.68 ( ↓ 5.03)
WikiText2
PPL ( ↓ )
ARB-RC
128
89.59
16.99
107.92
17.08
26.03
DBellQuant
128
542.06
28.52
2416.90
25.10
58.16
Table 3: Perplexity ( ↓ ) on C4, WikiText2, and PTB, and average zero-shot reasoning accuracy ( ↑ ) for Qwen models. All methods use a block size of 128. Green annotations report the absolute improvement over ARB-RC.
Optimization
Qwen-2.5-1.5B
LLaMA-2-7B
LQ
ΔSReg≥0
Mask
C4
Wiki
C4
Wiki
{\color[rgb]{1,0,0}\times}
{\color[rgb]{1,0,0}\times}
{\color[rgb]{1,0,0}\times}
128.77
107.92
20.56
16.36
{\color[rgb]{0,0.75,0.16}\checkmark}
{\color[rgb]{1,0,0}\times}
{\color[rgb]{1,0,0}\times}
115.69
98.18
29.12
26.24
{\color[rgb]{1,0,0}\times}
{\color[rgb]{0,0.75,0.16}\checkmark}
{\color[rgb]{1,0,0}\times}
119.14
99.45
19.97
15.66
{\color[rgb]{1,0,0}\times}
{\color[rgb]{1,0,0}\times}
{\color[rgb]{0,0.75,0.16}\checkmark}
86.00
73.07
19.91
16.17
{\color[rgb]{0,0.75,0.16}\checkmark}
{\color[rgb]{0,0.75,0.16}\checkmark}
{\color[rgb]{1,0,0}\times}
109.53
90.58
18.94
15.20
Table 4: Ablation study about the contribution of each component, using a fixed random seed.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: The difference between the token similarity matrix between full-precision model and quantized model, with and without SReg constraint, demonstrating the effectiveness of the proposed method.
Figure 3: Comparison of attention scores with and without the token-geometry preservation constraint against the full-precision model. Attention scores produced with the SReg constraint more closely match those of the pretrained model than those obtained without SReg constraint, demonstrating the effectiveness of the proposed method.
Model
ARB-X
ARB-RC
Ours
OPT-6.7B
87m
65m
93m
Llama-2-7B
91m
54m
76m
Llama-2-13B
147m
100m
119m
Appendix
Table 5: Quantization time comparison between 1-bit LLM methods and ours across different models.
Model
Prefill
Decode
FP16
2.83
0.87
ARB-RC
11.44
14.15
Ours
12.42
14.52
Appendix
Table 6: Inference throughput (tokens/s) on the Intel Xeon Gold 6548Y+ CPU Wang et al. (2023) . Higher is better.
Size
Method
PPL (↓)
C4
WikiText2
32
ARB-RC
109.45
95.08
Ours
83.44
76.56
128
ARB-RC
109.83
89.59
Ours
74.83
68.45
512
ARB-RC
127.84
89.13
Appendix
Table 7: Ablation study on the calibration set size for Qwen-2-1.5B.
Model
β
PPL (↓)
C4
WikiText2
OPT-6.7B
0
16.24
14.56
0.5
16.21
14.53
1
16.18
14.59
Appendix
Table 8: Ablation study for the hyper-parameter β .
Algorithm 1 Our-RC with Second-Order Relational Projection