Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Organizations: Huawei Paris Research Center, Paris, France
Abstract
Weight binarization has emerged as a promising strategy to reduce the complexity of large language models (LLMs). Existing approaches fall into post-training binarization, which is simple but causes severe performance loss, and training-aware methods, which depend on full-precision latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMs in the Boolean domain, eliminating the need for latent weights. This enhances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.
Figures & tables
| Method | Train from Scratch | Post-training Binarization | Finetune from FP Model | Calibration Data | Weight Update | Multiple Binary Bases | Higher-bit Salient Weights |
|---|---|---|---|---|---|---|---|
| b it n et ( Wang et al., 2023 ) | ✓ | ✗ | ✗ | NA | FP latent-weights | ✗ | ✗ |
| b i llm ( Huang et al., 2024 ) | ✗ | ✓ | ✗ | ✓ | NA | ✓ | ✓ |
| pb - llm ( Yuan et al., 2024 ) | ✗ | ✓ | ✗ | ✓ | NA | ✗ | ✓ |
| stbllm ( Dong et al., 2025 ) | ✗ | ✓ | ✗ | ✓ | NA | ✓ | ✓ |
| arb - llm ( Li et al., 2025 ) | ✗ | ✓ | ✗ | ✓ | NA | ✓ | ✓ |
| b it s tack ( Wang et al., 2025 ) | ✗ | ✓ | ✗ | ✗ | NA | ✓ | ✗ |
| Model | Method | Wbits | Perplexity ( ) | Zero-shot Accuracy ( ) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Wiki2 | C4 | BoolQ | PIQA | Hella. | WinoG. | ARC-e | ARC-c | Average | |||
| opt -1.3 b | fp 16 | 16 | 14.62 | 14.72 | 57.82 | 72.42 | 53.70 | 59.51 | 50.97 | 29.52 | 53.99 |
| pb - llm | 1.7 | 272.83 | 175.42 | 62.17 | 54.24 | 27.25 | 50.27 | 27.98 | 23.72 | 40.94 | |
| b i llm | 1.11 | 69.45 | 63.92 | 61.92 | 59.52 | 33.81 | 49.32 | 34.38 | 22.35 | 43.55 | |
| o ne b it | 1 | 20.36 | 20.76 | 57.85 | 66.53 | 39.21 | 54.61 | 42.80 | 23.97 | 47.50 | |
| m o s | 1 | 18.45 | 18.83 | 60.34 | 68.66 | 41.99 | 53.99 | 44.87 | 26.19 | 49.34 | |
| OPT Model | WBits | Wiki2 | C4 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 125M | 350M | 1.3B | 2.7B | 6.7B | 125M | 350M | 1.3B | 2.7B | 6.7B | ||
| full-precision | 16 | 27.65 | 22.00 | 14.63 | 12.47 | 10.86 | 26.56 | 22.59 | 16.07 | 14.34 | 12.71 |
| rtn ( Yao et al., 2022 ; Dettmers et al., 2022 ) | 3 | 1.3e3 | 64.57 | 1.3e4 | 1.6e4 | 5.8e3 | 834 | 55.49 | 5.2e3 | 1.1e4 | 5.3e3 |
| optq ( Frantar et al., 2023 ) | 3 | 53.85 | 33.79 | 20.97 | 16.88 | 14.86 | 42.41 | 31.33 | 21.63 | 18.17 | 17.14 |
| mbok [Ours] | 3 1 | 29.10 | 23.12 | 15.30 | 13.09 | 11.03 | 28.62 | 22.10 | 15.68 | 14.00 | 12.33 |
| weight size | fp 16 | quip# ( Tseng et al., 2024a ) | qtip ( Tseng et al., 2024b ) | mbok (Ours) |
|---|---|---|---|---|
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| true | true | false | false | false | true | true | false |
| true | false | true | true | true | false | false | false |
| false | true | false | false | true | false | false | true |
| false | false | true | true | false | true | true | true |
| Model | Method | Wbits | Perplexity ( ) | Zero-shot Accuracy ( ) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Wiki2 | C4 | BoolQ | PIQA | Hella. | WinoG. | ARC-e | ARC-c | Average | |||
| opt -1.3 b | mbok (2 kernels) | 2 1 | 16.13 | 16.61 | 58.53 | 70.67 | 48.11 | 56.75 | 48.19 | 27.90 | 51.69 |
| mbok (3 kernels) | 3 1 | 15.30 | 15.68 | 60.64 | 70.78 | 50.71 | 56.83 | 48.82 | 28.49 | 52.71 | |
| mbok (4 kernels) | 4 1 | 14.83 | 14.92 | 60.95 | 70.85 | 51.02 | 56.85 | 49.13 | 29.24 | 53.01 | |
| ll a ma -7 b | mbok (2 kernels) | 2 1 | 6.83 | 8.53 | 69.20 | 74.32 | 64.80 | 60.30 | 49.05 | 34.90 | 58.76 |
| mbok (3 kernels) | 3 1 | 6.20 | 7.76 | 67.89 | 76.15 | 68.91 | 63.30 | 48.94 | 37.62 | 60.47 | |
| Model | Method | Wbits | Perplexity ( ) | Zero-shot Accuracy ( ) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Wiki2 | C4 | BoolQ | PIQA | Hella. | WinoG. | ARC-e | ARC-c | Average | |||
| ll a ma 2-7 b | fp 16 | 16 | 5.47 | 6.97 | 71.10 | 76.88 | 72.94 | 67.09 | 53.58 | 40.61 | 63.70 |
| pb - llm | 1.7 | 76.75 | 85.92 | 62.17 | 52.82 | 26.87 | 50.11 | 26.89 | 24.31 | 40.53 | |
| b i llm | 1.11 | 27.72 | 36.34 | 62.14 | 59.19 | 35.18 | 53.11 | 34.22 | 26.54 | 45.06 | |
| o ne b it | 1 | 8.60 | 10.74 | 63.06 | 70.40 | 54.24 | 56.67 | 40.82 | 29.35 | 52.42 | |
| m o s | 1 | 7.88 | 9.75 | 65.02 | 71.55 | 59.41 | 56.18 | 41.84 | 30.03 | 54.01 | |
| Prompt: | What are the capitals of China, US and Russia? |
|---|---|
| FP16: | The capital of China is Beijing. The capital of the United States is Washington D.C. and the capital of Russia is Moscow. |
| MBOK: | The capital of China is Beijing. The capital of the US is Washington D.C. and the capital of Russia is Moscow. |
| Prompt: | I have a friend who lives in London and I would like to visit him. How would I go there from Paris? |
| FP16: | You can take the Eurostar from Gare du Nord in Paris to St. Pancras Station in London. The trip takes about 2 hours and 15 minutes. |
| MBOK: | The fastest way to get from Paris to London is to take the Eurostar train. The Eurostar departs from the Gare du Nord station in Paris and arrives at the St. Pancras International station in London. The trip takes about 2 hours and 15 minutes. |
| Prompt: | What is Fete du Travail? |
| bit-width | method | opt -125 m | opt -350 m | opt -1.3 b |
|---|---|---|---|---|
| q u ip ( Chee et al., 2023 ) | 34.22 | 25.19 | 16.21 | |
| s hift a dd llm ( You et al., 2024 ) | 31.29 | 24.24 | 21.53 | |
| 2 | mbok [Ours] | 29.10 | 23.12 | 15.03 |
| q u ip ( Chee et al., 2023 ) | 347.40 | 672.30 | 41.64 | |
| s hift a dd llm ( You et al., 2024 ) | 51.15 | 40.24 | 29.03 | |
| 3 | mbok [Our] | 28.60 | 24.54 | 16.13 |
| method | Wiki2 | ARc-e | ARC-c | PIQA | Hella. | WinoG. |
|---|---|---|---|---|---|---|
| awq ( Lin et al., 2024 ) | 1.8e5 | 26.3 | 26.7 | 50.9 | 26.5 | 49.3 |
| b it s tack ( Wang et al., 2025 ) | 29.93 | 32.3 | 25.6 | 62.4 | 42.8 | 53.6 |
| db - llm ( Chen et al., 2024 ) | 7.23 | 45.2 | 33.5 | 73.1 | 61.9 | 61.7 |
| mbok [Ours] | 6.87 | 44.8 | 34.2 | 75.0 | 65.6 | 61.7 |
| method | Wiki2 ( ) | C4 ( ) |
|---|---|---|
| quip# ( Tseng et al., 2024a ) | 6.86 | 8.36 |
| qtip ( Tseng et al., 2024b ) | 6.52 | 7.99 |
| mbok [Ours] | 6.83 | 8.53 |
| weight size | fp 16 | quip# ( Tseng et al., 2024a ) | qtip ( Tseng et al., 2024b ) | mbok (Ours) |
|---|---|---|---|---|
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) |
| weight size | fp 16 | quip# ( Tseng et al., 2024a ) | qtip ( Tseng et al., 2024b ) | mbok (Ours) |
|---|---|---|---|---|
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) | ||
| ( ) | ( ) | ( ) |
LLM Quantization
Large language model quantization reduces numerical precision to lower memory use and accelerate training or inference.
See all 438 papers