cs.LGDec 25, 2025

Beyond local reconstruction: Geometry-preserving output alignment for binary LLM quantization

Authors: Dung Anh Hoang, Cuong Pham, Cuong Nguyen, Trung le, Jianfei Cai, Thanh-Toan Do

Organizations: Department of Data Science and AI, Monash University · Centre for Vision, Speech and Signal Processing, University of Surrey

Abstract

Recent advances in post-training quantization for large language models (LLMs) have demonstrated that low-bit quantization methods can maintain most of the original model performance. Despite such successes, 1-bit quantization remains particularly challenging. A common strategy in 1-bit quantization is to determine binary weights by matching full-precision parameters, following a weight-driven criterion. However, this objective is not directly aligned with the quantized model's objective, which is to preserve the model's output behavior under the impact of quantization. A natural alternative is to adopt output-driven criteria that minimize discrepancies in model outputs using calibration data. Surprisingly, naive output-driven approaches often perform even worse in the 1-bit regime. In this paper, we show that this failure arises from two fundamental issues: error accumulation across layers and, more critically, anisotropic distortion of the representation space. Based on this observation, we introduce a geometry-preserving output-alignment framework for 1-bit LLM quantization. First, we formulate output alignment as a constrained optimization problem that improves reconstruction while explicitly controlling changes in token similarity, preventing reconstruction gains from being achieved at the expense of representation geometry. Second, we propose a geometry-aware salient-mask selection strategy that identifies quantization configurations according to their ability to preserve token relations, rather than relying solely on reconstruction error. These components enable effective output-driven optimization while retaining the structural information lost by naive alignment. Extensive experiments across multiple LLM families and downstream tasks demonstrate consistent improvements over existing 1-bit PTQ methods.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs

    Sep 5, 2026Zhixiong Zhao, Zukang Xu, Guangyu Sun +2LLM QuantizationPost-Training Quantization

  2. SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models

    Apr 11, 2026Han Liu, Haotian Gao, Xiaotong Zhang +5LLM QuantizationPost-Training Quantization

  3. Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs

    Apr 9, 2026Binxing Xu, Hao Gu, Lujun Li +8LLM QuantizationQuantization-Aware Training