cs.LGNov 17, 2026

HADVQ: Hessian Aware Directional Vector Quantization

Authors: Daniil Li, Mars Davletshin, Vadim Klyzhenko, Artem Danilishin, Valery Sandul, Daniil Ryazanov, Ekaterina Muravleva

Organizations: Applied AI Institute, Moscow, Russia · Luzin Research Center, Moscow, Russia

Abstract

Ultra-low-bit quantization of large language models is a challenging task because aggressive compression severely distorts model parameters and disproportionately harms a small number of highly sensitive channels. We present a vector quantization method tailored for this regime. Our approach begins with random preconditioning, which regularizes the weight space and makes it more amenable to quantization. We then represent weights in polar coordinates and decouple the quantization of directions and magnitudes. To quantize directions, we leverage the structure induced by random preconditioning and design a structured codebook based on the E8E_8 lattice, yielding a compact yet expressive representation. To further reduce degradation, we detect extremely important channels using a proxy Hessian and preserve them with an additional cost of only 0.16 bits. Together, these components provide an effective quantization pipeline for LLMs in the extremely low-bit regime. Empirical results demonstrate improved accuracy, highlighting the benefit of combining geometric structure, lattice-based codebooks, and selective channel preservation.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Leech Lattice Vector Quantization for Efficient LLM Compression

    Mar 11, 2026Tycho F. A. van der Ouderaa, Mart van Baalen, Paul Whatmough +1LLM QuantizationLLM Compression

  2. XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization

    Sep 30, 2026Xiaofan Que, Nir Elkayam, Spandan Pyakurel +2LLM QuantizationLLM Inference Acceleration