Neural Network Quantization

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 125% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 270

All topics

Block-Sphere Vector Quantization
May 19, 2026Heesang Ann, Joongkyu Lee, Min-hwan OhRotation-Based Quantization Vector Quantization
Seoul National University
OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond
May 19, 2026Zunhai Su, Rui Yang, Chao Zhang +11KV Caching Rotation-Based Quantization
Tsinghua University · Meituan LongCat Team · The University of Hong Kong +3
Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
May 19, 2026Jinghe Zhang, Daliang Xu, Chenghua Wang +5Mixed-Precision Quantization LLM Quantization
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
May 19, 2026Zukang Xu, Xing Hu, Dawei YangLLM Quantization Rotation-Based Quantization
1Houmo AI.
A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
May 18, 2026Lei DongLLM Quantization Ternary Quantization
Independent Researcher
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
May 18, 2026Le Su, Xing Luo, Zhi JinLLM Quantization Post-Training Quantization
School of Intelligent Systems Engineering Shenzhen Campus of Sun Yat-sen University Shenzhen, Guangdong 518107, China · Department of Frontier Research Peng Cheng Laboratory Shenzhen, Guangdong 518055, China
Attention Sinks and Outliers in Attention Residuals
May 18, 2026Haozheng Luo, Haoran Dai, Shaoyang Zhang +10LLM Quantization Attention Residual Connections
Department of Computer Science, Northwestern University · Department of Computer Science, Illinois Institute of Technolog · Department of Computer Science and Engineering, University of Michigan +5
OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
May 18, 2026Zhongzhu Zhou, Donglin Zhuang, Jisen Li +4Rotation-Based Quantization LLM Inference Acceleration
Together AI · University of Sydney · University of Illinois Urbana-Champaign
StatQAT: Statistical Quantizer Optimization for Deep Networks
May 18, 2026Mehmet Aktukmak, Daniel Huang, Ke DingQuantization-Aware Training Activation Quantization
Intel
Q-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
May 17, 2026Vedant Waykole, Haroon R. LoneNon-IID Federated Learning Edge ML
IISER Bhopal · India
Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings
May 17, 2026Wenxuan XiaoNeural Representation Geometry Embedding Compression
Changsha University
CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
May 16, 2026Houji Wen, Jiangyong Yu, Jun Li +1Cross-Attention 4-Bit Quantization
1Nanjing University of Science and Technology · 2Houmo AI
E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring
May 16, 2026Wenjun Wang, Yanggan Gu, Shuo Cai +44-Bit Quantization Post-Training Quantization
The Hong Kong Polytechnic University · PolyU-Daya Bay Technology and Innovation Research Institute
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
May 16, 2026Sayeh Sharify, Mahsa Salmani, Hesham MostafaDiffusion Transformer Diffusion Model Quantization
d-Matrix Santa Clara, CA 95054
Not All Tasks Quantize Equally: Fisher-Guided Quantization for Visual Geometry Transformer
May 15, 2026Yipu Zhang, Jintao Cheng, Weilun Feng +5Feed-Forward 3D Reconstruction Post-Training Quantization
Department of Electronic and Computer Engineering, HKUST · State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences +2
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization
May 14, 2026Peilin Sun, Jianxin WuActivation Outliers Post-Training Quantization
State Key Laboratory of Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inference
May 14, 2026Thomas WittLLM Quantization Vector Quantization
Gemini Stiftung Leipzig
Provable Quantization with Randomized Hadamard Transform
May 13, 2026Ying Feng, Piotr Indyk, Michael Kapralov +2Vector Quantization Randomized Sketching
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
Search Your Block Floating Point Scales!
May 12, 2026Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu +104-Bit Quantization LLM Inference Acceleration
Cornell University, Ithaca, NY, USA · Together AI, USA · University of California San Diego, La Jolla, CA, USA +1
SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization
May 12, 2026Chengzhu Bao, Xianglong Yan, Zhiteng Li +3LLM Quantization 4-Bit Quantization
Shanghai Jiao Tong University · Tencent Hunyuan
MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
May 12, 2026Yupeng Su, Ruijie Zhang, Ziyue Liu +24-Bit Quantization Muon Optimizer
University of California, Santa Barbara
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
May 11, 2026Ryan Lucas, Mehdi Makni, Xiang Meng +2LLM Quantization Alternating Direction Method of Multipliers
MIT Operations Research Center · MIT Sloan School of Management · MIT Center for Statistics
LoKA: Low-precision Kernel Applications for Recommendation Models At Scale
May 11, 2026Liang Luo, Yinbin Ma, Quanyu Zhu +21Mixed-Precision Quantization GPU Kernel Optimization
Meta AI
Nano-U: Efficient Terrain Segmentation for Tiny Robot Navigation
May 11, 2026Federico Pizzolato, Francesco Pasti, Nicola BellottoRobot Navigation Semantic Segmentation
Dept of Information Engineering, University of Padua, Italy
PermuQuant: Lowering Per-Group Quantization Error by Reordering Channels for Diffusion Models
May 10, 2026Yongsen Cheng, Kai Liu, Kaiwen Tao +5Diffusion Model Quantization Post-Training Quantization
Shanghai Jiao Tong University · Huawei Noah’s Ark Lab
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
May 10, 2026Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu +1Efficient Inference Mixture-of-Experts Inference
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
May 9, 2026Euntae Choi, Sumin Song, Sungjoo YooLLM Quantization Quantization-Aware Training
Seoul National University
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
May 9, 2026Beshr IslamBouli, David JinLLM Quantization 4-Bit Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA.

May 19, 2026, cs.LGJ/K move · Enter open · S save

Block-Sphere Vector Quantization

Heesang Ann, Joongkyu Lee, Min-hwan Oh

Seoul National University

Rotation-Based Quantization Vector Quantization Neural Network Quantization

Open paper AskPDF arXiv

Vector quantization is a fundamental primitive for scalable machine learning systems, enabling memory-efficient storage, fast retrieval, and compressed inference. Recent rotation-based quantizers such as EDEN, RabitQ, and TurboQuant have introduced strong guarantees and empirical performance, but the surrounding comparisons have been difficult to interpret because they rely on different distortion criteria, probability regimes, and implementation assumptions. As our first contribution, we provide a unified theoretical comparison of these methods and show that their relative advantages are criterion-dependent rather than absolute: EDEN and TurboQuant are favorable for MSE distortion, EDEN is also effective for expected inner-product distortion, and RabitQ provides strong high-probability control. This comparison further clarifies that EDEN provides particularly strong guarantees for expected distortion measures. As our second contribution, we introduce Block-Sphere Quantization (BlockQuant), a new rotation-based block quantization algorithm designed around the spherical geometry of randomly rotated vectors. Unlike coordinate-wise quantizers, BlockQuant quantizes blocks on the sphere, preserving the geometry of rotated embeddings more faithfully. We prove that this block-spherical design theoretically improves over the baselines considered in this paper for both reconstruction MSE and expected inner-product distortion. Our experiments on real embedding datasets and long-context LLM inference tasks show practical gains that are consistent with our theoretical improvements.

Momentum

Related topics

Latest papers 270