Neural Network Quantization

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 125% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 270

All topics

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
Jun 25, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +2LLM Quantization Ternary Quantization
1Intel Labs China
SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference
Jun 25, 2026Haoqian Meng, Yilun Luo, Yafei Zhao +44-Bit Quantization LLM Inference Acceleration
School of Computer Science and Technology, Tianjin University
What Survives When You Compress a Recursive Reasoner for the Edge?
Jun 25, 2026Pearse Jim, Steven Kolawole, Opegbemi Matthias Busoye +2Model Compression Efficient Neural Network Inference
ML Collective · Carnegie Mellon University
Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization
Jun 24, 2026Kamar Hibatallah Baghdadi, Kawther Guoual Belhamidi, Sara Belhadj +2Hierarchical RL Neural Network Quantization
aNational School of Artificial Intelligence (ENSIA), Sidi Abdellah Campus, Algiers, Algeria · bCosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France. · aNational School of Artificial Intelligence (ENSIA), Sidi Abdellah
Neural Network Quantization by Learning Low-Loss Subspaces
Jun 23, 2026Vladimir Protsenko, Mikhalina Kharkevich, Alexander Vashchilko +1Neural Network Optimization Neural Network Quantization
Huawei, Montreal, Canada
RoPE-Aware Bit Allocation for KV-Cache Quantization
Jun 23, 2026Fengfeng Liang, Yuechen Zhang, Jiaya JiaKV Caching LLM Inference Acceleration
1Hong Kong University of Science and Technology · 2The Chinese University of Hong Kong · 3MiMo, Xiaomi Corporation
HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models
Jun 22, 2026Yuval Domb, Hadar Sackstein, Tomer SolbergLLM Quantization Diffusion Model Quantization
ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers
Jun 20, 2026Changjun Li, Runqing Jiang, Lian Xu +3Vision Transformer Activation Quantization
School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · Computer Science and Software Engineering, The University of Western Australia, Perth, Australia · University of Oxford, Oxford, United Kingdom
Displacement Is Not Direction: Evaluating Fidelity Metrics for Quantized LLM Deployment
Jun 17, 2026Miloš Nikolić, Ali Hadi Zadeh, Enrique Torres Sanchez +1LLM Quantization LLM Evaluation
ByteShape · University of Toronto · Vector Institute for Artificial Intelligence
Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models
Jun 16, 2026Ramprasath Ganesaraja, Sahil Dilip Panse, Swathika NLLM Quantization Ternary Quantization
EdgeVerve Systems Limited
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Jun 15, 2026Yuanteng Chen, Peisong Wang, Zhilei Liu +9Mixed-Precision Quantization LLM Quantization
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · 3Zhongguancun Academy
MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization
Jun 14, 2026Yangjia Hu, Haodong Wang, Zicong Hong +8LLM Quantization 4-Bit Quantization
HKUST · EPFL · MetaX Integrated Circuits Co., Ltd
AQ4SViT: An Automated Quantization Framework with Search Gating Policy for Compressing Spiking Vision Transformers
Jun 14, 2026Rachmad Vidya Wicaksana Putra, Saad Iftikhar, Muhammad ShafiqueSpiking Transformers Neural Network Quantization
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates (UAE)
Signed Symmetric Quantization for Few-Bit Integers
Jun 12, 2026Ian Colbert, Eashan Dash, Pablo Monteagudo-Lago +5LLM Quantization Weight-Only Quantization
AMD
Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score
Jun 11, 2026Mariya Pavlova, Harrison Bo Hua Zhu, Lidia Vitanova +2Mixed-Precision Quantization Post-Training Quantization
Department of Computing, Imperial College London, London, United Kingdom · Department of Public Health, University of Copenhagen, Copenhagen, Denmark · GATE Institute, Sofia University St. Kliment Ohridski, Sofia, Bulgaria +1
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Jun 11, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +3Mixed-Precision Quantization LLM Quantization
Houmo AI, China · 2Southeast University, Nanjing, China
Multi-Bitwidth Quantization for LLMs Using Additive Codebooks
Jun 11, 2026Liza Babaoglu, Shuangyi Chen, Ashish KhistiLLM Quantization Weight-Only Quantization
University of Toronto
Reducing the Complexity of Deep Learning Models for EEG Analysis on Wearable Devices
Jun 10, 2026Farough Shayeste Roodi, Parham Zilouchian Moghaddam, Mahdi Mohammadi-nasab +3Electroencephalography EEG Seizure Detection
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran · Intelligent Future Technologies (IFT), Mälardalen University, Väster˚as, Sweden · Department of Computer Systems, Tallinn University of Technology (TalTech), Tallinn, Estonia
Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs
Jun 10, 2026Deep Gandhi, Ali Asaria, Tony SalomoneMixed-Precision Quantization Diffusion Transformer
Transformer Lab
What Limits Does Quantization Place on Dense Top- $k$ Retrieval? A Theoretical Study
Jun 10, 2026Koki Okajima, Tsukasa YoshidaInformation Retrieval Low-Bit Quantization
NTT, Inc.
Optimal Post-Training Quantization Scales and Where to Find Them
Jun 9, 2026Juan Amboage, Pablo Monteagudo-Lago, Ian Colbert +2LLM Quantization Weight-Only Quantization
AMD
UniSVQ: 2-bit Unified Scalar-Vector Quantization
Jun 9, 2026Haoyu Wang, Haiyan Zhao, Xingyu Yu +4LLM Quantization Vector Quantization
Department of Computer Science and Technology, Tsinghua University, Beijing, China · School of Software and Microelectronics, Peking University, Beijing, China · School of Physics, Beihang University, Beijing, China
Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
Jun 8, 2026Hanyang Li, Jianhao Ma, Ying CuiQuantization-Aware Training Neural Network Optimization
Department of IEOR, University of California, Berkeley · Department of Statistics and Data Science, University of Pennsylvania
APEX4: Efficient Pure W4A4 LLM Inference via Intra-SM Compute Rebalancing
Jun 7, 2026Hong Guo, Nianhui Guo, Weixing Wang +3LLM Quantization GPU Kernel Optimization
Hasso Plattner Institute · German University of Digital Science
SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Serving
Jun 4, 2026Hongyuan Liu, Yawei Li, Zhiqiang Que +3LLM Quantization LLM Serving
University of Electronic Science and Technology of China · University of Bristol · 3ETH Zurich
Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models
Jun 4, 2026Hancheol Park, Geonho Lee, Tairen Piao +1LLM Quantization Mixture-of-Experts Quantization
Nota Inc., South Korea.
AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5
Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling
Jun 3, 2026Ting-An Chen, Chin-Yuan Yeh, De-Nian YangLong-Tail Learning Low-Bit Quantization
Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taiwan +1
QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
Jun 3, 2026Pasindu Wickramasinghe, Achyuta Muthuvelan, Rachmad Vidya Wicaksana Putra +2Mixed-Precision Quantization LLM Quantization
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates
Characterizing the Impact of NVFP4 Quantization for Low-Power Edge AI Deployment
Jun 3, 2026Ovishake Sen, Venkata Nithin Kamineni, Daniel Lobo +3Efficient Neural Network Inference Edge Inference
Department of Electrical and Computer Engineering, University of Florida, Gainesville, USA

Jun 25, 2026, cs.CLJ/K move · Enter open · S save

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan+1

1Intel Labs China

LLM Quantization Ternary Quantization LLM Compression LLM Inference Acceleration Post-Training Quantization Neural Network Quantization

Open paper AskPDF arXiv

In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware training to mitigate severe performance degradation, CAT-Q is a simple yet effective post-training quantization scheme that is readily applicable to LLMs with diverse architectures and model sizes. It has two key components, learnable modulation (LM) and softened ternarization (ST), which are coupled from an optimization perspective. LM leverages a composition of learnable factors to modulate the distribution of pre-trained high-precision weights and the ternary threshold, making them less sensitive to ternarization. ST further introduces a differentiable transition function to guide the ternarization process toward stable convergence. We show that, for pre-trained LLMs with 1.7B to 8B parameters, CAT-Q can efficiently quantize them into ternary models using only 512 calibration samples, while achieving superior performance than the seminal BitNet 1.58-bit v1 and v2 families (with 1.3B to 7B parameters) trained with 100B tokens, yielding about a 100,000X reduction in training tokens. Moreover, we show for the first time that CAT-Q can quantize much larger pre-trained LLMs having 14B to 235B parameters into leading ternary models within just 8 to 60 hours on 8 A100-80GB GPUs. Code is available at https://github.com/IntelChina-AI/BitTern.

Momentum

Related topics

Latest papers 270