Neural Network Quantization

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 125% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 270

All topics

FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference
Aug 24, 2026Gongwei Lee, Ji Liu, Juncheng Jia +1Mixed-Precision Quantization LLM Quantization
School of Computer Science and Technology, Soochow University, Suzhou, China · Hithink Research, Hangzhou, China · Electronic Engineering, Tsinghua University, Beijing, China
Beyond Dense Adam States: Adaptive Log-Space Quantization for Memory-Efficient Optimizers
Aug 23, 2026Yan WangDeep Learning Optimization Memory-Efficient Optimization
Independent Researcher
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Aug 13, 2026Shuhan Wang, Yilin Luo, Nan Xu +1LLM Quantization 4-Bit Quantization
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
Aug 12, 2026Joao V. Cavalcanti, Ashia C. WilsonLLM Quantization Post-Training Quantization
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology · Cambridge, MA 02139, USA
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
Aug 11, 2026He-Yen Hsieh, H. T. KungLLM Quantization Weight-Only Quantization
Harvard University
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
Aug 10, 2026Achille Jacquemond, Yuma Ichikawa, Akira SakaiLLM Quantization Post-Training Quantization
Fujitsu Limited · RIKEN Center for AIP · Tokai University
Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving
Aug 9, 2026Matteo GrellaTernary Quantization Mixture-of-Experts Inference
Crisis24
LegoLM: Structured Weight Sharing for Large Language Models
Aug 9, 2026Joseph BinghamLLM Quantization LLM Compression
Deparment of Biology, Technion – Israel Institute of Technology
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
Aug 8, 2026Chenxi Zhou, Pengfei Cao, Jinyu Ye +5LLM Quantization Weight-Only Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
Aug 7, 2026Yongge Ma, Guoan Wang, Feiyu Wang +5LLM Quantization Post-Training Quantization
School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University +2
MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model
Aug 7, 2026Zijun Jiang, Yangdi LyuMixed-Precision Quantization Efficient Neural Network Inference
Microelectronics Thrust, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511453, China
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
Aug 7, 2026Xuetian GaoLLM Quantization LLM Inference Acceleration
Which Decisions Low-Bit Quantization Breaks, and How to Predict Them
Aug 6, 2026Zekun Wu, Swati Dhiman, Adriano KoshiyamaPost-Training Quantization Low-Bit Quantization
Holistic AI · UCL Centre for Artificial Intelligence, University College London
BaKron: Efficient Quantization with Kronecker-Factored Hessians
Aug 6, 2026Johann Birnick, Rayan SaabPost-Training Quantization Neural Network Quantization
Department of Mathematics · University of California San Diego · La Jolla, California, United States +1
One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantization
Aug 5, 2026Yuma Ichikawa, Moeto MishimaPost-Training Quantization Neural Network Quantization
Fujitsu Limited, RIKEN for AIP · RIKEN for AIP · Fujitsu Limited
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Aug 4, 2026Seokho Han, Dongwei Wang, Jinhee Kim +4Mixed-Precision Quantization Diffusion Model Quantization
Department of Electrical and Computer Engineering, Sungkyunkwan University, Korea · Department of Electrical and Computer Engineering, University of Arizona, USA · Department of Electrical and Computer Engineering, Duke University, USA +1
NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
Aug 3, 2026Yizhe Chen, Wenshuai Yao, Saiya Wang +6Mixed-Precision Quantization Compute-in-Memory
School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China · School of Integrated Circuits, Peking University, Beijing, China · Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China
Output-Aware Rotation for INT2 KV-Cache Quantization
Aug 3, 2026Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong +4LLM Inference KV Caching
University of Southern California · Seoul National University · Inha University
FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling
Aug 3, 2026Xianglong Yan, Hong Liu, Chengzhu Bao +4LLM Quantization 4-Bit Quantization
Shanghai Jiao Tong University · Tencent
SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits
Aug 1, 2026Kazi Ahmed Asif Fuad, Lizhong ChenKolmogorov-Arnold Networks Neural Network Quantization
Department of EECS Oregon State University Corvallis, OR 97331
MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers
Jul 30, 2026Md. Mehrab Hossain Opi, Robiul Islam Ryad, Md. Umar FarukMixed-Precision Quantization Vision Transformer
Department of Computer Science and Engineering Khulna University of Engineering & Technology (KUET) Khulna 9203, Bangladesh
QuantWAMs: Calibrating at the Right Granularity for World Action Models
Jul 30, 2026Jiacheng Zhou, Jinfan Lv, Ruixuan Li +4World Action Models Post-Training Quantization
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · School of Data Science and Engineering, East China Normal University · Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University
GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Byeongcheol Kim +2AI Accelerator Inference Rotation-Based Quantization
School of Electrical Engineering, Korea Advanced Institue of Science and Technology (KAIST)
Enabling Fully Integer-Only Inference for Lightweight Detection Transformers
Jul 27, 2026Thanh Cong Le, Michal Szczepanski, Martyna PorebaEfficient Transformer Inference Efficient Inference
Université Paris-Saclay, CEA, List, Palaiseau, F-91120, France
MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
Jul 27, 2026Jianlin Yu, Jing Lin, Linghui Kong +13Video Diffusion Models Diffusion Model Quantization
Huawei Technologies Co., Ltd.
When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation
Jul 25, 2026Mojtaba SoltanalianLow-Bit Quantization Neural Network Quantization
Department of Electrical and Computer Engineering University of Illinois Chicago 851 S. Morgan St., MC 154 Chicago, IL 60607, USA
MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
Jul 25, 2026Ashitabh Misra, Madhav Agrawal, Arham Jain +1Mixed-Precision Quantization LLM Quantization
University of Illinois at Urbana-Champaign
KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers
Jul 23, 2026Yann Bouquet, Alireza Khodamoradi, Kristof Denolf +1Diffusion Transformer Diffusion Model Quantization
EPFL, Lausanne, Switzerland. · Advanced Micro Devices, Inc., Longmont, CO, USA.
C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs
Jul 23, 2026Jiameng Li, Han Zhou, Matthew B. BlaschkoVLM Quantization Post-Training Quantization
1KU Leuven · 2Tiangong University
Local Stability and Gaussian Smoothing of Quantized Neural Networks
Jul 22, 2026Sergey Salishev, Anton Makarov, Oleg GranichinNeural Network Activation Functions Neural Network Approximation Theory
St. Petersburg State University, 199034, Saint Petersburg, Russia

Aug 24, 2026, cs.LGJ/K move · Enter open · S save

FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference

Gongwei Lee, Ji Liu, Juncheng Jia, Ji Wu

School of Computer Science and Technology, Soochow University, Suzhou, China · Hithink Research, Hangzhou, China · Electronic Engineering, Tsinghua University, Beijing, China

Mixed-Precision Quantization LLM Quantization LLM Inference Acceleration Weight-Only Quantization Neural Network Quantization

Open paper AskPDF arXiv

Recent years have witnessed remarkable achievements of Large Language Models (LLMs) in multiple domains, while the excessive resource requirements of LLMs hinder the deployment on resource-constrained devices. Although model quantization stands out as an effective approach, conventional quantization approaches typically incur severe performance degradation due to uniform bit-width or simple heuristic sensitivity evaluation. In this paper, we propose a novel Fisher information-based Adaptive Mixed Precision Weight Quantization approach, i.e., FAMPWQ, which performs layer-adaptive weight quantization for effective LLM inference on commodity GPUs. First, we propose a system model with a novel Fisher information metric to measure the layer-wise sensitivity to quantization. Second, we propose a reinforcement learning-based bit-width allocator in FAMPWQ, which generates an adaptive bit-width allocation strategy based on the Fisher information sensitivity metric. Extensive experiments on 7 models and 5 benchmarks demonstrate that FAMPWQ significantly outperforms 7 baseline approaches in terms of PPL (up to 3.39 smaller), accuracy (up to 6.87% higher), and LLM-as-a-judge comparison (up to 76% win rate).

Momentum

Related topics

Latest papers 270