Neural Network Quantization

Follow this topic Ask about this topic

Momentum

36 papers in the last four weeks, up 125% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 270

All topics

Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization
Sep 27, 2026I Kennedy, T KennedyMixed-Precision Quantization LLM Quantization
baa.ai Auckland, New Zealand
JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization
Sep 27, 2026Kaicheng Yang, Kaisen Yang, Chunyu Liu +8Diffusion Model Quantization Quantization-Aware Training
Shanghai Jiao Tong University · Tsinghua University · The Hong Kong Polytechnic University
Chameleon: Dynamic Format Adapter for Efficient Diffusion
Sep 27, 2026Arnab Sanyal, Sandeep ChinchaliDiffusion Model Quantization Post-Training Quantization
UT SWARM Lab, Department of Electrical & Computer Engineering The University of Texas, Austin TX 78712
PulseQuant: Propagation-Guided Subspace Correction for 4-Bit Video Diffusion Transformers
Sep 27, 2026Yutong Wang, Xingtong Ge, Enhuai Liu +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong +1
Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement
Sep 24, 2026Clément Laroche, Rasmus Kongsgaard OlssonEfficient Neural Network Inference Speech Enhancement
GN A/S, Audio Research, Lauptrupbjerg 7, 2750, Denmark
Predicting Quantization Price for Selecting PTQ Configurations Before Deployment
Sep 23, 2026Junbin Qiu, Jian Mu, Weitong Zhang +1Post-Training Quantization Neural Network Quantization
RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models
Sep 23, 2026David Población-Criado, Dario Garcia-Gasulla, Eduardo QuinonesMixed-Precision Quantization Edge Inference
Barcelona Supercomputing Center (BSC) · Universitat Politècnica de Catalunya - BarcelonaTech (UPC)
Disaggregated Quantization: Specializing LLM Prefill and Decode
Sep 22, 2026Andrei Panferov, Maximilian Kleinegger, Sweta Priyadarshi +2LLM Quantization LLM Inference
NVIDIA · ISTA
Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
Sep 22, 2026Akihiro Yoshida, Yuma IchikawaMixed-Precision Quantization LLM Quantization
Fujitsu Limited, Institute of Science Tokyo · Fujitsu Limited, RIKEN Center for AIP
When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMs
Sep 21, 2026Yeji Kim, Mi-Young Kim, Randy GoebelLLM Quantization Faithfulness of Language Model Explanations
University of Alberta
On the Efficiency-Safety Dilemma in Large Reasoning Models
Sep 20, 2026Yifei Yang, Zouying Cao, Xingrui Wang +4LLM Inference Acceleration LLM Jailbreak Attacks
AGI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University · Ant Group +2
Predict Before You Deploy: Offline Prediction of Quantization-Induced Task Degradation for World Action Models
Sep 16, 2026Jiuyi Xu, Jinjia Guo, Meida Chen +2World Action Models Post-Training Quantization
Robotics Program, Colorado School of Mines, Golden, CO USA. · Department of Civil and Coastal Engineering, University of Florida, Gainesville, FL USA. · University of Southern California - Institute for Creative Technology, Los Angeles, CA USA. +1
Colla-Q: Toward Collaborative Experts in MoE Quantization via Minimax Precision Balancing
Sep 16, 2026Eunju Shin, Jongbin RyuMixed-Precision Quantization Mixture-of-Experts Quantization
Ajou University, South Korea
Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Duality
Sep 15, 2026Jonghyeon Lim, Changhoon YimVisual SSMs Activation Quantization
Intelligent Image Processing Laboratory · Department of Computer Science and Engineering, Konkuk University · Seoul, Republic of Korea
Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
Sep 14, 2026Hyojung HanText Embedding Evaluation Weight-Only Quantization
ThakiCloud
Efficient AI Model Deployment Using Quantization Analysis Tool
Sep 14, 2026Dwith Chenna, Kanishka MacherlaEfficient Neural Network Inference Neural Network Quantization
Electrical and Computer Engineering Department University of Maryland College Park, MD, USA
Attention Quantization for Tabular Foundation Models
Sep 14, 2026Jonas M. Kübler, Benjamin Jäger, Klemens Flöge +2Efficient Transformer Inference Tabular Foundation Models
Prior Labs
Why Does Post-Training Quantization Work?
Sep 10, 2026Yuxiang Chen, Michael Beyer, Jun Zhu +1LLM Quantization Post-Training Quantization
Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University · College of AI, Tsinghua University · Bosch AI Research, Renningen, Germany
KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
Sep 8, 2026Lexington Whalen, Yuki Ito, Ryo Sakamoto4-Bit Quantization Neural Network Quantization
1SB Intuitions, Tokyo, Japan.
When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
Sep 7, 2026Ismail Erbas, Xavier Intes, Vikas PandeyRecurrent Neural Networks Post-Training Quantization
Department of Biomedical Engineering Center for Modeling, Simulation, and Imaging in Medicine Rensselaer Polytechnic Institute Troy, NY 12180
DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation
Sep 3, 2026Shuaiting Li, Zelin Gao, Haibin Shen +3Diffusion Model Guidance Video Diffusion Models
1Robbyant · 2ZJU · 4HKUST +1
A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization
Sep 2, 2026Xianghong Fang, Wenlong Mou, Yuan Yuan +2Vector Quantization Product Quantization
University of Toronto · 2Boston College · 3Vijil
The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globally
Sep 1, 2026Jundong Hu, Shekar RamachandranMixed-Precision Quantization LLM Quantization
PayPal AI
A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies
Sep 1, 2026Ahmad Alfan Alfian Irfan, Nur Ahmad Khatim, Mansur AriefAutonomous Driving Safety Evaluation Knowledge Distillation
Information Technology, Universitas Muhammadiyah Yogyakarta, Indonesia · Division of Information Science, Nara Institute of Science and Technology (NAIST), Japan · Industrial and Systems Engineering (ISE) Department and IRC for Smart Mobility and Logistics (IRC-SML), King Fahd University of Petroleum and Minerals (KFUPM), Saudi Arabia
HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inference
Aug 31, 2026Chun-Ting Chen, Dongmin Han, Hangyeol Mun +6LLM Quantization LLM Inference Acceleration
Cornell University New York, NY, USA · Intel Corporation Santa Clara, CA, USA · Intel Corporation Hillsboro, OR, USA
Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMs
Aug 31, 2026Deokjae Lee, Sihun Chu, Hyun Oh SongMixed-Precision Quantization LLM Quantization
Department of Computer Science and Engineering, Seoul National University · Neural Processing Research Center · 3Neural Processing Research Center +2
TopGQ: Fast GNN Post-Training Quantization Leveraging Topology Information
Aug 31, 2026Dain Kwon, Kanghyun Choi, Hyeyoon Lee +4Graph Neural Networks Activation Quantization
Seoul National University Seoul, South Korea
A Target-Centric Survey of Quantization-Aware Training
Aug 30, 2026Jiamin Song, Mengjie Zhao, Zijing Wang +6LLM Quantization Quantization-Aware Training
1Northeastern University, China · 2Shandong University, China · 3CIS, LMU Munich; MCML, Germany
HyQuant: Hybrid-Precision Quantization for LLM Attention
Aug 28, 2026Jiatong Ding, Bingxin Xing, Yu Zhang +9Mixed-Precision Quantization LLM Quantization
Shanghai Jiao Tong University · Xi’an Jiaotong University · Xiamen University +1
DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization
Aug 27, 2026Tao Zhang, Jianchao Tan, Pingwei Sun +6Mixed-Precision Quantization LLM Quantization
South China University of Technology · Meituan · East China Normal University

Sep 27, 2026, cs.CLJ/K move · Enter open · S save

Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization

I Kennedy, T Kennedy

baa.ai Auckland, New Zealand

Mixed-Precision Quantization LLM Quantization Post-Training Quantization Neural Network Quantization

Open paper AskPDF arXiv

A single random Gaussian probe gives an unbiased estimate of the squared Frobenius norm of a layer's quantization error. The estimator is well-behaved because round-to-nearest error is spectrally flat. Across 1,683 tensors from a 35B MoE and a 9B dense model, effective dimensionality is 0.93 to 0.96 times the i.i.d. noise value of the same shape, and on the MoE the median is unchanged from 2-bit to 8-bit. The probe coefficient of variation is predictable from tensor shape. One probe measures per-tensor sensitivity to within 4 to 7%; twenty probes reach 1.3 to 1.4%.RAM applies the propagated form of this estimator to budget-targeted mixed-precision quantization with no calibration data. Gaussian probes carrying the network's own input statistics score every tensor at six bit-widths. A knapsack solver allocates bits under an exact byte budget, with guardrails against catastrophic 2-bit assignments. One probe pass serves any budget. Isolated and propagated scores rank tensors independently on Qwen3.5-35B-A3B (Spearman -0.01), yet the propagated probe rank-correlates 0.81 to 0.83 with the GPTQ layer objective from real activations, while the isolated estimator is uncorrelated with it. That objective is the wrong allocation target: at matched bytes on Qwen3.8-27B, a block-output probe beats a vendor IQ3_M mix and an oracle that allocates from the real-activation objective. On Qwen3-8B the propagated probe ties HAWQ-V2 at matched bytes. Across seven architectures from 8B to 122B, with probe timing up to a 400B model in nine minutes on one workstation, RAM reaches 3.5 to 13.6% lower median WikiText-2 perplexity than size-comparable uniform 4-bit builds on the tested MoE models. (Black Sheep Ai baa.ai)

Momentum

Related topics

Latest papers 270