4-Bit Quantization

Follow this topic Ask about this topic

Four-bit quantization studies how to run or train neural networks with compact INT4 or FP4 values, reducing memory use and data movement for deployment. Post-training methods use calibration, learned scales or codebooks, rotations, and outlier or residual compensation; quantization-aware training instead adapts model parameters to low precision during training. Work spans large language models, vision and diffusion models, and inference components such as attention and KV caches, with evaluations measuring accuracy alongside latency, throughput, and memory. Many systems now approach higher-precision quality in specific settings, but results remain sensitive to activation outliers, calibration data, model architecture, and hardware format, so robustness and practical speedups are still key challenges.

Latest papers

91 papers14 papers in the last four weeks, up 56% on the four weeks before. 0.1% of all new papers.

PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States
Oct 21, 2026Patrick Haller, Filipe Laitenberger, Alan AkbikRecurrent-State Quantization LLM Quantization
Humboldt-Universität zu Berlin
PACT: NVFP4 Data-Flow LoRA Fine-Tuning on Consumer GPUs
Oct 18, 2026Lei Gao, Hossein Entezari Zarch, Chaoyi Jiang +1Memory-Efficient Fine-Tuning Parameter-Efficient Fine-Tuning
University of Southern California
What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes
Oct 17, 2026Anjulie Agrusa, Andrei Panferov, Elizabeth Wei +7Low-Precision Training Language Model Pretraining
NVIDIA · Institute of Science and Technology Austria · Red Hat AI
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Oct 15, 2026Sunghyun Wee, Geunjae Choi, Hyeonjin Kim +2LLM Quantization Neural Network Quantization
Seoul National University · LG Electronics
Mixed Precision Scaling Laws for Quantized Pretraining
Oct 13, 2026Jinuk Kim, Rajarshi Saha, Muhammed Fatih Balin +1Mixed-Precision Quantization LLM Quantization
Seoul National University · Amazon Annapurna Labs
When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limits
Oct 12, 2026Tianhao GuoQuantization Error Compensation Neural Network Quantization
College of Physics and Electronic Engineering, Shanxi University, Taiyuan 030006, China
Error-Based Bidirectional Predictive Coding Enables Deep, 4-Bit Predictive Coding Networks
Oct 11, 2026Garrett Kinman, Zeljko ZilicPredictive Coding 4-Bit Quantization
Department of Electrical and Computer Engineering, McGill University, Montréal, Québec, Canada
Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
Oct 8, 2026Hanyang Li, Shao Tang, Daniel Thomas Braithwaite +7Stochastic Rounding 4-Bit Quantization
University of California, Berkeley · Nubank
Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers
Oct 8, 2026Shiwen Wang, Pengxiang Zhao, Xiaoming YuanDiffusion Model Quantization Post-Training Quantization
The University of Hong Kong · Huawei Technologies Co., Ltd.
OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization
Oct 6, 2026Neriah Ben David, Ori Meir, Or OrdentlichLLM Quantization Neural Network Quantization
School of Computer Science and Engineering Hebrew University of Jerusalem Jerusalem, Israel
LatentQuant: Preserving the Policy-Facing Latent Contract under NVFP4 VAE Quantization
Oct 2, 2026Ziye Deng, Lufang Chen, Shuyu Feng +7Neural Network Quantization 4-Bit Quantization
UC San Diego · XSquare Robot · Sun Yat-Sen University +2
SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models
Sep 30, 2026Yunhan Wang, Haodong Wang, Zhiming Liu +74-Bit Quantization Efficient Inference for World Action Models
Tongji · HKUST · HIT +1
Replay the Curvature: Accurate and Scalable NVFP4 Quantization for Large Language Model Inference
Sep 29, 2026Ruiyi Ding, Jie Li, Kang He +4LLM Quantization Post-Training Quantization
KlingAI Research · University of Science and Technology of China · Fudan University +2
QuantForge: Discovering Residual Decompositions for MXFP4 Post-Training Quantization
Sep 28, 2026Qiulin Shang, Zhoutong Wu, Jie Hu +1LLM Quantization Large Language Model-Guided Optimization
Peking University
JustQuant: You Don't Need Smoothing, SVD, or Rotation for 4-Bit Activation Quantization
Sep 27, 2026Kaicheng Yang, Kaisen Yang, Chunyu Liu +8Neural Network Quantization Diffusion Model Quantization
Shanghai Jiao Tong University · Tsinghua University · The Hong Kong Polytechnic University
PulseQuant: Propagation-Guided Subspace Correction for 4-Bit Video Diffusion Transformers
Sep 27, 2026Yutong Wang, Xingtong Ge, Enhuai Liu +4Video Diffusion Models Diffusion Transformer
The University of Sydney · The Hong Kong University of Science and Technology · The Chinese University of Hong Kong +1
Q-WAM: 4-Bit Quantization of World Action Models with Action-Subspace Protection
Sep 27, 2026Arash Akbari, Arman Akbari, Jingwu Luo +7Mixed-Precision Quantization 4-Bit Quantization
Northeastern University · EmbodyX · University of Minnesota - Twin Cities +1
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
Sep 26, 2026Yanlong Chen, Yining Chen, Song Zhang +2LLM Quantization Rotation-Based Quantization
Nanyang Technological University, Singapore · Nanjing University of Posts and Telecommunications, Nanjing, China · Nanjing University, Nanjing, China +1
Global Ranks Survive, Selected Heads Shift: BOS-Sink Topology under 4-bit Weight-Only Quantization
Sep 20, 2026Kuanlin Chen, Chen-Wei Kuo, Cheng-En OuAttention Head Analysis 4-Bit Quantization
Independent Researcher Taoyuan, Taiwan · National Tsing Hua University Hsinchu, Taiwan
MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration
Sep 17, 2026Yuan Liao, Jae-sun SeoAI Accelerator Inference VLM Quantization
Cornell Tech, Cornell University New York, NY, USA
KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
Sep 8, 2026Lexington Whalen, Yuki Ito, Ryo Sakamoto4-Bit Quantization Neural Network Quantization
1SB Intuitions, Tokyo, Japan.
Scale-QLoRA: Code-Invariant Adapter Merging for Native 4-bit Microscaling LLMs
Sep 3, 2026Tung-Ling Li, Jiale Huang, Lee-Chi Wang +1Model Merging LLM Quantization
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Sep 3, 2026Sergii Kozyrev, Davyd MaiborodaLLM Quantization 4-Bit Quantization
Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs
Aug 27, 2026Tanzila Rahman, Mehran Taghian Jazi, Yunke Peng +10Activation Quantization Low-Bit Quantization
Huawei
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning
Aug 24, 2026Md Romyull IslamFine-Tuning Quantization-Aware LoRA
Kennesaw State University
PRQuant: Permutation Residual Quantization for Low-Overhead Inference
Aug 16, 2026Peiran Wang, Anqi Wang, Jiaying Zhao +10LLM Quantization Low-Bit Quantization
Huawei Technologies Co., Ltd. · Tsinghua University. · Nanjing University of Science and Technology.
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Aug 13, 2026Shuhan Wang, Yilin Luo, Nan Xu +1LLM Quantization 4-Bit Quantization
NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Space
Aug 13, 2026Eleftherios Mylonas, Angelos Kouprizas, Michael Birbas +14-Bit Quantization Hardware-Aware NAS
Dept. of ECE University of Patras Patras, Greece
Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecasting
Aug 12, 2026Junyi Ye, Ivy Gateri WanjikuFinancial Forecasting 4-Bit Quantization
School of Computing, Montclair State University · Montclair, New Jersey, USA
Cosine Similarity Is Not Evidence: Measuring the Noise Floor of Interpretability Transfer Under Quantization
Aug 12, 2026Pranav VarshneyLLM Interpretability Low-Bit Quantization
University of Michigan

Oct 21, 2026J/K move · Enter open · S save

PACE: Periodic and Channel-Aligned Encoding of Four-Bit Recurrent States

Patrick Haller, Filipe Laitenberger, Alan Akbik

Humboldt-Universität zu Berlin

Recurrent-State Quantization LLM Quantization Neural Network Quantization 4-Bit Quantization Post-Training Quantization Mixed-Precision Quantization

Open paper AskPDF arXiv

Linear-attention language models keep their context in a recurrent state of fixed size, where an inference system stores one copy of this state for every sequence it decodes at the same time. Storing the state at lower precision therefore lets more sequences share an accelerator. Existing state quantizers round the state after every generated token and report that four bits lose accuracy unless the bit width is raised or the model is fine-tuned.\par We introduce PACE, a post-hoc four-bit state quantizer. PACE rounds the state only once every few generated tokens, as chunked kernels already do once per chunk during prefill, and keeps the tokens generated in between in a small buffer. Rounding less often reduces the error that accumulates in the state. Additionally, we find that the large entries of the gated-delta state gather in columns, because some channels of the update written at each token remain large across many tokens. PACE therefore shares each quantization scale within one column. It also keeps the heads that retain rounding error longest at eight bits. On four hybrid models from 4B to 35B parameters, PACE with mixed four- and eight-bit heads is competitive with eight-bit DAMP on the three larger models at about two thirds of its memory, PACE with four-bit heads throughout scores higher than Q-Mamba at equal memory, and rounding every few tokens preserves the decoding speedup of the buffer.