LLM Quantization

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

33 papers in the last four weeks, up 43% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 251

All topics

TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
May 19, 2026Zukang Xu, Xing Hu, Dawei YangLLM Quantization Rotation-Based Quantization
1Houmo AI.
A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization
May 18, 2026Lei DongLLM Quantization Ternary Quantization
Independent Researcher
GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
May 18, 2026Zhangyang Yao, Haiyan Zhao, Haoyu Wang +3Mixed-Precision Quantization LLM Quantization
Beihang University · Tsinghua University · ByteDance Inc
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
May 18, 2026Le Su, Xing Luo, Zhi JinLLM Quantization Post-Training Quantization
School of Intelligent Systems Engineering Shenzhen Campus of Sun Yat-sen University Shenzhen, Guangdong 518107, China · Department of Frontier Research Peng Cheng Laboratory Shenzhen, Guangdong 518055, China
Attention Sinks and Outliers in Attention Residuals
May 18, 2026Haozheng Luo, Haoran Dai, Shaoyang Zhang +10LLM Quantization Attention Residual Connections
Department of Computer Science, Northwestern University · Department of Computer Science, Illinois Institute of Technolog · Department of Computer Science and Engineering, University of Michigan +5
WinQ: Accelerating Quantization-Aware Training of Language Models Around Saddle Points
May 17, 2026Dongyue Li, Zechun Liu, Kai Yi +6LLM Quantization Quantization-Aware Training
1Northeastern University, MA · 2Meta AI, CA.
Measuring Maximum Activations in Open Large Language Models
May 15, 2026Luxuan Chen, Han Tian, Xinran Chen +9LLM Quantization Activation Outliers
1Shanghai Jiao Tong University · 2Baidu Inc. · 4Nankai University +1
Widening the Gap: Exploiting LLM Quantization via Outlier Injection
May 14, 2026Xiaohua Zhan, Kazuki Egashira, Robin Staab +2LLM Quantization LLM Backdoor Attacks
ETH Zurich
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
XFP: Quality-Targeted Adaptive Codebook Quantization with Sparse Outlier Separation for LLM Inference
May 14, 2026Thomas WittLLM Quantization Vector Quantization
Gemini Stiftung Leipzig
High-Rate Quantized Matrix Multiplication II
May 13, 2026Or Ordentlich, Yury PolyanskiyLLM Quantization Rate-Distortion Theory
the Hebrew University of Jerusalem, Israel · MIT, USA
Multi-Scale Dequant: Eliminating Dequantization Bottleneck via Activation Decomposition for Efficient LLM Inference
May 13, 2026Lingchao Zheng, Yuwei Fan, Jun Li +5LLM Quantization LLM Inference Acceleration
Huawei
Grid Games: The Power of Multiple Grids for Quantizing Large Language Models
May 12, 2026Vage Egiazarian, Erik Schultheis, Andrei Panferov +3LLM Quantization 4-Bit Quantization
ISTA · Aril Computer · ETH Zürich +1
SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization
May 12, 2026Chengzhu Bao, Xianglong Yan, Zhiteng Li +3LLM Quantization 4-Bit Quantization
Shanghai Jiao Tong University · Tencent Hunyuan
PRISM: A Geometric Risk Bound for Decomposing Drift into Scale, Shape, and Head
May 12, 2026Chieh-Yen Lin, Shao-Hua SunLLM Quantization LLM Evaluation
Appier AI Research · National Taiwan University
Efficient LLM-based Advertising via Model Compression and Parallel Verification
May 12, 2026Wenxin Dong, Chang Gao, Guanghui Yu +9LLM Quantization LLM Compression
Baidu Inc. Beijing, China
ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models
May 11, 2026Ryan Lucas, Mehdi Makni, Xiang Meng +2LLM Quantization Alternating Direction Method of Multipliers
MIT Operations Research Center · MIT Sloan School of Management · MIT Center for Statistics
Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
May 11, 2026Yao Shu, Zilin ZhuLLM Quantization Zeroth-Order Optimization
HKUST-GZ
BCJR-QAT: A Differentiable Relaxation of Trellis-Coded Weight Quantization
May 11, 2026Venugopalan IyengarLLM Quantization Quantization-Aware Training
Theory-optimal Quantization Based on Flatness
May 11, 2026Xiusheng Huang, Zhe Li, Xuanwu Yin +5LLM Quantization Activation Quantization
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2
Pretraining large language models with MXFP4 on Native FP4 Hardware
May 11, 2026Musa Cim, Poovaiah Palangappa, Miro Hodak +3Language Model Pretraining LLM Quantization
1The Pennsylvania State University · 2Advanced Micro Devices, Inc.
Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
May 9, 2026Yuzhuang Xu, Xu Han, Yuxuan Li +2LLM Quantization Post-Training Quantization
1Harbin Institute of Technology, Harbin, China · 2Tsinghua University, Beijing, China
LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Loss
May 9, 2026Euntae Choi, Sumin Song, Sungjoo YooLLM Quantization Quantization-Aware Training
Seoul National University
AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization
May 9, 2026Beshr IslamBouli, David JinLLM Quantization 4-Bit Quantization
Massachusetts Institute of Technology, Cambridge, MA, USA.
Finer is Better (with the Right Scaling)
May 8, 2026Clemens Schaefer, Gil TabakLLM Quantization 4-Bit Quantization
Google LLC, Mountain View, Ca
Litespark Inference For CPUs: Ultra-Fast SIMD Framework for Ternary (1.58-bit) Language Models
May 7, 2026Nii Osae Osae Dade, Tony Morri, Moinul Hossain Rahat +2LLM Quantization Efficient Neural Network Inference
Mindbeam AI
Normalized Architectures are Natively 4-Bit
May 7, 2026Maxim Fishman, Brian Chmiel, Ron Banner +2LLM Quantization Neural Representation Geometry
∧NVIDIA · ◦Department of Electrical and Computer Engineering - Technion, Haifa, Israel
Saliency-Aware Regularized Quantization Calibration for Large Language Models
May 7, 2026Yanlong Zhao, Xiaoyuan Cheng, Huihang Liu +6LLM Quantization Language Model Calibration
University of Science and Technology of China · University College London · 3Shanghai University of Finance and Economics +5
OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization
May 6, 2026Zhikai Li, Zhen Dong, Xuewen Liu +2LLM Quantization Weight-Only Quantization
Institute of Automation, Chinese Academy of Sciences · University of California, Santa Barbara.
QuIDE: Mastering the Quantized Intelligence Trade-off via Active Optimization
May 5, 2026Xiantao JiangMixed-Precision Quantization LLM Quantization
College of Information Engineering, Shanghai Maritime University Shanghai 201306, China

May 19, 2026, cs.LGJ/K move · Enter open · S save

TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization

Zukang Xu, Xing Hu, Dawei Yang

1Houmo AI.

LLM Quantization Rotation-Based Quantization Activation Quantization Post-Training Quantization Neural Network Quantization

Open paper AskPDF arXiv

As Large Language Models (LLMs) advance toward practical deployment, the Microscaling FP4 (MXFP4) format has emerged as a cornerstone for next-generation low-bit inference, owing to its ability to balance high dynamic range with hardware efficiency. However, directly applying MXFP4 to LLM activation quantization inevitably leads to significant accuracy degradation. In this paper, we theoretically analyze the error structure of MXFP4 activation quantization, revealing that the root cause of this performance drop lies in two structural imbalances between activation distributions and the MXFP4 block floating-point format: (1) extreme inter-block variance imbalance and (2) intra-block codebook utilization imbalance. To address these challenges, we propose TORQ (Two-level Orthogonal Rotation for MXFP4 Quantization), a training-free Post-Training Quantization (PTQ) framework designed to reshape the geometric properties of the activation space through optimal coordinate transformations. At the macroscopic level, TORQ leverages the Schur-Horn theorem to redistribute activation energy via inter-block orthogonal rotation, preventing high-variance blocks from driving up shared scaling factors and thereby preserving the precision of small-magnitude elements. At the microscopic level, TORQ employs maximum-entropy-guided intra-block rotation to alleviate codebook collapse and maximize the MXFP4 codebook's information capacity. Experiments on mainstream LLMs such as LLaMA3 and Qwen3 show that TORQ significantly improves the accuracy of MXFP4 activation quantization compared to existing methods: on Qwen3-32B, the perplexity on WikiText is reduced to 8.43 (vs. 7.61 for BF16), and the average accuracy increases from 38.40% with direct RTN to 73.63% (vs. 74.82% for BF16), substantially narrowing the gap between 4-bit floating-point quantization and full-precision inference.

Momentum

Related topics

Latest papers 251