Mixed-Precision Quantization

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 100

All topics

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization
Jun 3, 2026Wanqi Yang, Yuexiao Ma, Alexander Conzelmann +4Mixed-Precision Quantization Mixture-of-Experts Language Models
Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center +5
QuBLAST: A Framework for Quantizing Large Language Models with Block-Level Compression Approach and Activation Scaling Strategy
Jun 3, 2026Pasindu Wickramasinghe, Achyuta Muthuvelan, Rachmad Vidya Wicaksana Putra +2Mixed-Precision Quantization LLM Quantization
eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Mixed-Precision Quantization LLM Quantization
Advanced Micro Devices, Inc. (AMD)
dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats
Jun 2, 2026Giuseppe Franco, Ian Colbert, Pablo Monteagudo-Lago +2Mixed-Precision Quantization LLM Quantization
AMD
LLM Compression with Jointly Optimizing Architectural and Quantization choices
Jun 2, 2026Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi +1Mixed-Precision Quantization LLM Quantization
UiT The Arctic University of Norway · University of Oslo, Norway
QVGGT: Post-Training Quantized Visual Geometry Grounded Transformer
May 29, 2026Zhizhen Pan, Hesong Wang, Huan WangMixed-Precision Quantization 3D ViTs
Westlake University · Beijing University of Posts and Telecommunications
WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization
May 26, 2026Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen +3Mixed-Precision Quantization LLM Quantization
CAIR, VinUniversity, Vietnam
Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training
May 24, 2026Ayush K. Varshney, Konstantinos Vandikas, Šarūnas Girdzijauskas +2Mixed-Precision Quantization Quantization-Aware Training
Ericsson Research, Ericsson AB, Stockholm, Sweden · KTH Royal Institute of Technology, Stockholm, Sweden · RISE Research Institutes of Sweden, Stockholm, Sweden
MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation
May 23, 2026Dahoon Park, Jahyun Koo, Sangwoo Hwang +1Mixed-Precision Quantization AI Accelerator Inference
† School of Electrical Engineering, Korea University, South Korea · ‡ Department of Electrical Engineering and Computer Science, DGIST, South Korea
BitsMoE: Cost-Aware Bit Allocation in Spectral Space for MoE LLM Quantization
May 22, 2026Jiayu Zhao, Zihan Teng, Minhao Fan +4Mixed-Precision Quantization LLM Quantization
School of Microelectronics, University of Science and Technology of China · College of Computing and Data Science, Nanyang Technological University · School of Electrical and Electronic Engineering, Nanyang Technological University
ThriftAttention: Selective Mixed Precision for Long-Context FP4 Attention
May 21, 2026Joe SharrattMixed-Precision Quantization Self-Attention
GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMs
May 21, 2026Jianing Deng, Song Wang, Dongwei Wang +4Mixed-Precision Quantization LLM Quantization
University of Pittsburgh · University of Central Florida · University of Arizona +1
MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization
May 20, 2026Zhong Wang, Zukang Xu, Xing Hu +1Mixed-Precision Quantization Vision-Language Models
Bauman Moscow State Technical University · Houmo AI
ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models
May 19, 2026Arash Akbari, Arman Akbari, Masih Eskandar +11Mixed-Precision Quantization VLM Quantization
1Northeastern University · University of Georgia · 3Cisco Systems
Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
May 19, 2026Jinghe Zhang, Daliang Xu, Chenghua Wang +5Mixed-Precision Quantization LLM Quantization
GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
May 18, 2026Zhangyang Yao, Haiyan Zhao, Haoyu Wang +3Mixed-Precision Quantization LLM Quantization
Beihang University · Tsinghua University · ByteDance Inc
TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
May 16, 2026Hanzhang Shen, Haoran Wu, Yiren Zhao +1Mixed-Precision Quantization LLM Inference Acceleration
University of Cambridge · Imperial College London
When Bits Break Recourse: Counterfactual-Faithful Quantization
May 16, 2026Chaymae Yahyati, Ismail Lamaakal, Khalid El Makkaoui +1Mixed-Precision Quantization Counterfactual Evaluation
Mohammed First University
A Hardware-Aware, Per-Layer Methodology for Post-Training Quantization of Large Language Models
May 14, 2026Earl KillianMixed-Precision Quantization LLM Quantization
LoKA: Low-precision Kernel Applications for Recommendation Models At Scale
May 11, 2026Liang Luo, Yinbin Ma, Quanyu Zhu +21Mixed-Precision Quantization GPU Kernel Optimization
Meta AI
Amortized-Precision Quantization for Early-Exit Vision Transformers
May 8, 2026Rui Fang, Hsi-Wen Chen, Ming-Syan ChenMixed-Precision Quantization Dynamic Neural Networks
National Taiwan University
QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMs
May 5, 2026Pratik Honavar, Tejpratap GVSLMixed-Precision Quantization Multi-Agent LLM Systems
QuIDE: Mastering the Quantized Intelligence Trade-off via Active Optimization
May 5, 2026Xiantao JiangMixed-Precision Quantization LLM Quantization
College of Information Engineering, Shanghai Maritime University Shanghai 201306, China
Performance Optimization and Comparative Analysis of Generative AI Models on Advanced Accelerators
May 5, 2026Amitash Nanda, Javier Hernandez Nicolau, Madhusudan Gujral +3Mixed-Precision Quantization AI Accelerator Inference
University of California San Diego, San Diego, CA, USA · San Diego Supercomputer Center, San Diego, CA, USA
Mixed-Precision Information Bottlenecks for On-Device Trait-State Disentanglement in Bipolar Agitation Detection
May 4, 2026Joydeep ChandraMixed-Precision Quantization Information Bottleneck
Department of Computer Science & Technology, Tsinghua University, Beijing, China.
EdgeLPR: On the Deep Neural Network trade-off between Precision and Performance in LiDAR Place Recognition
May 4, 2026Pierpaolo Serio, Hetian Wang, Zixiang Wei +4Mixed-Precision Quantization Efficient Neural Network Inference
Department of Information Engineering, University of Pisa, Pisa, Italy. · School of Engineering and Material Science, Queen Mary University of London, London, U.K. · Research and Development, Toyota Material Handling Manufacturing Italy, Bologna, Italy.
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
May 4, 2026Wei Tao, Xiaoyang Qu, Peiqiang Wang +4Mixed-Precision Quantization VLM Quantization
Huazhong University of Science and Technology, Wuhan, Hubei, China · Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, Guangdong, China · Tsinghua University, Shenzhen International Graduate School, Shenzhen, Guangdong, China
GETA-3DGS: Automatic Joint Structured Pruning and Quantization for 3D Gaussian Splatting
May 3, 2026Baobing Zhang, Wanxin Sui3DGS Compression Mixed-Precision Quantization
Robotics Research Group, School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield, AL10 9AB, United Kingdom · Department of Electronic and Electrical Engineering, Brunel University London, Kingston Lane, Uxbridge, Middlesex, UB8 3PH, United Kingdom
CoQuant: Joint Weight-Activation Subspace Projection for Mixed-Precision LLMs
Apr 29, 2026Zhe Ding, Su Pan, Duowei PanMixed-Precision Quantization LLM Quantization
School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing, China · Amazon AGI, Seattle, WA, USA
QuantClaw: Precision Where It Matters for OpenClaw
Apr 24, 2026Manyi Zhang, Ji-Fu Li, Zhongao Sun +5Mixed-Precision Quantization LLM Quantization
Huawei Technologies · National University of Singapore · University of Science and Technology of China
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference Efficiency Mixed-Precision Quantization
1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University
On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks
Apr 22, 2026Aarav Gupta, Gururaj Deshpande, Chandreyi ChakrabortyMixed-Precision Quantization Diffusion Model Quantization
College of Computing · Georgia Institute of Technology
LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization
Apr 20, 2026Yann Bouquet, Alireza Khodamoradi, Sophie Yáng Shen +2Mixed-Precision Quantization Diffusion Transformer
Research and Advancement Development Team, Advanced Micro Devices, Inc, USA · Computer Vision Laboratory, Ecole Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland.
Bitwise Systolic Array Architecture for Runtime-Reconfigurable Multi-precision Quantized Multiplication on Hardware Accelerators
Feb 26, 2026Yuhao Liu, Salim Ullah, Akash KumarMixed-Precision Quantization FPGA-Based Neural Network Acceleration
Dresden University of Technology, Germany · Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI Dresden/Leipzig), Germany · Ruhr University Bochum, Germany
Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantization
Dec 8, 2025Haidong Kang, Jun Du, Guo YuMixed-Precision Quantization Large Language Model-Guided Optimization
School of Computer and Communication Engineering, Northeastern University · School of Software Engineering, Beijing Jiaotong University
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
Nov 9, 2025Amit LeVi, Raz Lapid, Rom Himelstein +3Mixed-Precision Quantization LLM Quantization
1Technion—Israel Institute of Technology · 2Intuit · 3Ben-Gurion University of the Negev +1
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision Quantization LLM Quantization
University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1
On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision Quantization LLM Quantization
Southern University of Science and Technology · The University of Hong Kong
Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method
Jul 24, 2025Qingcheng Zhu, Yangyang Ren, Linlin Yang +6Mixed-Precision Quantization LLM Quantization
Beihang University · Communication University of China · Beijing Jiaotong University
Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
Date pendingKexin Chu, Dawei Xiang, Zixu Shen +3Mixed-Precision Quantization GPU Acceleration
University of Connecticut · University of California, Santa Cruz · Independent Researcher

Jun 3, 2026, cs.LGJ/K move · Enter open · S save

AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization

Wanqi Yang, Yuexiao Ma, Alexander Conzelmann, Xiawu Zheng+3

Max Planck Institute for Intelligent Systems · ELLIS Institute Tübingen · Tübingen AI Center+5

Mixed-Precision Quantization Mixture-of-Experts Language Models Mixture-of-Experts Quantization Neural Network Quantization

Open paper AskPDF arXiv

Mixture-of-Experts (MoE) architectures scale model capacity through sparse expert activation, but their deployment remains memory-bound because all expert weights must reside in memory. Mixed-precision quantization can substantially reduce this footprint by assigning different bit-widths to different experts. Existing approaches, however, typically rely on calibration data to estimate expert importance and determine bit allocation. For frontier MoE LLMs, the original training data, and hence the true training distribution, is proprietary and inaccessible. As a result, calibration sets are inevitably imperfect surrogates, and this can misestimate expert utilization and lead to suboptimal bit allocation. Motivated by the substantial cross-expert quality variability observed in modern MoE models, and by the success of Heavy-Tailed Self-Regularization (HT-SR) theory at predicting neural network model quality without access to training or testing data, we propose AlphaQ, a calibration-free bit-allocation method for MoE quantization. AlphaQ draws on HT-SR theory and follows a simple principle: experts with more heavy-tailed weight spectra are typically better trained and hence should receive higher bit-widths, while experts with weaker heavy-tailed structure can be quantized more aggressively. AlphaQ operationalizes this principle by measuring expert-wise spectral heavy-tailedness and solving a budget-constrained optimization problem that minimizes total quantization error under a global bit-budget constraint. Across several MoE models, AlphaQ consistently outperforms calibration-based baselines under matched bit budgets. Notably, on Qwen1.5-MoE, AlphaQ achieves near full-precision accuracy with an average expert precision of only 3.5 bits, while delivering more than 4

\times

memory compression. Our code is available at https://github.com/Superone77/AlphaQ.

Momentum

Related topics

Latest papers 100