Activation Quantization

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 76

All topics

Adaptive Joint Compression and Synchronisation in Federated Split Learning for IoT Rainfall Prediction
Jun 23, 2026Wenjie Ding, Yi Sin Lin, Jiale Liu +8Edge Computing Split Federated Learning
School of Computing · Newcastle University · Newcastle upon Tyne, United Kingdom
ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers
Jun 20, 2026Changjun Li, Runqing Jiang, Lian Xu +3Vision Transformer Activation Quantization
School of Electronics and Communication Engineering, Shenzhen Campus, Sun Yat-sen University, Shenzhen, China · Computer Science and Software Engineering, The University of Western Australia, Perth, Australia · University of Oxford, Oxford, United Kingdom
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training
Jun 14, 2026Janghwan Lee, Sihwa Lee, Jinseok Kim +4LLM Quantization Quantization-Aware Training
1Hanyang University, 2Rebellions Inc., Republic of Korea.
TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization
Jun 11, 2026Zhixiong Zhao, Zukang Xu, Zhixuan Chen +3Mixed-Precision Quantization LLM Quantization
Houmo AI, China · 2Southeast University, Nanjing, China
Holding the FP8 Quality Ceiling at 8-Bit Weights and Activations: INT8 and GGUF Post-Training Quantization of Ideogram 4.0 for Consumer GPUs
Jun 10, 2026Deep Gandhi, Ali Asaria, Tony SalomoneMixed-Precision Quantization Diffusion Transformer
Transformer Lab
DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics
Jun 10, 2026Zimo Zhao, Maolin Wang, Bowen Yu +3Mixed-Precision Quantization LLM Quantization
City University of Hong Kong · Hong Kong, China · Zhejiang University of Technology +1
Trainable Smooth-Rotation Transforms with Learned Channel Scales for LLM Quantization
Jun 7, 2026Patrik Czakó, Gábor Kertész, Sándor SzénásiLLM Quantization 4-Bit Quantization
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University Budapest, Hungary · John von Neumann Faculty of Informatics, Obuda University Budapest, Hungary
OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
Jun 5, 2026Haoqi Wang, Lorenz K. Mueller, Jiawei Zhuang +2LLM Quantization 4-Bit Quantization
School of Computer and Communication Sciences, EPFL, Switzerland · 2Huawei, Switzerland · 3Swiss Data Science Center, ETHZ & EPFL, Switzerland
STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models
Jun 3, 2026Xin Yan, Aqiang Wang, Zhenglin Wan +2LLM Quantization Diffusion Model Quantization
School of Artificial Intelligence, Beijing Normal University, Beijing, China · Department of Computer Science, National University of Singapore, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore
Characterizing the Impact of NVFP4 Quantization for Low-Power Edge AI Deployment
Jun 3, 2026Ovishake Sen, Venkata Nithin Kamineni, Daniel Lobo +3Efficient Neural Network Inference Edge Inference
Department of Electrical and Computer Engineering, University of Florida, Gainesville, USA
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
Jun 3, 2026Yue Wu, Changyuan Wang, Zixuan Wang +2LLM Quantization 4-Bit Quantization
Massive Spikes in LLMs are Bias Vectors: Mechanistic Uncovering and Spike-Free Quantization
Jun 1, 2026Yung-Chin Chen, Chung Peng Lee, Ze-Wei Liou +1LLM Quantization Activation Outliers
Princeton University, NJ, USA · EnCharge AI, CA, USA
ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
May 30, 2026Li Lin, Xiaojun WanLLM Quantization 4-Bit Quantization
Wangxuan Institute of Computer Technology, Peking University
Ω-QVLA: Robust Quantization for Vision-Language-Action Models via Composite Rotation and Per-step Scaling
May 27, 2026Xinyu Wang, Mingze Li, Sicheng Lyu +6VLM Quantization Efficient VLA Model Inference
McGill University · Université de Montréal · Mila – Quebec AI Institute +2
Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V
May 26, 2026Junhao Wu, Dezhong Yao, Hai JinVideo Diffusion Models Diffusion Transformer
School of Computer Science and Technology, Huazhong University of Science and Technology Wuhan, Hubei, China 430074
Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2
May 26, 2026Zhanfeng Feng, Shuai Guo, Xin Di +3Video Diffusion Models 4-Bit Quantization
University of Science and Technology of China Hefei, China
The Rescue Effect: Spatio-Semantic Early Exit Bypasses Quantization Collapse in CLIP
May 26, 2026Kahyeon Nam, Hyesong ChoiVLM Quantization Activation Quantization
Soongsil University Seoul, Republic of Korea
The Quantization Benefits of Residual-Free Transformers
May 25, 2026Yiping Ji, Mahalakshmi Sabanayagam, Peyman Moghadam +2Transformer Activation Outliers
Australian Institute for Machine Learning, Adelaide University · DATA61, CSIRO
Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training
May 25, 2026Yingying Cheng, Jinquan Shi, Li Zhou +4LLM Quantization Quantization-Aware Training
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
May 25, 2026Ke Li, Dong An, Xiaoling Zang +6LLM Quantization Activation Quantization
School of Software Technology, Zhejiang University · Ant Group · College of Computer Science and Technology, Zhejiang University of Technology +3
Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training
May 24, 2026Ayush K. Varshney, Konstantinos Vandikas, Šarūnas Girdzijauskas +2Mixed-Precision Quantization Quantization-Aware Training
Ericsson Research, Ericsson AB, Stockholm, Sweden · KTH Royal Institute of Technology, Stockholm, Sweden · RISE Research Institutes of Sweden, Stockholm, Sweden
TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization
May 19, 2026Zukang Xu, Xing Hu, Dawei YangLLM Quantization Rotation-Based Quantization
1Houmo AI.
StatQAT: Statistical Quantizer Optimization for Deep Networks
May 18, 2026Mehmet Aktukmak, Daniel Huang, Ke DingQuantization-Aware Training Activation Quantization
Intel
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
May 16, 2026Sayeh Sharify, Mahsa Salmani, Hesham MostafaDiffusion Transformer Diffusion Model Quantization
d-Matrix Santa Clara, CA 95054
Measuring Maximum Activations in Open Large Language Models
May 15, 2026Luxuan Chen, Han Tian, Xinran Chen +9LLM Quantization Activation Outliers
1Shanghai Jiao Tong University · 2Baidu Inc. · 4Nankai University +1
Grid Games: The Power of Multiple Grids for Quantizing Large Language Models
May 12, 2026Vage Egiazarian, Erik Schultheis, Andrei Panferov +3LLM Quantization 4-Bit Quantization
ISTA · Aril Computer · ETH Zürich +1
A Composite Activation Function for Learning Stable Binary Representations
May 12, 2026Seokhun Park, Choeun Kim, Kwanho Lee +3Concept Bottleneck Models Binary Neural Networks
Department of Statistics, Seoul National University · Department of Applied Mathematics, University of Twente
ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
May 11, 2026Chayne Thrash, Ali Abbasi, Soheil KolouriRotation-Based Quantization LLM Inference Acceleration
Department of Computer Science Vanderbilt University
Theory-optimal Quantization Based on Flatness
May 11, 2026Xiusheng Huang, Zhe Li, Xuanwu Yin +5LLM Quantization Activation Quantization
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Academy of Artificial Intelligence +2
LoopQ: Quantization for Recursive Transformers
May 8, 2026Rui Fang, Hsi-Wen Chen, Ming-Syan ChenRecurrent Transformers Language Modeling
National Taiwan University

Jun 23, 2026, cs.LGJ/K move · Enter open · S save

Adaptive Joint Compression and Synchronisation in Federated Split Learning for IoT Rainfall Prediction

Wenjie Ding, Yi Sin Lin, Jiale Liu, Baoyi Liu+7

School of Computing · Newcastle University · Newcastle upon Tyne, United Kingdom

Edge Computing Split Federated Learning Activation Quantization Federated Learning

Open paper AskPDF arXiv

Federated split learning (FSL) enables collaborative training across bandwidth-constrained IoT devices, but repeated activation and gradient exchange creates a communication bot-tleneck. Prior work optimises either activation compression or synchronisation frequency in isolation. This paper presents an FSL framework for IoT rainfall prediction that jointly regulates activation compression and the synchronisation interval \r{ho} via a latency driven scheduler on a server with per client EMA smoothing. The system is evaluated on hourly ERA5 data from 11 weather stations through a 17 scenario simulation matrix and a four scenario Raspberry Pi deployment over a real wide-area link. The simulation matrix validates scheduler switching across low, high, and mixed latency profiles, while the Pi deployment validates the high latency endpoint selected by the same policy. AUPRC varies only slightly across configurations (0.6381-0.6484 in simulation; within 0.011 on Pi), indicating that aggressive quantisation and sparser aggregation do not materially degrade predictive quality in this setting. On Pi, the selected endpoint (int8 with rho=3) achieves an 87% reduction in activation upload payload and a 54% reduction in synchronisation traffic relative to the float32 baseline, while reducing runtime jitter from +/-688 s to +/-10 s.

Momentum

Related topics

Latest papers 76