LLM Quantization

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

33 papers in the last four weeks, up 43% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 251

All topics

How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine
Aug 22, 2026Shahir M ALLM Quantization AI Accelerator Inference
Independent researcher
In-Cell Learning: Language Models That Update Their Own Weights in Sequence Without Changing the File They Ship
Aug 21, 2026Zifeng Liu, Yaxin Lu, Xuanhan Wu +6LLM Quantization Knowledge Editing
Big Data and Artificial Intelligence Center, The Third Affiliated Hospital of Sun Yat-sen University · Institute for Frontier Interdisciplinary Research in Health Sciences and Technology, Sun Yat-sen University · Sun Yat-sen University Institute of Artificial Intelligence +7
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction
Aug 14, 2026Vincent Counathe, Ben Athiwaratkun, Christopher De Sa +1LLM Quantization Quantization-Aware Training
Cornell University · Together AI
When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation
Aug 13, 2026Shuhan Wang, Yilin Luo, Nan Xu +1LLM Quantization 4-Bit Quantization
SoftWater: Class-Aware Rate Allocation for Softmax Quantization
Aug 12, 2026Joao V. Cavalcanti, Ashia C. WilsonLLM Quantization Post-Training Quantization
Department of Electrical Engineering and Computer Science · Massachusetts Institute of Technology · Cambridge, MA 02139, USA
Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed
Aug 12, 2026Haokun Lin, Kaijie Zhu, Haobo Xu +4LLM Quantization LLM Pruning
Institute of Automation, CAS Beijing, China · Tsinghua University Beijing, China · Harvard Medical School Boston, U.S. +1
Language-Conditional Dequantization: Recovering What Quantization Steals from Non-English Languages
Aug 12, 2026Nirmal ThomasLLM Quantization Post-Training Quantization
Prathama International
ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantization
Aug 11, 2026He-Yen Hsieh, H. T. KungLLM Quantization Weight-Only Quantization
Harvard University
From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization
Aug 10, 2026Achille Jacquemond, Yuma Ichikawa, Akira SakaiLLM Quantization Post-Training Quantization
Fujitsu Limited · RIKEN Center for AIP · Tokai University
LegoLM: Structured Weight Sharing for Large Language Models
Aug 9, 2026Joseph BinghamLLM Quantization LLM Compression
Deparment of Biology, Technion – Israel Institute of Technology
Quantization Degradation in Large Language Models: A Signal-Noise Perspective
Aug 8, 2026Chenxi Zhou, Pengfei Cao, Jinyu Ye +5LLM Quantization Weight-Only Quantization
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
CurveFP: Co-Designing Numerical Representation and Product Arithmetic for Language Models
Aug 8, 2026Ye QiaoLLM Quantization Low-Bit Quantization
Department of Electrical Engineering and Computer Science University of California, Irvine Irvine, CA, USA
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
Aug 7, 2026Yongge Ma, Guoan Wang, Feiyu Wang +5LLM Quantization Post-Training Quantization
School of Computer Science, Peking University · School of Software and Microelectronics, Peking University · School of Physics, Peking University +2
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
Aug 7, 2026Xuetian GaoLLM Quantization LLM Inference Acceleration
Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning
Aug 5, 2026Yue Han, Dianlin WangLLM Quantization Fine-Tuning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
Quantization Effects on Biomedical LLM Reliability
Aug 4, 2026Anton Rasmussen, Hong QinLLM Quantization Text Classification
Old Dominion University Norfolk, VA 23529, USA
Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs
Aug 4, 2026Yu Luo, Bo Dong, Wenhua Cheng +1Mixed-Precision Quantization LLM Quantization
Intel
ARCHead: Activation-Metric Residual Correction for Large Language Model Output Heads
Aug 3, 2026Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali YukselLLM Quantization LLM Compression
Independent Researcher · aiXplain, Inc.
FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling
Aug 3, 2026Xianglong Yan, Hong Liu, Chengzhu Bao +4LLM Quantization 4-Bit Quantization
Shanghai Jiao Tong University · Tencent
Attend to Your Own Thoughts: Breaking the Barrier for Post-Training Quantization of Reasoning LLMs through the Lens of 1.58-Bit Quantization
Aug 2, 2026Shigeng Wang, Chao Li, Yangyuxuan Kang +2LLM Quantization Ternary Quantization
Intel Labs China
Studying quantization trade-offs for efficient inference deployment in machine translation
Jul 31, 2026Jim Zhao, Sohir Maskey, Koen Oostermeijer +2LLM Quantization LLM Inference Acceleration
University of Basel · Aleph Alpha Research · Graphcore
LightRot: A Light-Weighted Rotation Scheme and Architecture for Accurate Low-Bit Large Language Model Inference
Jul 30, 2026Sangjin Kim, Yuseon Choi, Jungjun Oh +2LLM Quantization AI Accelerator Inference
PIM Semiconductor Design Research Center (AI-PIM), Daejeon 34141, South Korea
HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models
Jul 29, 2026Hei Yi Mak, Shadan Golestan, Hoang Le +10LLM Quantization 4-Bit Quantization
Huawei
Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization
Jul 28, 2026Akshay SasiLLM Quantization Memorization in Language Models
Independent Researcher
MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
Jul 25, 2026Ashitabh Misra, Madhav Agrawal, Arham Jain +1Mixed-Precision Quantization LLM Quantization
University of Illinois at Urbana-Champaign
QuantiBias: Benchmarking Quantization-Induced Bias in LLMs
Jul 23, 2026Emilio FerraraOpen-Ended Generation LLM Quantization
Thomas Lord Department of Computer Science University of Southern California
GaugeQuant: Online Learning of Quantization-Optimal Bases from LLM Symmetries
Jul 22, 2026Miguel P. Bento, João F. SeabraLLM Quantization Rotation-Based Quantization
Independent Researcher · Lisbon, Portugal
MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
Jul 20, 2026Simla Burcu Harma, Danila Mishin, Zhengyuan Su +7Mixed-Precision Quantization LLM Quantization
EcoCloud, EPFL · Tsinghua University · MangoBoost Inc. +2
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1LLM Quantization Mixture-of-Experts Language Models
University of Illinois Urbana-Champaign
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Jul 17, 2026Zhengyang Zhuge, Hao Yu, Xin Wang +4LLM Quantization RL for Language Models
Qwen Team, Alibaba Inc.

Aug 22, 2026, cs.LGJ/K move · Enter open · S save

How Weight Encoding Affects Language Model Placement and Performance on the Apple Neural Engine

Shahir M A

Independent researcher

LLM Quantization AI Accelerator Inference On-Device Language Model Inference LLM Inference Acceleration

Open paper AskPDF arXiv

Weight compression can alter accelerator placement as well as memory traffic, complicating the interpretation of inference speedups. We investigate this interaction on the Apple Neural Engine through the public Core ML deployment path. Five independently trained language-model checkpoints span two architectures and dense fp16, int8, and ternary weights encoded with two-bit lookup tables. We combine compiler device plans, synchronized memory-controller measurements, and compute-unit exclusion controls for a fixed single-token forward workload. On an M1, the smaller fp16 export executes on the CPU despite permitting ANE execution, whereas its compressed counterparts exhibit ANE activity. The int8 export reduces warm forward latency by a factor of 1.9. The larger fp16 export also uses the ANE, indicating that dense encoding alone does not determine placement. Separate M3 energy measurements support the same direction of change. These results establish encoding-dependent placement in the measured deployment stack and show that compression comparisons require joint measurement of backend selection, latency, and traffic.

Momentum

Related topics

Latest papers 251