cs.LGNov 12, 2026

SoScale: Second-Order Scale Search for FP4 Quantization

Authors: Pengxiang Zhao, Xing Li, Hui-Ling Zhen, Weizhe Lin, Xianzhi Yu, Zhenhua Dong

Organizations: Huawei Technologies Co., Ltd.

Abstract

Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.

Figures & tables

Explore similar work

CardsList
  1. FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling

    Aug 3, 2026Xianglong Yan, Hong Liu, Chengzhu Bao +4LLM Quantization4-Bit Quantization

  2. SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization

    May 12, 2026Chengzhu Bao, Xianglong Yan, Zhiteng Li +3LLM Quantization4-Bit Quantization

  3. Search Your Block Floating Point Scales!

    May 12, 2026Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu +104-Bit QuantizationLLM Inference Acceleration