Mixed Precision Scaling Laws for Quantized Pretraining

Jinuk Kim; Rajarshi Saha; Muhammed Fatih Balin; Youngsuk Park

cs.LGOct 13, 2026

Mixed Precision Scaling Laws for Quantized Pretraining

Authors: Jinuk Kim, Rajarshi Saha, Muhammed Fatih Balin, Youngsuk Park

Organizations: Seoul National University · Amazon Annapurna Labs

Abstract

Since scaling laws were introduced for modern LLMs (Kaplan et al., 2020), several works have extended them to account for quantization, tracking the emerging low-precision data types supported on AI accelerators. We develop a more accurate scaling law for quantized pretraining with mixed precision across the forward and backward passes, grounded in optimization theory and validated empirically. Our formulation separates the effects of quantization on model capacity and on optimization efficiency, and explicitly models the persistent optimization error from forward and backward quantization identified through convergence analysis. To select the regimes for fitting and evaluating our law, we first decompose existing 4-bit training recipes into their forward and backward components, isolating those that most affect loss. Our Qwen3 experiments under selected 4-bit regimes, together with published Llama-style experiments, show that our law predicts loss with up to $11\times$ lower MSE than existing laws as model size and token count scale. Finally, we extend our law to layer-wise mixed precision and identify the optimal quantized-pretraining recipe, i.e., model size, training tokens, and per-layer forward/backward precision, under a hardware-aware compute budget. Unlike prior precision scaling laws, our law predicts that the optimal precision increases with the compute budget.

Figures & tables

Figure 1: Compute-optimal precision allocation predicted by our scaling law. With each Transformer block using MXFP8 or NVFP4 in each pass, the panels show the optimal fraction of NVFP4 blocks in the forward pass $p_{f}^{*}$ (top, left) and backward pass $p_{b}^{*}$ (top, right), the optimal model size $N^{*}$ (bottom, left), and tokens per parameter $D^{*}/N^{*}$ (bottom, right), given the compute budget ( $x$ -axis) and the NVFP4 speedup over BF16 ( $y$ -axis), with the MXFP8 speedup fixed at $1.8\times$ (Section 4.3 ).

Panel of Figure 4 — Figure 4: Predicted versus observed loss of layer-wise mixed-precision runs on the held-out 910M model, with $\gamma_{S}$ fitted on the three smaller models. The endpoints $L_{1}$ and $L_{2}$ in Eq. 2 are predicted by our law (left) or the Quartet law (right). Markers indicate the mixed pass and the blocks kept in higher precision, and each point corresponds to one fraction $p$ and one token budget.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Quartet grid (Llama), model size extrapolation, for (a) the Quartet law and (b) our law. Each law is fitted to the 30M–100M models and evaluated on the held-out 200M models.

Figure 6: Quartet grid (Llama), data extrapolation, for (a) the Quartet law and (b) our law. Each law is fitted to 25–400 tokens per parameter and evaluated at the held-out 800 tokens per parameter.

Figure 7: Our grid (Qwen3), model size extrapolation, for the Quartet law. The law is fitted to the 280M–640M models and evaluated on the held-out 910M models.

Figure 8: Our grid (Qwen3), model size extrapolation, for our law. The law is fitted to the 280M–640M models and evaluated on the held-out 910M models.

Figure 9: Our grid (Qwen3), data extrapolation, for the Quartet law. The law is fitted to all but the largest token budget and evaluated at the held-out 35.8B tokens.

Figure 10: Our grid (Qwen3), data extrapolation, for our law. The law is fitted to all but the largest token budget and evaluated at the held-out 35.8B tokens.

Figure 11: Normalized increase in loss of layer-wise mixed-precision runs versus the fraction

p

of blocks in NVFP4, with endpoints

L_{1}

and

L_{2}

predicted by (a) the Quartet law and (b) our law. Solid curves show

p^{\gamma_{S}}

with

\gamma_{S}

fitted on the calibration runs, and dotted lines show linear interpolation (

\gamma_{S}=1

Panel of Figure 12 — Figure 12: Compute-optimal precision allocation predicted by our scaling law, as in Figure 1 , with the MXFP8 speedup over BF16 set to (a) $1.6\times$ and (b) $1.4\times$ instead of $1.8\times$ .

Mixed-Precision Quantization

Mixed-precision quantization assigns different numeric precisions across a model to balance accuracy with memory, speed, energy, and hardware constraints.

See all 159 papers

Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Oct 6, 2026Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2LLM Quantization Quantization-Aware Training
ISTA · EPFL, ISTA
MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
Jul 25, 2026Ashitabh Misra, Madhav Agrawal, Arham Jain +1Mixed-Precision Quantization LLM Quantization
University of Illinois at Urbana-Champaign
Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization
Jul 1, 2026Fei Wang, Chao Xue, Taoran Liu +3Mixed-Precision Quantization LLM Quantization
South China University of Technology · Ocean University of China · Shenzhen Campus of Sun Yat-sen University +1

Oct 6, 2026J/K move · Enter open · S save

Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training

Alexandra Volkova, Matin Ansaripour, Erik Schultheis, Christoph H. Lampert+1

ISTA · EPFL, ISTA

LLM Quantization Quantization-Aware Training Quantization Mixed-Precision Quantization Adaptive Quantization Low-Precision Training+2

Open paper AskPDF arXiv

Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common remedy is mixed-precision training, in which high precision is assigned to some of the layers to maintain performance while keeping the cost constrained. This approach then requires precision assignments for model layers during training. We provide a new approach, called Q-PACE, consisting of a second-order sensitivity model that predicts the loss increase as a sum of quantization noise MSE weighted by per-layer curvature coefficients. During training, we periodically re-compute these coefficients using perturbations across layers, and re-assign precision. Pretraining and supervised fine-tuning experiments on LLMs of up to 4B parameters show that Q-PACE consistently improves over existing mixed-precision training recipes, and achieves comparable loss at substantially lower total memory budgets. We further find that quantization sensitivity is highly predictable by depth and layer type, and its stability during training allows for infrequent, cheap recalibration.

Abstract

Mixed-Precision Quantization

Explore similar work

Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training

MixQuant: Adaptive Mixed-Precision Quantization for Large Language Models

Beyond Activation Alignment:The Alignment-Diversity Tradeoff in Task-Aware LLM Quantization