Since scaling laws were introduced for modern LLMs (Kaplan et al., 2020), several works have extended them to account for quantization, tracking the emerging low-precision data types supported on AI accelerators. We develop a more accurate scaling law for quantized pretraining with mixed precision across the forward and backward passes, grounded in optimization theory and validated empirically. Our formulation separates the effects of quantization on model capacity and on optimization efficiency, and explicitly models the persistent optimization error from forward and backward quantization identified through convergence analysis. To select the regimes for fitting and evaluating our law, we first decompose existing 4-bit training recipes into their forward and backward components, isolating those that most affect loss. Our Qwen3 experiments under selected 4-bit regimes, together with published Llama-style experiments, show that our law predicts loss with up to 11× lower MSE than existing laws as model size and token count scale. Finally, we extend our law to layer-wise mixed precision and identify the optimal quantized-pretraining recipe, i.e., model size, training tokens, and per-layer forward/backward precision, under a hardware-aware compute budget. Unlike prior precision scaling laws, our law predicts that the optimal precision increases with the compute budget.
Figures & tables
Figure 1: Compute-optimal precision allocation predicted by our scaling law. With each Transformer block using MXFP8 or NVFP4 in each pass, the panels show the optimal fraction of NVFP4 blocks in the forward pass pf∗ (top, left) and backward pass pb∗ (top, right), the optimal model size N∗ (bottom, left), and tokens per parameter D∗/N∗ (bottom, right), given the compute budget ( x -axis) and the NVFP4 speedup over BF16 ( y -axis), with the MXFP8 speedup fixed at 1.8× (Section 4.3 ).
Figure 2
Figure 4: Predicted versus observed loss of layer-wise mixed-precision runs on the held-out 910M model, with γS fitted on the three smaller models. The endpoints L1 and L2 in Eq. 2 are predicted by our law (left) or the Quartet law (right). Markers indicate the mixed pass and the blocks kept in higher precision, and each point corresponds to one fraction p and one token budget.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Quartet grid (Llama), model size extrapolation, for (a) the Quartet law and (b) our law. Each law is fitted to the 30M–100M models and evaluated on the held-out 200M models.
Figure 6: Quartet grid (Llama), data extrapolation, for (a) the Quartet law and (b) our law. Each law is fitted to 25–400 tokens per parameter and evaluated at the held-out 800 tokens per parameter.
Figure 7: Our grid (Qwen3), model size extrapolation, for the Quartet law. The law is fitted to the 280M–640M models and evaluated on the held-out 910M models.
Figure 8: Our grid (Qwen3), model size extrapolation, for our law. The law is fitted to the 280M–640M models and evaluated on the held-out 910M models.
Figure 9: Our grid (Qwen3), data extrapolation, for the Quartet law. The law is fitted to all but the largest token budget and evaluated at the held-out 35.8B tokens.
Figure 10: Our grid (Qwen3), data extrapolation, for our law. The law is fitted to all but the largest token budget and evaluated at the held-out 35.8B tokens.
Figure 11: Normalized increase in loss of layer-wise mixed-precision runs versus the fraction p of blocks in NVFP4, with endpoints L1 and L2 predicted by (a) the Quartet law and (b) our law. Solid curves show pγS with γS fitted on the calibration runs, and dotted lines show linear interpolation ( γS=1 ).
Figure 12: Compute-optimal precision allocation predicted by our scaling law, as in Figure 1 , with the MXFP8 speedup over BF16 set to (a) 1.6× and (b) 1.4× instead of 1.8× .
Mixed-precision quantization assigns different numeric precisions across a model to balance accuracy with memory, speed, energy, and hardware constraints.
Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common remedy is mixed-precision training, in which high precision is assigned to some of the layers to maintain performance while keeping the cost constrained. This approach then requires precision assignments for model layers during training. We provide a new approach, called Q-PACE, consisting of a second-order sensitivity model that predicts the loss increase as a sum of quantization noise MSE weighted by per-layer curvature coefficients. During training, we periodically re-compute these coefficients using perturbations across layers, and re-assign precision. Pretraining and supervised fine-tuning experiments on LLMs of up to 4B parameters show that Q-PACE consistently improves over existing mixed-precision training recipes, and achieves comparable loss at substantially lower total memory budgets. We further find that quantization sensitivity is highly predictable by depth and layer type, and its stability during training allows for infrequent, cheap recalibration.
Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2
Mixed-precision quantization improves the accuracy of post-training quantization by allocating higher bitwidths to sensitive layers, but existing methods solve the allocation for a single fixed memory budget. In practice the budget varies across deployments and is unknown at calibration time. Adaptive quantization addresses this with one offline calibration that serves any budget, yet current methods score layer sensitivity in a manner that does not consider its dependency on quantization levels of other layers. We show that a layer's sensitivity depends strongly on the bitwidths of its upstream layers and that this dependence shifts the resulting preferred bit allocation. We propose MixQuant, a technique-agnostic adaptive framework that wraps any base quantizer. MixQuant marginalizes each layer's distortion over random quantized upstream configurations to obtain budget-agnostic scores, calibrates the quantizer's parameters on plans the allocator itself produces, and penalizes allocations that leave layers at the lowest bitwidths. A single greedy pass then serves any budget at deployment. Across Llama-3.2-3B, Llama-2-7B, and Mistral-7B under AWQ and GPTQ, MixQuant outperforms adaptive and mixed-precision baselines in every setting, improving average accuracy by up to 8 points and reducing perplexity from 12.43 to 10.70 at the tightest budget, while matching an ILP solver at negligible deployment cost.
Mixed-precision quantization (MPQ) has become a key technique for deploying large language models under stringent memory and compute constraints. We first identify a phenomenon that we term the Perplexity Illusion: layers ranked as important by perplexity-based sensitivity show little rank correlation with those that are most influential for complex reasoning performance, with Kendall τ≈0 in our analysis. We further reveal an Alignment-Diversity Tradeoff: using only target-task calibration data can degrade post-quantization performance, whereas incorporating general-domain data stabilizes sensitivity estimation and improves robustness across tasks. Based on these observations, we propose TASA (Task-Aware Sensitivity Analysis), a two-level framework that jointly optimizes calibration-data composition and mixed-precision bit allocation. Specifically, TASA searches for a calibration-data mixture using a training-free gradient-trace alignment criterion, and then aggregates perplexity and reasoning-oriented sensitivity signals to guide both inter-layer and intra-layer bit allocation. Experiments on LLaMA-3-8B and Qwen2.5-7B reveal a precision inversion: appropriately allocated 3.5-bit models can match or surpass less task-aware 4-bit baselines. At an average precision of 3.5 bits, TASA matches or outperforms several competitive 4-bit uniform baselines in aggregate accuracy, and improves over the strongest W3 baseline on GSM8K by more than 20 absolute points on LLaMA-3-8B. These results show that calibration-data composition substantially affects task-sensitive quantization, a factor underexplored in prior work.
Fei Wang, Chao Xue, Taoran Liu +3
South China University of Technology · Ocean University of China · Shenzhen Campus of Sun Yat-sen University +1