Wearable sports sensors can recognize actions on the device itself, where lower-precision numbers reduce model storage and memory traffic. Two 4-bit block-scaled formats, MXFP4 from the Open Compute Project, introduced in 2023, and NVFP4 from NVIDIA, introduced in 2025, are drawing interest. Published evaluations cover general deep-learning benchmarks and large language models; to our knowledge, nobody has yet measured what they cost a small wearable-sensor model. We train a 159,492-parameter 1D CNN to recognize four basketball actions from wrist-accelerometer data in Hang-Time HAR and test it on players it has never seen, using five-fold cross-validation grouped by player. Every comparison between number formats is made fold by fold. In full precision the model reaches 69.26% macro-F1. Measured TFLite FP16 and INT8 models lose at most 0.16 percentage points. In simulation, 4-bit weights cost 1.45 points with MXFP4 and 0.72 points with NVFP4. When activations are quantized to 4 bits as well, MXFP4 loses 5.14 points and NVFP4 only 0.75, and NVFP4 comes out ahead in all five held-out-player folds. The ranking survives two alternative simulator settings, with NVFP4 leading by 3.49 to 4.40 points. A 2×3 ablation with 4-bit weights and activations (two block sizes and three scale rules) points to scale selection as the main source of MXFP4's loss. The reference MXFP4 conversion we simulate rounds the scale down, which can pick scales that are too small, so many blocks clip their largest values. Rounding the scale up recovers 3.18 points with 32-value blocks and 3.63 points with 16-value blocks, and switching to an 8-bit floating-point scale, as NVFP4 does, adds another 0.84 and 1.20 points. Block size matters less. Under this round-down rule, 16-value blocks score 0.44 points below 32-value blocks in all five folds; under the other two rules there is no consistent effect.
Figures & tables
Version
File (KiB)
Macro-F1
Δ vs. FP32 (folds lower)
Same answer as FP32 (%)
p50 latency (ms)
FP32
630.9
0.6926 ± 0.0494
0 (identical)
100.00
0.0867
FP16
323.8
0.6925 ± 0.0494
− 0.0001 (3/5 lower)
99.99
0.0866
INT8, dynamic range
181.0
0.6924 ± 0.0494
− 0.0003 (3/5 lower)
99.81
0.0807
INT8, full integer
182.8
0.6910 ± 0.0495
− 0.0016 (5/5 lower)
98.68
0.0865
Table 1: Deployable TFLite versions ( measured ). Macro-F1 is mean ± sd over 5 player-grouped folds. Δ is the paired mean difference from FP32, with the number of folds in which the version scored lower. Latency: median of 5 interleaved rounds of 2,000 single-window runs, 1 thread, on a Google Colab CPU virtual machine (VM; Intel Xeon @ 2.20 GHz, KVM, 2 vCPUs = 1 core × 2 threads; Ubuntu 24.04.5; Python 3.13.16; TensorFlow 2.21.0). Latencies of the same files differed by 11.6–24.6% between two Colab sessions, so these differences are within noise.
Weights only (W4 / W8)
Weights + activations (W4A4 / W8A8)
Format
Bits/weight
Weights KiB
Macro-F1
Δ (folds lower)
Macro-F1
Δ (folds lower)
INT8
8.12
165.0
0.6928
+ 0.0002 (1/5)
0.6930
+ 0.0004 (2/5)
INT4
4.12
88.2
0.6921
− 0.0006 (2/5)
0.5493
− 0.1433 (5/5)
MXFP4
4.25
90.8
0.6782
− 0.0145 (5/5)
0.6412
− 0.0514 (5/5)
NVFP4
4.50
95.6
0.6854
− 0.0072 (5/5)
0.6851
− 0.0075 (4/5)
Table 2: Simulated formats (fake quantization; no speed or energy). Bits/weight include scale storage: per-channel FP32 scales for INT8 and INT4, block scales for the FP4 formats, and NVFP4’s FP32 tensor scale. Weights KiB is calculated for the quantized kernels plus FP32 biases and batch-normalization parameters. Δ is the paired mean difference from FP32 (0.6926), with the number of folds (of 5) in which the format scored lower. INT4 and INT8 use one scale per window for activations, so their W4A4/W8A8 rows are not like-for-like with the blocked FP4 formats.
Figure 1: Change in macro-F1 from FP32 for every fold (markers) and the mean of the 5 folds (black line), simulated . The text above each group gives the number of folds (groups of test players) in which that format scored lower than FP32. Fold 4 (red) accounts for most of MXFP4’s mean loss. Open markers: INT4 W4A4 uses one activation scale per window and is not like-for-like with the blocked FP4 formats. n=5 folds; the same 5 trained models throughout.
Block
E8M0 down
E8M0 up
E4M3 + tensor
32
− 0.0145 / − 0.0514
− 0.0063 / − 0.0196
− 0.0080 / − 0.0112
16
− 0.0167 / − 0.0558
− 0.0073 / − 0.0195
− 0.0072 / − 0.0075
Table 3: Ablation study ( simulated ): mean change in macro-F1 from FP32, weights only / weights + activations. MXFP4 = block 32 with E8M0 rounded down; NVFP4 = block 16 with E4M3.
Figure 2: Ablation study, W4A4, simulated : per-fold change in macro-F1 from each design step (markers; fold 4 red; black line = mean of 5 folds). B is the block size. The text above each group gives the number of folds in which the step helped or hurt. Rounding the E8M0 scale up gives most of the gain; an E4M3 scale adds a smaller, also consistent, gain. Under the MX rule (E8M0 rounded down), 16-value blocks are slightly worse than 32-value blocks.
Variant
MXFP4
NVFP4
NVFP4 − MXFP4
Main
− 0.0514 (5/5)
− 0.0075 (4/5)
+ 0.0439 (5/5)
Ties to even
− 0.0514 (5/5)
− 0.0075 (4/5)
+ 0.0440 (5/5)
Layer-1 patch blocks
− 0.0444 (5/5)
− 0.0095 (5/5)
+ 0.0349 (5/5)
Table 4: Sensitivity of W4A4 results ( simulated ). Change in macro-F1 from FP32, with folds lower (of 5); last column: NVFP4 minus MXFP4, with folds in which NVFP4 is higher. Main: the notebook’s definitions (ties to smaller; one layer-1 activation block per time step).
On hardware whose fastest matrix datapath is INT8×INT8, such as the Neural Accelerators of Apple's M5 GPUs, 4-bit weights must become INT8 operands. Existing MXFP4 kernels already map FP4 (E2M1) values to integers by doubling them, but apply every 32-element block scale inside the reduction. We show when no per-block work is needed: a row of MXFP4 weights is exactly one INT8 vector times one power-of-two scale if its block exponents span at most three, so a linear layer becomes a single INT8 GEMM with one rescale per output. Enforcing the condition adds at most 1/16 of a row's largest rounding error to the blocks it changes, and real weights already satisfy it: 97.9-99.97% of rows across seven models, including 99.93% of the 95M rows of a 125B-parameter mixture-of-experts (MoE) model, Qwen3.8-Flash-Next. The resulting spread-constrained MXFP4 files stay standard MXFP4. With integer instead of power-of-two block multipliers, an integer-exact format (16-wide blocks, multipliers 1..10, 4.26 bits) runs as one exact INT8 GEMM and is more accurate than NVFP4 (4.5 bits) with its standard scale rule on five small models, within 0.06 perplexity of NVFP4 with scale search. On Qwen3.8-Flash-Next's routed experts with INT8 activations it is as close to the FP8 model as NVFP4 with scale search (KL divergence 0.0557 vs. 0.0553) and closer than MLX's MXFP4 and affine INT4; with every matrix quantized it solves 94.8% of GSM8K vs. 91.0% for MLX's MXFP4. Quantizing only the 74k routed expert matrices (96% of the parameters) to either of our formats with INT8 activations costs at most 0.2% perplexity. A complete W4A8 system on the M5 produces outputs bit-identical to the quantized reference; prefill is 1.34-1.74x faster than MLX on a dense model, 1.10-1.57x on two MoE models up to 35B parameters and 1.22-1.29x on Qwen3.8-Flash-Next, where shared non-GEMM work bounds the gain.
Shinyoung Kang, Junhyung Ahn, Yujun Kim +1
UT Southwestern Medical Center · Department of Data and Decision Sciences, Emory University · Fuqua School of Business, Duke University +1
NVFP4 has grown increasingly popular as a 4-bit format for quantizing large language models due to its hardware support and its ability to retain useful information with relatively few bits per parameter. However, the format is not without limitations: recent work has shown that NVFP4 suffers from its error distribution, resulting in large amounts of quantization error on near-maximal values in each group of 16 values. In this work, we leverage this insight to design new Adaptive Block-Scaled Data Types that can adapt to the distribution of their input values. For four-bit quantization, our proposed IF4 (Int/Float 4) data type selects between FP4 and INT4 representations for each group of 16 values, which are then scaled by an E4M3 scale factor as is done with NVFP4. The selected data type is denoted using the scale factor's sign bit, which is currently unused in NVFP4, and we apply the same insight to design formats for other bit-widths, including IF3 and IF6. When used to quantize language models, we find that IF4 outperforms existing 4-bit block-scaled formats, achieving lower loss during quantized training and achieving higher accuracy on many tasks in post-training quantization. We additionally design and evaluate an IF4 Multiply-Accumulate (MAC) unit to demonstrate that IF4 can be implemented efficiently in next-generation hardware accelerators. Our code is available at https://github.com/mit-han-lab/fouroversix.
Training large language models directly in 4-bit floating-point (FP) formats promises substantial improvements in throughput and energy efficiency. While some recipe design choices have been validated at scale, many promising approaches remain untested beyond small models and short token horizons, leaving open the question of which trends will hold. We present a systematic comparison of recent NVFP4 recipes at medium model scale, evaluating dense 8B and 30B-A3B MoE models trained for up to 1T tokens, with a focus on which ingredients are necessary for accuracy recovery. We propose a final recipe grounded in the principles behind established stable NVFP4 training at scale, incorporating state-of-the-art techniques such as unbiased gradient estimation with lower quantization error than stochastic rounding. To the best of our knowledge, this is the strongest FP4 training result demonstrated at this scale to date in loss gap to BF16. Through ablation studies, we find that: (i) each technique in the optimized recipe measurably improves loss trajectory, (ii) selective high-precision layers are necessary for recovering accuracy at scale, (iii) not all tensors in the backward pass benefit equally from de-biasing, leaving room to apply complementary error-reduction techniques to the remaining tensors.
Anjulie Agrusa, Andrei Panferov, Elizabeth Wei +7
NVIDIA · Institute of Science and Technology Austria · Red Hat AI