Recommendation and Ranking (R&R) models, particularly in digital advertising, require massive inference throughput but exhibit extreme sensitivity to precision loss. Unlike Large Language Models (LLMs) that frequently tolerate quantization-induced accuracy drops of 1% or more, production-grade R&R models enforce a stringent launch bar: model quality must remain within a 0.01% Normalized Entropy (NE) margin relative to the baseline, which commonly uses FP16 or FP32. This tight constraint has historically hindered the adoption of aggressive quantization in this domain. In this paper, we present an empirical evaluation of both MXFP8 and the 6-bit MXFP6 format for industrial-scale R&R inference on the AMD MI350X architecture. To the best of our knowledge, this is the first paper, across R&R and LLM domains, that reports significant speedups from native MXFP6 GEMMs, which quantize both weights and activations and compute directly on 6-bit elements, without sacrificing model quality. We demonstrate that we can selectively apply this 6-bit format to maximize overall QPS gain without breaching the strict NE parity threshold. We further show that MXFP8 is a viable alternative to row-wise FP8: the two formats have nearly the same accuracy on our models, so the faster kernel can be chosen for each GEMM shape. On two production ads ranking models, our implementation shows measured GEMM kernel speedups over FP16 of 1.33--1.55x for MXFP8 and 1.73--1.99x for MXFP6 across all supported linear layers, and up to 2.90x for MXFP6 on individual layers, alongside end-to-end (E2E) inference throughput gains from selective MXFP6 of 1.5--5.3% for Model A and 7.6--14.3% for Model B over their FP8 configurations.
Figures & tables
Precision
NVIDIA GB300
AMD MI350X
FP16
1×
1×
FP8 / MXFP8
2×
2×
MXFP6
2×
4×
MXFP4
6×
4×
NVFP4
6×
–
Table 1: Dense peak tensor throughput of each precision, relative to FP16 NVIDIA (2024) ; Advanced Micro Devices, Inc. (2026a) . MI350X has twice the MXFP8 peak throughput for MXFP6.
Figure 1: Inference work per example of the most complex launched late-stage ads ranking model at Meta, normalized to H1’24 ( 1× ), on a logarithmic scale. The sharp increase in H2’25 coincides with the first launch of the Adaptive Ranking Model Meta (2026b) ; H2’26 is an estimated target.
Format
Exp.
Man.
Fmax
Fmin
Dyn. Range
Scale
Granularity
Bits
FP16
5
10
65,504
2−24
≈1.10×1012
–
–
16
FP8 (tensor)
4
3
448
2−9
229,376
FP32
Whole tensor
8
FP8 (row)
4
3
448
2−9
229,376
FP32
Row (act.), channel (wt.)
≈8
MXFP8
4
3
448
2−9
229,376
E8M0
Block of 32
8.25
MXFP6
2
3
7.5
2−3
60
E8M0
Block of 32
6.25
NVFP4
2
1
6
2−1
12
E4M3 + FP32
Block of 16 + tensor
4.5
Table 2: Low-precision formats discussed in this paper. All formats have one sign bit. Bits per element include the amortized storage of the scales; FP16 is the baseline. MXFP8 and MXFP6 are listed with their high-mantissa variants (E4M3 and E2M3, respectively).
QSNR (dB)
Speedup over FP16
Speedup over FP16
Batch 1024
Batch 2048
Format
Min
P10
Median
Mean
Total
Per-shape
Shapes <1×
Total
Per-shape
Shapes <1×
Model A, 20 most compute-intensive linear layers (6 shapes)
FP16
70.9
71.1
71.7
71.8
1.00×
1.00×
–
1.00×
1.00×
–
FP8 (row)
32.5
33.1
34.6
35.5
1.64×
1.68×
0
1.82×
1.73×
0
MXFP8
32.3
32.8
34.6
35.4
1.67×
1.71×
0
1.70×
1.68×
0
Table 3: QSNR distribution and GEMM speedup over FP16 for the 20 most compute-intensive linear layers and for all linear layers of the two production models. For accuracy only, the FP16 row reports QSNR against FP32; the low-precision rows report QSNR against FP16. All speedups use FP16 execution time as the baseline, so FP16 is 1.00× by definition. MXFP6 speedups are those of the deep-pipelined kernel. Total speedup is the ratio of the summed GEMM time; per-shape speedup is the geometric mean over unique shapes. The all-layer blocks cover the call sites that every format supports.
Component
Label
Layers
K
N
Model A
MLP, wide input
MLP-W
1
1.6e5
6.1e3
MLP
MLP
7
2.3e4
6.1e3
Projection
Proj.
9
6.1e3
3.3e4
DCN down-proj.
DCN-D
1
1.6e5
1.0e3
DCN up-proj.
DCN-U
1
1.0e3
1.6e5
Table 4: Linear shapes of the 20 most compute-intensive layers of each model, grouped by component. K is the input dimension and N the output dimension; ranges indicate multiple shapes. Dimensions are rounded to two significant digits; aeb denotes a×10b .
Model
Parameters quantized to MXFP6
E2E QPS improvement
Tasks within 0.01% NE tolerance
Model A
80.8%
+5.3% / +1.5%
6 / 6
Model B
72.9%
+14.3% / +7.6%
3 / 3
Table 5: Summary of MXFP6 accuracy and end-to-end performance. Percentages use all unique linear-layer weights as the denominator, excluding biases and embeddings. QPS improvements are over the production FP8 configurations at batches 1024 / 2048, respectively.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Batch 1024
Batch 2048
Work band
Shapes
FP16 time
FP8
MXFP8
MXFP6
MXFP6 vs. MXFP8
FP16 time
FP8
MXFP8
MXFP6
MXFP6 vs. MXFP8
Model A
Smallest
11
4.4%
1.05
1.13
1.17
1.03
2.9%
1.07
1.18
1.20
1.02
Small
16
8.0%
1.23
1.00
1.17
1.16
6.5%
1.32
1.17
1.28
1.09
Medium
12
34.1%
1.46
1.31
1.65
1.26
33.3%
1.50
1.50
1.86
1.23
Largest
6
53.5%
1.62
1.66
2.29
1.38
57.2%
1.81
1.67
2.29
1.37
Appendix
Table 6: GEMM speedup by relative GEMM size, for the call sites that every format supports. FP16 time is the share of the total FP16 GEMM time; the FP8, MXFP8, and MXFP6 columns give the speedup over FP16, and the last column of each batch size gives the speedup of MXFP6 over MXFP8. Work per call is normalized to the largest supported shape of each model at batch 1024. The four bands have boundaries at approximately 0.05%, 0.50%, and 4.98% of that reference for Model A, and 0.12%, 1.18%, and 11.76% for Model B. Shapes keep the same band at batch 2048.
QSNR (dB)
Speedup, batch 1024
Speedup, batch 2048
Component
Label
Layers
K
N
FP8
MXFP8
MXFP6
FP8
MXFP8
MXFP6
FP8
MXFP8
MXFP6
Model A
MLP, wide input
MLP-W
1
1.6e5
6.1e3
32.9
32.8
31.2
1.82
1.96
2.72
2.19
1.94
2.78
MLP
MLP
7
2.3e4
6.1e3
34.8
34.7
33.1
1.40
1.55
2.46
1.63
1.32
2.27
Projection
Proj.
9
6.1e3
3.3e4
34.0
33.9
32.0
1.70
1.58
2.05
1.83
1.83
2.19
DCN down-proj.
DCN-D
1
1.6e5
1.0e3
40.8
40.8
39.8
2.52
2.11
2.85
1.98
2.06
2.90
Appendix
Table 7: QSNR and GEMM speedup over FP16 for the 20 most compute-intensive linear layers of each model, grouped by model component. QSNR is the mean over the layers of each component, and speedup is the ratio of their summed GEMM time. In Model A, all layers of a component share one GEMM shape; in Model B, some components span several shapes. Linear dimensions are rounded to two significant digits; aeb denotes a×10b .
Figure 2: Accuracy and performance of the 20 most compute-intensive linear layers of each model, grouped by model component (labels as in Table 4 ). Speedup bars start at the FP16 baseline ( 1× ).