cs.AINov 12, 2026

Advancing Quantization for R&R Models with Microscaling Formats

Authors: Yavuz Yetim, Bor-Yiing Su, Zhe Wang, Arthur Liu, Pranav Sharma, Ying Wang, Randy Shuai, Luke Pfister, 

Organizations: Meta Platforms, Inc., Menlo Park, USA · Advanced Micro Devices, Inc., Santa Clara, USA

Abstract

Recommendation and Ranking (R&R) models, particularly in digital advertising, require massive inference throughput but exhibit extreme sensitivity to precision loss. Unlike Large Language Models (LLMs) that frequently tolerate quantization-induced accuracy drops of 1% or more, production-grade R&R models enforce a stringent launch bar: model quality must remain within a 0.01% Normalized Entropy (NE) margin relative to the baseline, which commonly uses FP16 or FP32. This tight constraint has historically hindered the adoption of aggressive quantization in this domain. In this paper, we present an empirical evaluation of both MXFP8 and the 6-bit MXFP6 format for industrial-scale R&R inference on the AMD MI350X architecture. To the best of our knowledge, this is the first paper, across R&R and LLM domains, that reports significant speedups from native MXFP6 GEMMs, which quantize both weights and activations and compute directly on 6-bit elements, without sacrificing model quality. We demonstrate that we can selectively apply this 6-bit format to maximize overall QPS gain without breaching the strict NE parity threshold. We further show that MXFP8 is a viable alternative to row-wise FP8: the two formats have nearly the same accuracy on our models, so the faster kernel can be chosen for each GEMM shape. On two production ads ranking models, our implementation shows measured GEMM kernel speedups over FP16 of 1.33--1.55x for MXFP8 and 1.73--1.99x for MXFP6 across all supported linear layers, and up to 2.90x for MXFP6 on individual layers, alongside end-to-end (E2E) inference throughput gains from selective MXFP6 of 1.5--5.3% for Model A and 7.6--14.3% for Model B over their FP8 configurations.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

    Apr 1, 2026Zifei Xu, Sayeh Sharify, Hesham MostafaQuantization-Aware TrainingNeural Network Quantization

  2. Integer-Exact Microscaling: Running MXFP4 Language Models on INT8 Matrix Units

    Oct 31, 2026Shinyoung Kang, Junhyung Ahn, Yujun Kim +1LLM QuantizationNeural Network Quantization

  3. MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

    May 23, 2026Dahoon Park, Jahyun Koo, Sangwoo Hwang +1Mixed-Precision QuantizationAI Accelerator Inference