Organizations: University of British Columbia, Department of Electrical and Computer Engineering, Vancouver, BC, Canada · New York University, Courant Institute School of Mathematics, Computing and Data Science, New York, NY, USA
Floating point computing has dominated scientific computations since the 1950s because of its versatile ability to approximate real numbers with both sufficiently high precision and sufficiently great range. Since 1985, two specific formats have become standard across the industry, one with 32 bits which is adequate for many purposes and one with 64 bits which allows numbers with both much higher precision and much greater range for applications where these are needed. However, during the past decade, deep learning has come to dominate computation, and narrower floating point formats with lower precision and reduced range have steadily become more common. In this article we discuss this phenomenon, why it is happening, how various 16-bit, 8-bit and even 4-bit floating point formats have been developed, and efforts to standardize these formats.
Figures & tables
Operation
Energy (fJ)
Relative Cost
Relative Cost per bit
8 bit integer addition
7
1
1
32 bit integer addition
30
4
1
8 bit integer multiplication
70
10
10
32 bit integer multiplication
1480
210
53
16 bit floating-point addition ( bfloat16 )
110
16
8
32 bit floating-point addition ( binary32 )
380
54
14
Table 1. Energy costs of compute and memory in the 7 nm silicon technology node Jouppi et al., 2021 .
Figure 1. Bit-level organization of common floating point formats in machine learning. Symbols σ , E and M indicate the sign, exponent and significand (mantissa) bits. The precision p of the format is the number of significand bits plus one (including the hidden bit). Formats with 5 or fewer exponent bits are often combined with scale factors. The Open Compute Project (OCP) formats were all introduced in 2023; the IEEE P3109 formats are expected to be formally standardized in 2027. While some structures appear similar, the represented values can differ, as illustrated in Table 2 .
OCP
IEEE P3109
MXFP4
-E2M1
MXFP6
-E3M2
MXFP6
-E4M1
MXFP8
-E4M3
MXFP8
-E5M2
signed,
Table 2. Key differences between OCP and IEEE P3109 formats. While OCP MX (including its subset, OFP8) has only five floating point formats (two 8-bit, two 6-bit, and one 4-bit) plus a fixed-point format (not shown), P3109 defines a large family of formats parameterized by total width k and precision p as well as signed versus unsigned and extended real (with ±∞ ) versus finite value sets.