The AetherFloat Family: Block-Scale-Free Quad-Radix Floating-Point Architectures for AI Accelerators

Keita Morisaki

cs.ARFeb 26, 2026

The AetherFloat Family: Block-Scale-Free Quad-Radix Floating-Point Architectures for AI Accelerators

Authors: Keita Morisaki

Organizations: Independent Hardware/Software Co-Design Researcher Shinagawa City, Tokyo, Japan

Abstract

The IEEE 754 floating-point standard is the bedrock of modern computing, but its structural requirements -- a hidden leading bit, Base-2 bit-level normalization, and Sign-Magnitude encoding -- impose significant silicon area and power overhead in massively parallel Neural Processing Units (NPUs). Furthermore, the industry's recent shift to 8-bit formats (e.g., FP8 E4M3, OCP MX formats) has introduced a new hardware penalty: the strict necessity of Block-Scaling (AMAX) logic to prevent out-of-bound Large Language Model (LLM) activations from overflowing and degrading accuracy. The AetherFloat Family is a parameterizable architectural replacement designed from first principles for Hardware/Software Co-Design in AI acceleration. By synthesizing Lexicographic One's Complement Unpacking, Quad-Radix (Base-4) Scaling, and an Explicit Mantissa, AetherFloat achieves zero-cycle native integer comparability, branchless subnormal handling, and a verified 33.17% area, 21.99% total power, and 11.73% critical path delay reduction across the multiply-accumulate (MAC) unit. Instantiated as AetherFloat-8 (AF8), the architecture relies on a purely explicit 3-bit mantissa. Combined with Base-4 scaling, AF8 delivers a substantially wider dynamic range, acting as a ``Block-Scale-Free'' format for inference that circumvents dynamic scaling microarchitecture. Finally, a novel Vector-Shared 32-bit Galois Stochastic Rounding topology bounds precision variance while neutralizing the vanishing gradients that plague legacy formats. While AF16 serves as a near-lossless bfloat16 replacement via post-training quantization, AF8 is designed as a QAT-first inference format: its Block-Scale-Free property eliminates dynamic AMAX hardware at the cost of requiring quantization-aware fine-tuning for deployment.

Figures & tables

Fig. 1: Wobble Cost Analysis (SQNR). The Quad-Radix (Base-4) scaling introduces a measured mean SQNR penalty of $\approx 3.04$ dB relative to Base-2 bfloat16, acting as benign regularization during stochastic gradient descent.

Format	Base	Bias	Mantissa Scale	Normal Constraint	Subnormal Range	Special Values
AF16	4	63	$/64.0$	Leading Pair $\neq 00$	$M<64$	Extremity Sorting
AF8	4	7	$/2.0$	Leading Pair $\neq 00$	$M\in\{0,1\}$	Extremity Sorting

TABLE I: AetherFloat Format Specifications

Fig. 2: Vector-Shared Stochastic Rounding Ablation. Simulating Qwen2.5-7B next-token prediction confirms our Chunk=16 configuration effectively tracks ideal independent SR while amortizing hardware costs.

Metric	BF16 Baseline	AF16 (Ours)	FP8 E4M3 + AMAX	AF8 (Ours)
WikiText-2 PPL ( $\downarrow$ )	8.7368	8.7380	8.7951	10.6389
PIQA Acc ( $\uparrow$ )	0.7884	0.7878	0.7894	0.7775
HellaSwag Acc ( $\uparrow$ )	0.5990	0.5999	0.5975	0.5545

TABLE II: Zero-Shot Benchmarks for PTQ on Qwen2.5-7B

Fig. 3: 8-bit QAT Convergence (Qwen2.5-7B: AF8 vs FP8). Both formats exhibit transient loss spikes, with FP8 showing greater mid-to-late-training instability. AF8 demonstrates favorable convergence behavior by step 150 (single seed), confirming viable STE gradient flow without AMAX logic.

Feature	FP8 E4M3 (Baseline)	AF8 (Ours)	Delta
Multiplier Footprint	$4\times 4$ bit array	$3\times 3$ bit array ∗	Smaller
Total MAC Area	1018.48 $\mu\text{m}^{2}$	680.65 $\mu\text{m}^{2}$	-33.17%
Critical Path Delay	2426.30 ps	2141.60 ps	-11.73%
Total MAC Power	84.60 $\mu\text{W}$	66.00 $\mu\text{W}$	-21.99% †
Area $\times$ Delay Product	2,471,138 $\mu\text{m}^{2}{\cdot}$ ps	1,457,680 $\mu\text{m}^{2}{\cdot}$ ps	-41.01%
Block-Scaling	Required (AMAX ALU)	Scale-Free	Eliminated

TABLE III: Hardware Execution Overhead (Total MAC Unit)

Stochastic Rounding

See all 8 papers

Apr 9, 2026

HiFloat4 Format for Language Model Pre-training on Ascend NPUs

Training large foundation models at low numerical precision is one of the most promising directions for reducing the compute and memory cost of modern AI. Recent 4-bit floating-point formats such as MXFP4 and NVFP4 can be applied to linear GEMM operations in LLMs, but their limited dynamic range introduces numerical instability that prior work addresses by stacking stabilization mechanisms, typically executed at higher precision and partially eroding the efficiency gains that motivate FP4. In this work, we argue that numerical format design is itself a first-class lever for stable FP4 training, and present the first systematic study of FP4 LLM pretraining on energy-efficient Huawei Ascend NPUs. We compare the recently proposed HiFloat4 (HiF4) format against both MXFP4 and NVFP4, holding one recipe fixed across all three formats across dense (OpenPangu-1B, Llama3-8B) and Mixture-of-Experts (Qwen3-MoE-30B) architectures and executing all linear and expert GEMMs in FP4. At matched storage --- NVFP4 and HiF4 both spend 4.5 bits per value --- the three formats differ far more in what they require before they will train at all than in final accuracy. HiF4 reaches a relative loss of 1.55% with no stabilization, below fully stabilized MXFP4 (1.79%) and below NVFP4 carrying the per-tensor scaling it cannot train without (2.00%); NVFP4 diverges under every combination of stochastic rounding and Hadamard transform we tried. Our results suggest that stable, accurate FP4 training does not require an ever-growing stack of stabilization techniques; it requires the right numerical format.

Mehran Taghian, Yunke Peng, Xing Huang +22

Huawei

Low-Precision Training LLM Quantization LLM Training Language Model Pretraining

Ask PDF

May 12, 2026

Search Your Block Floating Point Scales!

Quantization has emerged as a standard technique for accelerating inference for generative models by enabling faster low-precision computations and reduced memory transfers. Recently, GPU accelerators have added first-class support for microscaling Block Floating Point (BFP) formats. Standard BFP algorithms use a fixed scale based on the maximum magnitude of the block. We observe that this scale choice can be suboptimal with respect to quantization errors. In this work, we propose ScaleSearch, an alternative strategy for selecting these scale factors: using a fine-grained search leveraging the mantissa bits in microscaling formats to minimize the quantization error for the given distribution. ScaleSearch can be integrated with existing quantization methods such as Post Training Quantization and low-precision attention, and is shown to improve their performance. Additionally, we introduce ScaleSearchAttention, an accelerated NVFP4-based attention algorithm, which uses ScaleSearch and adapted prior techniques to ensure near-0 performance loss for causal language modeling. Experiments show that ScaleSearch reduces quantization error by 27% for NVFP4 and improves language model PTQ by up to 15 points for MATH500 (Qwen3-8B), while ScaleSearchAttention improves Wikitext-2 PPL by upto 0.77 points for Llama 3.1 70B. The proposed methods closely match baseline performance while providing quantization accuracy improvements.

Tanmaey Gupta, Hayden Prairie, Xiaoxia Wu +10

Cornell University, Ithaca, NY, USA · Together AI, USA · University of California San Diego, La Jolla, CA, USA +1

4-Bit Quantization LLM Inference Acceleration Post-Training Quantization Neural Network Quantization

Ask PDF

Aug 28, 2026

H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference

The NVIDIA Blackwell architecture, with native support for the ultra-fine-grained NVFP4 format, opens new opportunities for accelerating large language model (LLM) inference. NVFP4's micro-block design, such as a group size of 16, offers strong representational flexibility for capturing local weight distributions and isolating outliers, but it also introduces a large and highly sensitive space of per-group scaling factors. Existing post-training quantization (PTQ) methods primarily focus on refining quantized weight values, leaving this scale-selection step underexplored. To address this gap, we propose H-Scale, a lightweight post-processing method for NVFP4 per-group scale refinement. Instead of minimizing plain weight reconstruction error, H-Scale selects hardware-valid group scales using a diagonal second-order proxy derived from calibration activations, thereby targeting layer output perturbation more directly. It is designed as a drop-in replacement for RTN-style scale selection in diverse NVFP4 pipelines, requires only modest offline calibration, and introduces strictly zero overhead at inference time. Under a fixed evaluation protocol, experiments on mainstream LLMs show that H-Scale generally improves a broad range of NVFP4 baselines and brings several variants closer to the BF16 reference.

Hao Yu, Zheng Li, Dayiheng Liu +1

Qwen Team, Alibaba Inc.

LLM Quantization Neural Network Quantization Post-Training Quantization Low-Bit Quantization

Ask PDF

Abstract

Stochastic Rounding

Explore similar work

HiFloat4 Format for Language Model Pre-training on Ascend NPUs

Search Your Block Floating Point Scales!

H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference