Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusters

Lenore M. Mullin; Gaetan Hains

cs.DCSep 27, 2026

Validating Memory-Optimal Transformer Kernels on Real Hardware: From Formal Derivation to Measured Performance Across Two HPC Clusters

Authors: Lenore M. Mullin, Gaetan Hains

Organizations: Professor Emerita, College of Nanotechnology, Science, and Engineering, University at Albany (SUNY), Albany, NY, USA.

Abstract

We validate memory-optimal cost functions for transformer kernels derived via the Mathematics of Arrays (MoA). Companion Papers I-IV formally derive kernels for attention forward, backward, fused forward+backward, decode, and the complete block (RMSNorm, gated MLP) as a hardware-independent specification (DNF) transformed to a machine-specific realization (ONF) via gamma, with verification to machine precision against PyTorch. This paper checks those predictions against measured performance on two HPC clusters (Purdue Anvil, NCSA Delta) across CPU and GPU. Three results stand out. (1) We identify and fix a GPU regression: fusing forward+backward, proven to avoid materializing an O(n^2) intermediate, initially ran slower than naive on GPU due to atomic contention. Profiling confirmed 2.00x more atomic instructions; a targeted ONF rewrite reversed it, yielding up to 2.5x speedup. (2) Identical derivations produce markedly different real costs by topology: 535x NUMA-locality penalty on one cluster vs <3x oversubscription on another, showing optimal deployment is a function of the machine's array structure. (3) We report a partially resolved anomaly: identical denotational computations run faster in C than Fortran on CPU but faster in Fortran than C on GPU, narrowed to one dominant kernel and one memory-latency stall mechanism (3.17x time gap matches 3.35x stall gap). We treat hardware-specific optimization as a routine ONF rewrite with fixed, verified DNF, a candidate methodology for scaling AI onto evolving hardware without re-deriving correctness.

Figures & tables

$n$	Best thread count	Speedup at best (vs. 1 thread)	128 threads vs. best
64	4	1.4 $\times$	14.5 $\times$ slower
128	8	4.0 $\times$	5.5 $\times$ slower
256	16	7.3 $\times$	3.8 $\times$ slower
512	32	16.4 $\times$	2.3 $\times$ slower
1024	64	23.4 $\times$	1.7 $\times$ slower
2048	128	43.3 $\times$	— (128 optimal)

Table 1: Paper I forward pass, Delta CPU, best thread count per sequence length and the cost of over-threading small problems.

$n$	A100 (s)	H200 (s)	H200 speedup	A100 local exp.	H200 local exp.
64	0.000242	0.000166	1.46 $\times$	—	—
128	0.000383	0.000235	1.63 $\times$	0.66	0.50
256	0.000788	0.000450	1.75 $\times$	1.04	0.94
512	0.002899	0.001759	1.65 $\times$	1.88	1.97
1024	0.008808	0.005421	1.62 $\times$	1.60	1.62
2048	0.031952	0.027075	1.18 $\times$	1.86	2.32

Table 2: Paper I forward pass, real GPU data post-fix. Local exponent computed between each consecutive pair of points, same definition as the flawed v1 table this supersedes.

	Read (MB)	Write (MB)	Total (MB)
Predicted (naive DNF element count)	6.291	69.206	75.497
Measured ( ncu , real A100)	9.045	142.333	151.378
Ratio (measured / predicted)	1.44 $\times$	2.06 $\times$	2.01 $\times$

Table 3: Measured DRAM traffic (kernel ID 2, the timed

N{=}2048

run) against a naive element-count prediction from the DNF: read

Q,K,V

once each, write

A

(

n\times n

) and Out once each, 8 bytes per double-precision element.

Explore similar work

Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels
Jun 5, 2026Lenore Mullin, Gaetan HainsMatrix Multiplication Transformer Architectures
University at Albany, SUNY, Albany, NY, USA · LACL, Université Paris-Est Créteil, Créteil, France
MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel
Jul 21, 2026Lenore Mulin, Gaetan HainsMatrix Multiplication Transformer Attention
University at Albany, SUNY · LACL, Universit´e Paris-Est Cr´eteil
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
May 19, 2026Han Guo, Jack Zhang, Arjun Menon +4Graphics Processing Unit Kernels Matrix Multiplication
1Massachusetts Institute of Technology · *Part of this work was done at Together AI. · 2Princeton University +2

Jun 5, 2026, cs.LGJ/K move · Enter open · S save

Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels

Lenore Mullin, Gaetan Hains

University at Albany, SUNY, Albany, NY, USA · LACL, Université Paris-Est Créteil, Créteil, France

Matrix Multiplication Transformer Architectures Mathematics

Open paper AskPDF arXiv

The attention mechanism is the dominant computational bottleneck in modern transformer-based AI. Its standard implementation incurs quadratic memory traffic in the sequence length~

n

, and DRAM accesses cost 100--1000

\times

more energy than arithmetic operations on contemporary hardware, so any analysis focused solely on FLOP counts fundamentally mischaracterises the bottleneck. We present a Mathematics of Arrays (MoA) reformulation of scaled dot-product attention and its numerically stable softmax, deriving a Denotational Normal Form (DNF) that eliminates all intermediate arrays -- including the implicit transposed-key buffer and every softmax temporary -- by algebraic construction rather than empirical tuning. The DNF achieves

O(n_{dk} + n{_{dv}})

data movement versus

O(n^2 + n_{dk} + n_{dv})

for the standard implementation, where

n

is the sequence length,

dk

is the key dimensionality and

dv

the value dimensionality, and is verified numerically against PyTorch at full double-precision floating-point on concrete inputs. Unlike hardware-specific accelerators or empirical tiling schemes such as FlashAttention, MoA simultaneously provides array fusion, shape-transformation correctness, and predictive cost models from a single algebraic framework. Memory minimality is a theorem established before any code is written. A predictive performance model projects

2

100\times

speedup and

2

50\times

energy reduction, with the advantage widening at exascale. The derivation establishes a formally verified pipeline from Python specification through (ONF) Operational Normal Form, and dimension-lifted hardware mapping, providing performance-portable AI kernels of direct relevance to DARPA edge-deployment and DOE exascale priorities.

Abstract

Explore similar work

Attention at the Theoretical Minimum: A Mathematics of Arrays Framework for Memory-Optimal Transformer Kernels

MoA-Structured Decode Attention DNF Derivation, KV-Cache Accumulation, GQA/MQA, and OpenACC Kernel

CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs