Accuracy is not compatibility: two Evo 2 checkpoints hinge on one FP8 matrix multiplication

Xun Ding; Ziyuan Qin

cs.LGOct 29, 2026

Accuracy is not compatibility: two Evo 2 checkpoints hinge on one FP8 matrix multiplication

Authors: Xun Ding, Ziyuan Qin

Organizations: Independent researcher · Emory University

Abstract

FP8 formats specify how values are stored, not how products are summed; this gap can break checkpoints. Evo 2, a DNA language model, documents a Hopper FP8 requirement for its 1B and 40B checkpoints. On two Blackwell (sm_120) GPUs both fail the developers' forward test (the 7B passes); on an RTX 5070 Ti the 40B's next-base accuracy is 25.7%, near guessing. Greater accuracy breaks them, as BioNeMo saw for the 40B on Hopper: natively, 99.996% of one projection's outputs are correctly rounded (H100: about 30%), and fp32 accumulation of the same operands fails. Our H100 FP8 GEMM emulator, mmaemu (bit-exact on captured 1B projections), localises the dependence to the 1B's first projection (of 21), where exact sums fail but pass when shrunk by the H100's 1.4% (measured in-sample), not 2.1%. The 40B fails with only its first projection native. Emulating only that projection, weights unchanged, raises 40B BRCA1 variant-classification AUROC from 0.434 to 0.869 (H100: 0.868) on the RTX 5070 Ti, despite a narrow forward-test miss, and 0.481 to 0.867 on an RTX PRO 6000. We tested one model family but not training origin; releases should specify reference arithmetic.

Figures & tables

Figure 1: Arithmetic accuracy and checkpoint compatibility are different targets. a , Evo 2 1B. Left: share of the 11,796,480 outputs of block 0’s FP8 projection on one 2,048-bp window that equal the correctly rounded bf16 value (one GEMM, whose rows embed only four distinct tokens: 23,040 distinct dot products; other projections and windows were not analysed). Right: next-base accuracy of the whole model over the four upstream test prompts (printed by the test; not its pass criterion). The comparison is across devices: the reference GPUs ran the official Transformer Engine path, whereas the RTX 5070 Ti’s native FP8 projections called torch._scaled_mm ; panel b’s fp32 rows are the within-device control. b , Each square is one FP8 input projection, in model order; the first is block 0’s. All runs used the RTX 5070 Ti, with every other operation native. Emulated squares are live mmaemu hopper-te ; the hatched square is a lookup table of emulated hopper-te block-0 outputs; fp32 squares accumulate the same E4M3 operands in IEEE fp32. Filled circles pass the upstream forward test (within 0.001 of the expected loss), crosses fail, and the open circle lies 0.0011 from the expected value, just outside the tolerance; with the other 40B projections in bf16 the table gives 0.2168, within it (Table 1 ). Each loss is a mean of four bf16 per-prompt values, so distinct runs can print identical means (e.g. 1.2480). c , 40B AUROC on 500 BRCA1 variants (111 LoF), with conditional 95% variant-bootstrap intervals. Within each native/table pair the GPU, weights, software stack and cohort are fixed and only block 0’s FP8 projection changes; on the PRO 6000, native is the unmodified upstream path. The dashed line is the H100 reference (cu128), so comparing the PRO 6000 (cu13) column with it changes both device and stack; the dotted line marks 0.5. AUROCs are not ranked, and recovered variant scoring does not certify the forward or generation tests.

	Evo 2 1B (500 variants, 111 LoF)				Evo 2 40B (500 variants, 111 LoF) g
Run	Loss	AUROC (95% CI)	$r$	Sign (%)	Loss	AUROC (95% CI)	$r$	Sign (%)
H100, official	0.5022	0.733 (0.673–0.792)	ref.		0.2159	0.868 (0.829–0.904)	ref.
RTX 4060, official	n.r. a	0.738 (0.680–0.797)	0.987	84.2	out of memory (8 GB)
mmaemu hopper-te	0.5022 b	0.735 (0.675–0.793)	0.988	86.0	0.2161	0.871 (0.833–0.905)	0.998	97.4
mmaemu ada-te	0.5027	0.732 (0.672–0.791)	0.986	84.2	0.2161	0.871 (0.833–0.905)	0.998	96.2
native FP8	1.2480	0.559 (0.501–0.615) c	0.058 c	58.8 c	1.4023	0.434 (0.373–0.497)	$-$ 0.032	50.4

Table 1: Checkpoint behaviour under different FP8 projection implementations. The upper panel uses the reference GPUs and RTX 5070 Ti with the original cu128 (CUDA 12.8) stack; the lower panel uses the PRO 6000 with cu13 (CUDA 13). Interventions are confined to originally FP8 projections; bf16 GEMMs also change operand quantisation. Loss is the mean bf16 upstream-test loss; because each prompt’s loss is a bf16 value, distinct runs often print identical means. AUROC has a 95% variant-bootstrap interval.

r

and sign are Pearson correlation and same-sign percentage of per-variant

\Delta

against the H100. Both panels use the same 500 variants (111 LoF). PRO–H100 comparisons change both device and software stack. Reference-pair and emulated AUROC intervals overlap; no ranking is read.

Figure 2: Local arithmetic and matched residual interventions in Evo 2 1B. All displayed losses use the RTX 5070 Ti and the four upstream forward-test prompts, in nats/base. a–c , Only block 0’s FP8 accumulator changes; operands, scales and the epilogue are common. Loss is the upstream mean bf16 metric; the dashed line is the expected value, 0.501953125. Filled circles pass its tolerance, crosses fail and the open circle marks a narrow miss outside the tolerance (same marker as Fig. 1 b). a , The H100-like truncating chain (F13, equal to hopper-te bit for bit) and the exact accumulator times the H100’s measured gain (

a=0.985869

) pass; exact accumulation and the F13 accumulator divided by

a

fail. The gain is fitted to known arithmetic outputs, not loss, and is in-sample. These treatments do not independently vary error pattern and scale; output rounding remains. b , Nine sampled gains on the exact accumulator; no passing interval or optimum is inferred between points. Gains 0.993 and 1.007 lie at equal distance from the exact output; the shrinkage passes and the enlargement fails. c , Truncating-chain fraction bits are ordered categories with equal visual spacing. F14’s bf16 mean differs from F13’s passing mean only by one bf16 rounding step on one prompt, which takes it just outside the tolerance (its fp32 loss, 0.5027, is no higher than F13’s, 0.5028); from 15 bits the loss rises sharply. d , A separate intervention after block 0’s residual addition and before the MLP, using the block-0 F13 path with the other FP8 projections native. The native-shaped target is

\mathrm{bf16}(r_{F}+\lambda(r_{N}-r_{F}))

, where

r_{F}

and

r_{N}

are the block-0 F13 and native residuals. At each amplitude, five Gaussian directions match the realised target perturbation’s per-token L2 norms within 1% after bf16 rounding. One zero-amplitude reference is shared. Each point is the mean fp32 loss across four prompts; thin lines join discrete evaluations. At amplitude 0.75 one random direction narrowly misses the bf16 test tolerance and the native-shaped target also misses it; only the native-shaped target at full amplitude gives native-like loss (1.2488), while all five random directions retain low loss. These controls concern this node, checkpoint and prompt set.

Figure 3: BRCA1 variant-effect scoring on two sm_120 GPUs. All panels use the same 500 BRCA1 variants. a,b , 1B on the PRO 6000 (cu13). c,d , 40B on the PRO 6000 (cu13). e,f , 40B on the RTX 5070 Ti (cu128). Left panels compare native FP8 and the block-0 table of emulated hopper-te outputs with the archived H100 reference; each point is one variant, dashed lines mark equality and $r$ is Pearson correlation. Only block 0’s FP8 projection changes between native and table runs; the remaining model runs natively. Right panels show ROC curves; legend values are AUROCs. Panel f, whose legend sits below it, also includes full hopper-te and ada-te emulation and bf16 GEMMs. Table 1 gives 95% variant-bootstrap intervals; no ranking is inferred among the reference and emulated AUROCs, and AUROC agreement alone is not a model-level equivalence test. The H100 reference used cu128, as did the RTX 5070 Ti; PRO–H100 comparisons change both device and software. $\Delta$ is variant minus reference mean sequence log-probability; ROC curves use $-\Delta$ with LoF positive and functional/intermediate variants negative.

ML Reproducibility

Machine learning reproducibility studies whether model results and scientific claims can be independently checked and reliably repeated.

See all 138 papers

Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance
Aug 25, 2026Teng-Ruei ChenLLM Quantization Neural Network Quantization
Krixvon, Taipei 100, Taiwan
An 83-Format Numeric Catalog with Bit-Exact Conformance Vectors: A Vendor-Neutral Reference for FP8, BF16, MXFP4, and Microscaling Formats
Jun 8, 2026Dmitrii Vasilev
Trinity S3AI
Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
Sep 30, 2026Shuxiao Xie, Shuyang Xie, Yuan Cao +3LLM Inference Acceleration Low-Bit Quantization
Beijing Tongming Lake Information Technology Application Innovation Center (TLAIC), China · Fudan University Institute of Systems for Advanced Computing, China · Harbin Institute of Technology, China +2

Aug 25, 2026J/K move · Enter open · S save

Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance

Teng-Ruei Chen

Krixvon, Taipei 100, Taiwan

LLM Quantization Neural Network Quantization GPU Kernel Correctness Verification ML Reproducibility

Open paper AskPDF arXiv

Conformance suites for quantized GEMM kernels ask whether two implementations agree within a tolerance. We measure what such a suite can detect. Injecting nine faults into a reference INT8 pipeline over 8,232 layer--fault--regime cells of Qwen3-1.7B, we find that every one of five epilogue faults -- scale precision, double rounding, multiplication order, output truncation, fused ordering -- moves the output by at most a single bfloat16 spacing, and by exactly one whenever it moves it at all, across 5,880 cells. A tolerance of one spacing is therefore blind to the entire class by construction: four of the five faults are detected by no check in the suite, and the fifth only under power-of-two scales. Faults that violate the accumulator's exactness preconditions, or that break operand sharing, are detected without exception, and a null fault never fires. What a tolerance-based suite of this shape establishes is therefore narrower than interchangeability: that the preconditions hold, that operands are shared, and that differences stay within one spacing. The power-of-two constraint that exposes the one detected fault is also deployable. Requantizing every weight scale to its nearest power of two makes CUTLASS and Triton agree bitwise at every linear layer (196/196 and 252/252, against 8/196 and 10/252 under the checkpoints' own scales) and yields byte-identical generated token sequences at 1.7B, 8B and 14B (8/8 prompts, against 0/8 at all three). Observed perplexity point estimates are +0.32%, -0.28% and +0.48%; the 90% intervals cover zero at the two smaller sizes but not at 14B, reaching +0.71% and +0.76%. A previously reported +157% perplexity for this intervention was an artifact of a probe that rewrote scales without requantizing the weights; separating the effects attributes 99.8% of it to the resulting weight--scale mismatch rather than to the power-of-two constraint itself.

Abstract

ML Reproducibility

Explore similar work

Deterministic LLM Inference Across GPU Kernels: Power-of-Two INT8 Quantization Scales and the Limits of Tolerance-Based Conformance

An 83-Format Numeric Catalog with Bit-Exact Conformance Vectors: A Vendor-Neutral Reference for FP8, BF16, MXFP4, and Microscaling Formats

Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication