Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models

Chashi Mahiul Islam; Alan Villarreal; Mao Nishino; Shaeke Salman; Xiuwen Liu

cs.AIApr 14, 2026

Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models

Authors: Chashi Mahiul Islam, Alan Villarreal, Mao Nishino, Shaeke Salman, Xiuwen Liu

Organizations: Department of Computer Science, Florida State University, Tallahassee, USA · Department of Mathematics, Florida State University, Tallahassee, USA

Abstract

As Large Language Models (LLMs) are increasingly integrated into agentic workflows, their unpredictability stemming from numerical instability has emerged as a critical reliability issue. While recent studies have demonstrated the significant downstream effects of these instabilities, the root causes and underlying mechanisms remain poorly understood. In this paper, we present a rigorous analysis of how unpredictability is rooted in the finite numerical precision of floating-point representations, tracking how rounding errors propagate, amplify, or dissipate through Transformer computation layers. Specifically, we identify a chaotic "avalanche effect" in the early layers, where minor perturbations trigger binary outcomes: either rapid amplification or complete attenuation. Beyond specific error instances, we demonstrate that LLMs exhibit universal, scale-dependent chaotic behaviors characterized by three distinct regimes: 1) a stable regime, where perturbations fall below an input-dependent threshold and vanish, resulting in constant outputs; 2) a chaotic regime, where rounding errors dominate and drive output divergence; and 3) a signal-dominated regime, where true input variations override numerical noise. We validate these findings extensively across multiple datasets and model architectures.

Figures & tables

Fig. 1 : Overview of effective directional number $D(\epsilon,v)$ at layer index 32 (Float32) for $v_{1},v_{50},v_{500},v_{2000},v_{4096}$ .

Fig. 2 : Layer-wise propagation profile at

\epsilon=0.1

. Directional structure is visible: higher-

\sigma

directions yield larger amplification than low-

\sigma

directions, consistent with a signal-dominated regime.

Fig. 3 : Layer-wise propagation profile at

\epsilon=10^{-10}

. Directional structure collapses: multiple singular directions, coordinate directions, and a random direction exhibit similar growth. This supports an instability-dominated regime where effective sensitivity is governed by scale rather than singular value.

Model	Dataset	Mean Inst.	Max Drift	Mean Margin	Min Margin
GPT-OSS	AdvBench	6.76 $\times 10^{5}$	2.92 $\times 10^{-4}$	0.6196	0.6196
GPT-OSS	TruthfulQA	2.63 $\times 10^{5}$	3.43 $\times 10^{-4}$	0.9767	0.9767
Llama-3.1	AdvBench	3.78 $\times 10^{6}$	1.02 $\times 10^{-4}$	0.6175	0.6175
Llama-3.1	TruthfulQA	8.84 $\times 10^{6}$	1.46 $\times 10^{-2}$	0.5351	0.5035

TABLE I : AGGREGATE STABILITY METRICS. Llama-3.1-8B RESULTS USE 100 PROMPTS PER DATASET (GPU). GPT-OSS-20B RESULTS USE 10 PROMPTS PER DATASET (CPU). MEDIAN INSTABILITY OF 0.0 INDICATES PREVALENT CONSTANT-LIKE PLATEAUS, WHILE LARGE MEAN INSTABILITY INDICATES RARE BUT LARGE FINITE-DIFFERENCE SPIKES.

Fig. 4 : Micro-continuity analysis: consecutive differences exhibit long constant-like stalls with rare discrete jumps, producing a staircase-like cumulative change in the output representation.

Metric	$v_{1}$ - $v_{2}$	$v_{1}$ - $v_{10}$	$v_{1}$ - $v_{4096}$
Flip Frequency (%)	16.14	16.29	15.42
Fragmentation	861	866	779
Crossing Density	51.0	52.5	49.0

TABLE II : DECISION BOUNDARY IRREGULARITY METRICS ACROSS SINGULAR VECTOR PLANES NEAR A NEAR-TIE POINT

x_{0}

Panel of Figure 7 — Fig. 5 : Binary decision maps near a constructed near-tie point. White: $L_{1}\geq L_{2}$ (Token 1 wins). Black: $L_{1}<L_{2}$ (Token 2 wins). The fragmented regions indicate that near-tie decisions can flip under microscopic perturbations, including along low- $\sigma$ directions.

Panel of Figure 8 — Fig. 6 : Directional stability in the $(v_{1},v_{2})$ plane. (a) Maximum stable perturbation $s_{\max}(\theta)$ varies by $4\times$ across angles. (b) Cartesian projection reveals asymmetric polygonal boundary. Geometric distortion demonstrates that stability is determined by input-dependent rounding dynamics, not by singular values.

Fig. 7 : Maximum stable perturbation magnitude along all 4096 singular vectors. Despite singular values spanning $\sigma_{1}=615.3$ to $\sigma_{4096}\approx 0$ , the stability boundary remains nearly constant ( $s_{\max}\sim 10^{-10}$ ) across the entire spectrum. This universality proves instability pervades the entire embedding manifold.

Panel of Figure 10 — Fig. 8 : Directional number vs. $\epsilon$ at layer index 32 under different floating-point precisions. Precision shifts the $\epsilon$ ranges where plateau behavior and rapid growth appear, while preserving the overall scale dependence.

Fig. 9 : Absolute directional condition number convergence via noise averaging. The dashed red line shows the unaveraged baseline, while the blue curve shows the averaged estimate stabilizing at $\sim$ 600 by $n=100$ samples, converging to the theoretical singular value (dotted line, 615.31). Averaging eliminates rounding-error artifacts and recovers true model sensitivity.

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 787 papers

Numerical stability analysis of large language models
Mar 13, 2025Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng +1Transformer Inference Transformer Attention
Faculty of Mathematics, University of Vienna, Kolingasse 14-16, 1090, Vienna, Austria · Huawei Technologies
Hidden Reliability Risks in Large Language Models: Systematic Identification of Precision-Induced Output Disagreements
Apr 2, 2026Yifei Wang, Tianlin Li, Xiaohan Zhang +4LLM Reliability LLM Evaluation
Shanghai Jiao Tong University, China · Beihang University, China · Nanyang Technological University, Singapore +1
Reliability Scaling Laws for Quantized Large Language Models
Jul 12, 2026Sirine Ayadi, Sándor Daróczi, Stephan Günnemann +1LLM Quantization Language Model Calibration
School of Computation, Information and Technology, Technical University of Munich Munich Data Science Institute · Pruna AI

Mar 13, 2025J/K move · Enter open · S save

Numerical stability analysis of large language models

Stanislav Budzinskiy, Wenyi Fang, Longbin Zeng, Philipp Petersen

Faculty of Mathematics, University of Vienna, Kolingasse 14-16, 1090, Vienna, Austria · Huawei Technologies

Transformer Inference Transformer Attention

Open paper AskPDF arXiv

Transformers are the state-of-the-art architecture for large language models, and a key to their scalability is the strategic usage of low-precision arithmetic. We develop a mixed-precision analysis of transformer inference, deriving bounds for the condition numbers and forward error of the architecture's constituent parts. Notably, we compare the numerical stability of LayerNorm and RMSNorm in the massive-outlier regime, tighten the error bound of softmax in the presence of attention sinks, and quantify the impact of its shifted evaluation on the sensitivity to perturbations. Furthermore, we derive novel sequence-length-independent bounds on the local Lipschitz constant of self-attention. Our worst-case error bound for transformer inference suggests that its numerical stability is determined by the interplay between weight magnitude and the growth of the residual stream. Crucially, and as validated by experiments with GPT-2, our analysis establishes that the scaling of residual-projection weights preserves the propagation of the relative rounding error unless it forces a qualitative transition in the dynamics of the residual stream.

Abstract

LLM Reliability

Explore similar work

Numerical stability analysis of large language models

Hidden Reliability Risks in Large Language Models: Systematic Identification of Precision-Induced Output Disagreements

Reliability Scaling Laws for Quantized Large Language Models