Scaling Laws of Global Weather Models

Yuejiang Yu; Langwen Huang; Alexandru Calotoiu; Torsten Hoefler

cs.LGFeb 26, 2026

Scaling Laws of Global Weather Models

Authors: Yuejiang Yu, Langwen Huang, Alexandru Calotoiu, Torsten Hoefler

Organizations: Department of Computer Science, ETH Zurich, Zurich, Switzerland.

Abstract

Data-driven models are revolutionizing weather forecasting. To optimize training efficiency and model performance, this paper analyzes empirical scaling laws within this domain. We investigate the relationship between model performance (validation loss) and three key factors: model size ( $N$ ), dataset size ( $D$ ), and compute budget ( $C$ ). Across a range of models, we find that Aurora exhibits the strongest data-scaling behavior: increasing the training dataset by 10x reduces validation loss by up to 3.2x. GraphCast demonstrates the highest parameter efficiency, yet suffers from limited hardware utilization. Our compute-optimal analysis indicates that, under fixed compute budgets, allocating resources to more total training data yields greater performance gains than increasing model size. Furthermore, we analyze model shape and uncover scaling behaviors that differ fundamentally from those observed in language models: weather forecasting models consistently favor increased width over depth. These findings suggest that future weather models should prioritize wider architectures and larger effective training datasets to maximize predictive performance.

Figures & tables

Figure 1: Scaling behavior of global weather models. We report the validation loss during training to evaluate model performance. We also report the training data required to reach 100 Pflop compute budget.

Model	Release Year	#Parameters	Backbone	Training Loss	#Surface + #Upper-air Variables	$\beta\uparrow$
Aurora	2024	1.3B	Swin Transformer	MAE	$\sim$ 4 + 5	0.51
AIFS	2024	255M	Graph Transformer	MSE	$\sim$ 10 + 5	0.46
Pangu	2023	276M	Swin Transformer	MAE	$\sim$ 4 + 5	0.43
GraphCast	2023	36.7M	Graph Neural Network	MSE	$\sim$ 5 + 6	0.36
SFNO	2023	433M	Spherical Fourier Operator	MSE	$\sim$ 8 + 5	0.34

Table 1: Summary of weather models. We report the default or most frequently investigated

N

in the corresponding papers. Variable counts treat one atmospheric variable with multiple vertical levels as a single variable and only include upper-air and surface variables. Our work presents:

\beta

is the data-scaling coefficient in

\mathcal{L}(D)=\alpha D^{-\beta}

and larger is better.

Figure 2: Data-scaling laws across weather forecasting models:

\mathcal{L}(D)=\alpha D^{-\beta}

. Aurora (red) achieves the best

\mathcal{L}

D=100

TB and also has the best

\beta

value, representing most efficient scaling with more data. The darker markers represent the larger models. Model specifications are provided in Appendix Table 6 .

Figure 3: Parameter-scaling laws for different weather prediction models trained on 15.0 TB and 30.0 TB datasets. Each marker corresponds to a model variant, with dashed lines denoting power-law fits of the form

\mathcal{L}(N)=\gamma N^{-\delta}

. Marker transparency indicates dataset size, with darker markers representing larger

D

. At fixed

D

, validation loss improves with larger

N

, and increasing

D

shifts the scaling curves downward while preserving or increasing the scaling exponent

\delta

within each model.

Figure 4: Wider models perform better. For each model, we use two configurations: one is wider and one is narrower. They have roughly the same

N

but different shapes. In all models, wider configurations consistently achieve lower validation loss. This implies weather forecasting benefits more from representational capacity (width) than from additional nonlinear transformations (depth).

Figure 5: Compute-optimal training. The panels illustrate

\mathcal{L}

as a function of

D

, with each curve representing a fixed

C

. The resulting parabolas identify the compute-optimal frontier—the specific ratio of

N

and

D

that minimizes loss for a given

C

. For each model, we have

C\sim ND

, which implies

N\sim C/D

along each curve. For Pangu, the loss is predominantly determined by

D

. Aurora and SFNO exhibit a clear minimum in the curve, indicating a trade-off between larger

N

and larger dataset size. GraphCast and AIFS shows mainly the left half of the parabolas, indicating performance is bottlenecked by insufficient

D

Figure 6: Variable-specific loss scaling for 10m u-component of wind (10U) and 2m temperature (2T) . We compare Aurora, GraphCast, Pangu, SFNO, and AIFS. Validation RMSE is plotted against the volume of training data in TB. Each line represents a different model configuration, with line color indicating

N

(lighter colors for narrower models, darker for wider). For 10U, Aurora achieves the lowest RMSE at all dataset sizes. For 2T, GraphCast’s performance is highly competitive and surpasses all models.

Model	Tflop/s	GPU peak (Tflop/s)	GPU util. (%)
Aurora	368	989 (32-bit)	37.2
AIFS	33.7	1979 (16-bit)	1.70
GraphCast	10.15	989 (32-bit)	1.03
Pangu	3.25	989 (32-bit)	0.33
SFNO	0.215	989 (32-bit)	0.022

Table 2: Computational efficiency of different models on H100 GPUs. Peak performance during training varies by precision: AIFS uses 16-bit half precision, while others use 32-bit single precision. Utilization is calculated relative to each model’s corresponding peak performance.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Optimizer	Learning rate	Betas	Parallelization	Batch size
Aurora	AdamW	$5\times 10^{-4}$	(0.9, 0.999)	DDP	16
GraphCast	Adam	$1\times 10^{-4}$	(0.9, 0.999)	DDP	16
Pangu	Adam	$5\times 10^{-4}$	(0.9, 0.999)	DDP	4
SFNO	AdamW	$1\times 10^{-4}$	(0.9, 0.95)	DDP	16
AIFS	AdamW	$1\times 10^{-4}$	(0.9, 0.95)	DDP	16

Appendix

Table 3: Training configurations for all models.

Variable name	Type	Short name	Unit
2m temperature	Surface	2T	K
10m u-component of wind	Surface	10U	m/s
10m v-component of wind	Surface	10V	m/s
Mean sea level pressure	Surface	MSL	Pa
Temperature	Upper-air	t—	K
Specific humidity	Upper-air	q—	kg/kg

Appendix

Table 4: Variable definitions. The dash line represents the pressure level measured in hPa.

Name	Loss weights $w_{j}$	Inv. var. weights $s_{j}$	Name	Loss weights $w_{j}$	Inv. var. weights $s_{j}$
2T	$1.0$	$2.4101\times 10^{-3}$	z300	$4.9793\times 10^{-2}$	$4.0072\times 10^{-8}$
10U	$0.1$	$3.7612\times 10^{-2}$	z400	$6.6390\times 10^{-2}$	$6.2134\times 10^{-8}$
10V	$0.1$	$4.8160\times 10^{-2}$	z500	$8.2988\times 10^{-2}$	$9.8700\times 10^{-8}$
MSL	$0.1$	$5.5049\times 10^{-7}$	z600	$9.9585\times 10^{-2}$	$1.5930\times 10^{-7}$
t50	$8.2988\times 10^{-3}$	$6.3288\times 10^{-3}$	z700	$1.1618\times 10^{-1}$	$2.6508\times 10^{-7}$
t100	$1.6598\times 10^{-2}$	$6.1954\times 10^{-3}$	z850	$1.4108\times 10^{-1}$	$5.8404\times 10^{-7}$

Appendix

Table 5: Per-variable loss weights and inverse variance weights.

Figure 7: CRPS comparison for 2m temperature (2T), temperature at 500 hPa (t500), and 10 m

u

-velocity of wind (u10) using 10 ensemble members. Results with 10 ensembles are within 2–5% of those obtained with 50 members, indicating minimal loss of accuracy and supporting the efficiency of reduced ensemble sizes. Model specifications are provided in Table 8 .

Model	Width	Depth	Size (M)
Aurora	256	(3, 5, 4)	162.4
Aurora	384	(3, 5, 4)	364.7
GraphCast	512	8	19.4
GraphCast	512	16	34.2
SFNO	384	8	289.4
SFNO	512	8	514.5

Appendix

Table 6: Model specifications for data-scaling analysis. The first row for each model is transparent. For depth in tuple, it means the number of layers in different components of the Encoder. The Decoders have the same numbers in a reversed order. For simplicity, we omit them here.

Model	Width	Depth	Size (M)
Aurora	128	(12, 20, 16)	155.2
Aurora	256	(3, 5, 4)	162.4
GraphCast	128	12	1.7
GraphCast	256	1	1.6
SFNO	256	8	128.6
SFNO	512	2	129.1

Appendix

Table 7: Model specifications for different shapes of same sizes. For depth in tuple, it means the number of layers in different components of the Encoder. The Decoders have the same numbers in a reversed order. For simplicity, we omit them here.

Model	Width	Depth	Size (M)
a	128	4	0.8
b	256	8	4.9
c	512	16	34.2

Appendix

Table 8: Model shapes and sizes for CRPS experiments.

Model	Label	Width	Depth	Size (M)
Aurora	Narrow	128	(3, 5, 4)	40.8
Aurora	Wide	384	(3, 5, 4)	364.7
GraphCast	Narrow	128	16	2.1
GraphCast	Wide	512	16	34.2
Pangu	Narrow	96	(2, 6)	258.8
Pangu	Wide	480	(3, 9)	599.4

Appendix

Table 9: Model specifications for variable-specific power law analysis. For depth in tuple, it means the number of layers in different components of the Encoder. The Decoders have the same numbers in a reversed order. For simplicity, we omit them here.

Model	Width	Depth	Size (M)
Aurora	64	(3, 5, 4)	10.3
	128	(1, 1, 1)	11.9
	64	(6, 10, 8)	19.9
	128	(1, 2, 2)	20.6
	128	(3, 5, 4)	40.8
	128	(6, 10, 8)	78.9

Appendix

Table 10: Model specifications for all models presented in this work. For depth in tuple, it means the number of layers in different components of the Encoder. The Decoders have the same numbers in a reversed order. For simplicity, we omit them here.

Scaling Laws

Scaling laws describe how model performance and learning dynamics change with model size, data, compute, training time, and other resources.

See all 72 papers

On Neural Scaling Laws for Weather Emulation through Continual Training
Mar 26, 2026Shashank Subramanian, Alexander Kiefer, Arnur Nigmetov +3Neural Scaling Laws Weather Forecasting
Lawrence Berkeley National Laboratory · Oak Ridge National Laboratory · ICSI and UC Berkeley +1
Towards Scaling Law Analysis For Spatiotemporal Weather Data
Apr 6, 2026Alexander Kiefer, Prasanna Balaprakash, Xiao WangWeather Forecasting Spatiotemporal Forecasting
Bredesen Center, University of Tennessee, Knoxville, Tennessee · Oak Ridge National Laboratory, Oak Ridge, Tennessee
Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World
May 9, 2026Christopher M. Bryant, Hao LiuLanguage Model Scaling Laws Neural Scaling Laws
Arena Physica

Mar 26, 2026J/K move · Enter open · S save

On Neural Scaling Laws for Weather Emulation through Continual Training

Shashank Subramanian, Alexander Kiefer, Arnur Nigmetov, Amir Gholami+2

Lawrence Berkeley National Laboratory · Oak Ridge National Laboratory · ICSI and UC Berkeley+1

Neural Scaling Laws Weather Forecasting

Open paper AskPDF arXiv

Neural scaling laws, which in some domains can predict the performance of large neural networks as a function of model, data, and compute scale, are the cornerstone of building foundation models in Natural Language Processing and Computer Vision. We study neural scaling in Scientific Machine Learning, focusing on models for weather forecasting. To analyze scaling behavior in as simple a setting as possible, we adopt a minimal, scalable, general-purpose Swin Transformer architecture, and we use continual training with constant learning rates and periodic cooldowns as an efficient training strategy. We show that models trained in this minimalist way follow predictable scaling trends and even outperform standard cosine learning rate schedules. Cooldown phases can be re-purposed to improve downstream performance, e.g., enabling accurate multi-step rollouts over longer forecast horizons as well as sharper predictions through spectral loss adjustments. We also systematically explore a wide range of model and dataset sizes under various compute budgets to construct IsoFLOP curves, and we identify compute-optimal training regimes. Extrapolating these trends to larger scales highlights potential performance limits, demonstrating that neural scaling can serve as an important diagnostic for efficient resource allocation. We open-source our code for reproducibility.

Abstract

Scaling Laws

Explore similar work

On Neural Scaling Laws for Weather Emulation through Continual Training

Towards Scaling Law Analysis For Spatiotemporal Weather Data

Practical Scaling Laws: Converting Compute into Performance in a Data-Constrained World