Generative 3D Gaussian Splatting for Arbitrary-ResolutionAtmospheric Downscaling and Forecasting

Tao Han; Zhibin Wen; Zhenghao Chen; Fenghua Lin; Junyu Gao; Song Guo; Lei Bai

cs.CVApr 9, 2026

Generative 3D Gaussian Splatting for Arbitrary-ResolutionAtmospheric Downscaling and Forecasting

Authors: Tao Han, Zhibin Wen, Zhenghao Chen, Fenghua Lin, Junyu Gao, Song Guo, Lei Bai

Organizations: Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, 999077, Hong Kong, China · Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China · Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, Guangdong, China · School of Computer and Information Sciences, University of Newcastle, Newcastle, 2308, NSW, Australia ·

Abstract

While AI-based numerical weather prediction (NWP) enables rapid forecasting, generating high-resolution outputs remains computationally demanding due to limited multi-scale adaptability and inefficient data representations. We propose the 3D Gaussian splatting-based scale-aware vision transformer (GSSA-ViT), a novel framework for arbitrary-resolution forecasting and flexible downscaling of high-dimensional atmospheric fields. Specifically, latitude-longitude grid points are treated as centers of 3D Gaussians. A generative 3D Gaussian prediction scheme is introduced to estimate key parameters, including covariance, attributes, and opacity, for unseen samples, improving generalization and mitigating overfitting. In addition, a scale-aware attention module is designed to capture cross-scale dependencies, enabling the model to effectively integrate information across varying downscaling ratios and support continuous resolution adaptation. To our knowledge, this is the first NWP approach that combines generative 3D Gaussian modeling with scale-aware attention for unified multi-scale prediction. Experiments on ERA5 show that the proposed method accurately forecasts 87 atmospheric variables at arbitrary resolutions, while evaluations on ERA5 and CMIP6 demonstrate its superior performance in downscaling tasks. The proposed framework provides an efficient and scalable solution for high-resolution, multi-scale atmospheric prediction and downscaling. Code is available at: https://github.com/binbin2xs/weather-GS.

Figures & tables

Fig. 1: Comparison of arbitrary-resolution atmospheric forecasting methods. (a) Previous methods require separate decoders for each resolution (e.g., 600 km, 160 km, 40 km), increasing model parameters and GPU usage. (b) Our method predicts the continuous 3D Gaussians using a single decoder. Arbitrary resolutions can then be rendered directly without retraining, enabling efficient arbitrary-resolution forecasting while reducing model complexity and GPU memory cost.

Fig. 2: Overview of the GSSA-ViT framework. A low-resolution atmospheric field on a latitude–longitude grid initializes continuous 3D Gaussians, which are encoded as input representations. GSSA-ViT uses scale-aware window attention and global attention to capture resolution-scale information and spatial dependencies, predicting Gaussian parameters. The Gaussians are rendered into high-resolution atmospheric fields at arbitrary resolutions, where different resolutions correspond to different sampling densities.

Upper-Air			Surface
Name	Description	Levels	Name	Description	Name	Description
Z	Geopotential	13	U10	x-direction wind at 10m height	U100	x-direction wind at 100m height
Q	Specific humidity	13	V10	y-direction wind at 10m height	V100	y-direction wind at 100m height
U	x-direction wind	13	T2M	Temperature at 2m height	TCC	total cloud cover
V	y-direction wind	13	MSL	Mean sea-level pressure	D2M	2-meter dewpoint temperature
T	Temperature	13	TP6H	total precipitation

Table 1: A summary of atmospheric variables. Specifically, the upper-air variables are available at 13 standard pressure levels, namely 50, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, and 1000 hPa.

MPI-ESM (5.625 ∘ ) to ERA5 (1.40625 ∘ )
Methods	Z500			T850			T2M			U10			V10
	LRMSE $\downarrow$	P $\uparrow$	M-b	LRMSE $\downarrow$	P $\uparrow$	M-b	LRMSE $\downarrow$	P $\uparrow$	M-b	LRMSE $\downarrow$	P $\uparrow$	M-b	LRMSE $\downarrow$	P $\uparrow$	M-b
Bicubic	1142.43	0.92	71.36	4.80	0.93	0.11	4.07	0.97	-0.05	5.49	0.44	-0.06	5.57	0.20	0.00
Bilinear	1114.65	0.92	71.23	4.64	0.94	0.10	3.97	0.97	-0.05	5.24	0.45	-0.06	5.34	0.20	0.00
ResNet [ 35 , 18 ]	825.75	0.96	-108.54	3.60	0.96	0.19	2.89	0.98	0.14	4.05	0.65	0.06	4.11	0.45	0.09
Unet [ 35 , 39 ]	858.35	0.95	35.10	3.66	0.96	-0.34	2.95	0.98	0.16	4.09	0.64	-0.06	4.13	0.44	0.08

Table 2: Performance comparison of atmospheric downscaling from MPI-ESM (5.625°) to ERA5 at three target resolutions. The first section reports results at 1.40625°, while the remaining sections evaluate finer resolutions (0.703125° and 0.3515625°). Lower LRMSE indicates better performance, higher P reflects stronger correlation, and M-b values closer to zero indicate smaller bias. The best results are highlighted in bold , and the second-best results are underlined .

Fig. 3: Performance comparison of different downscaling methods under varying downscaling ratios. The LRMSE is reported for five atmospheric variables (Z500, T850, T2m, U10, and V10) when downscaling from the CMIP (5.625°) to ERA5 targets with ratios ranging from ×4 to ×16. Lower values indicate better reconstruction accuracy.

Fig. 4: Global visualization of downscaling results from CMIP6 (5.625°) to ERA5 at 1.40625° resolution (×4). Each column corresponds to an atmospheric variable Z500, T850, V10, U10, and T2M. Each row shows the ground truth (GT) followed by outputs from six baselines Bilinear, Bicubic, MetaSR, LIIF, MINet, GSASR, and GSSA-ViT (Ours).

Fig. 5: Global visualization of downscaling results from CMIP6 (5.625°) to ERA5 at 0.703125° resolution (×8). Each column corresponds to an atmospheric variable Z500, T850, V10, U10, and T2M. Each row shows the ground truth (GT) followed by outputs from six baselines Bilinear, Bicubic, MetaSR, LIIF, MINet, GSASR, and GSSA-ViT (Ours).

Fig. 6: Global visualization of downscaling results from CMIP6 (5.625°) to ERA5 at 0.3515625° resolution (×16). Each column corresponds to an atmospheric variable Z500, T850, V10, U10, and T2M. Each row shows the ground truth (GT) followed by outputs from six baselines Bilinear, Bicubic, MetaSR, LIIF, MINet, GSASR, and GSSA-ViT (Ours).

Fig. 7: Regional visualization of downscaling results from CMIP6 (5.625°) to ERA5 at 1.40625° resolution (×4), focusing on the region spanning 10°–30°N and 45°–65°E. Each row corresponds to an atmospheric variable Z500, T850, V10, U10, and T2M. Each column shows the ground truth (GT) followed by outputs from six baselines Bilinear, Bicubic, MetaSR, LIIF, MINet, GSASR, and GSSA-ViT (Ours).

Methods	Z500		T850		T2M
	LRMSE $\downarrow$	M-b	LRMSE $\downarrow$	M-b	LRMSE $\downarrow$	M-b
Bicubic	269.67	0.04	1.99	0.00	3.11	0.00
Bilinear	134.07	0.04	1.50	0.00	2.46	0.00
GSASR [ 5 ]	134.44	-76.79	1.23	-0.44	1.79	-0.77
Unet [ 35 , 39 ]	43.84	-6.55	0.94	-0.06	1.10	-0.12
ViT [ 35 , 14 ]	85.32	-35.98	1.03	-0.01	1.25	-0.20

Table 3: Performance comparison of atmospheric downscaling from ERA5 (5.625°) to ERA5 (2.8125°). Performance is evaluated using latitude-weighted RMSE (LRMSE) and mean bias (M-b) for three variables (Z500, T850, T2m). Lower LRMSE indicates better performance, while M-b values closer to zero indicate smaller bias.

	Z500 $\downarrow$				T850 $\downarrow$				Q700 $\downarrow$				Wind850 $\downarrow$
Lead Time	6h	24h	72h	120h	6h	24h	72h	120h	6h	24h	72h	120h	6h	24h	72h	120h
ERA5 (1.40625 ∘ ) to ERA5 (0.703125 ∘ )
MetaSR [ 21 ]	58.09	195.66	722.96	1285.11	0.67	1.33	3.56	5.57	-	-	-	-	-	-	-	-
LIIF [ 11 ]	71.88	243.37	791.08	1276.04	0.76	1.55	3.74	5.58	-	-	-	-	-	-	-	-
MINet [ 10 ]	70.64	231.89	752.94	1238.48	0.72	1.43	3.65	5.44	-	-	-	-	-	-	-	-
NeuralGCM [ 25 ] (Bicubic)	81.92	95.21	174.56	333.48	0.86	1.05	1.41	1.76	0.69	0.83	0.97	1.25	2.11	2.52	3.76	5.15

Table 4: Medium-range forecasting performance at three resolutions. NeuralGCM (native resolution 1.40625 ∘ ) and Stormer (native resolution 1.40625 ∘ ) predictions are downscaled to different target resolutions using bilinear and bicubic interpolation, as well as three strong downscaling models (MetaSR, LIIF, and MINet), which are adapted to the forecasting setting by shifting ground-truth targets to the next time step. Results are reported as LRMSE for Z500, T850, Q700, and Wind850 at lead times of 6h, 24h, 72h, and 120h.

	T2M $\downarrow$				U10 $\downarrow$				V10 $\downarrow$				MSL $\downarrow$
Lead Time	6h	24h	72h	120h	6h	24h	72h	120h	6h	24h	72h	120h	6h	24h	72h	120h
ERA5 (1.40625 ∘ ) to ERA5 (0.703125 ∘ )
MetaSR [ 21 ]	0.88	1.73	4.70	6.90	0.80	1.65	3.88	5.12	0.83	1.73	3.99	5.10	-	-	-	-
LIIF [ 11 ]	0.98	2.52	4.41	5.73	0.90	1.90	4.52	6.06	0.94	1.98	4.67	6.17	-	-	-	-
MINet [ 10 ]	0.93	2.31	4.32	5.48	0.88	1.76	4.36	5.67	0.89	1.88	4.34	5.74	-	-	-	-
Stormer [ 36 ] (Bicubic)	1.40	1.42	1.64	1.91	1.05	1.26	1.68	2.49	1.11	1.29	1.88	2.51	88.99	104.42	177.32	322.64

Table 5: Medium-range forecasting performance at three resolutions. NeuralGCM (native resolution 1.40625 ∘ ) and Stormer (native resolution 1.40625 ∘ ) predictions are downscaled to different target resolutions using bilinear and bicubic interpolation, as well as three strong downscaling models (MetaSR, LIIF, and MINet), which are adapted to the forecasting setting by shifting ground-truth targets to the next time step. Results are reported as LRMSE for T2M, U10, V10, and MSL at lead times of 6h, 24h, 72h, and 120h.

Fig. 8: Performance comparison for global arbitrary-resolution prediction. The LRMSE is reported for five atmospheric variables (Z500, T850, T2M, U10, and V10). Subplots (a)–(c) correspond to predictions from ERA5 (1.40625°) to target resolutions of 0.703125°, 0.3515625°, and 0.24965326°, respectively. Results are evaluated at multiple lead times of 6, 24, 48, 72, 96, and 120 hours.

Fig. 9: Performance comparison of our model under 6-hour arbitrary-resolution prediction settings across multiple atmospheric variables and vertical pressure levels. The LRMSE is reported for five variables (Z, Q, T, U, and V) as a function of pressure level (hPa), with results shown under different downscaling ratios (×2, ×4, and ×5.6). Across all variables, the model exhibits consistent performance across different downscaling ratios, with only marginal variation in error, indicating strong robustness to changes in resolution.

Fig. 10: Global visualization of 6-hour arbitrary-resolution prediction from ERA5 at 1.40625° to 0.25° resolution. Each column corresponds to upper-level variables Z500, T850, U850, V850, and Q700. The first row shows the ground truth (ERA5 at 0.25°). Subsequent rows present results from different methods. Stormer (Bicubic) and Stormer (Bilinear) denote interpolations of Stormer predictions at native 1.40625° resolution to 0.25° using bicubic and bilinear schemes, respectively; NeuralGCM (Bicubic) and NeuralGCM (Bilinear) are defined analogously. The final row shows GSSA-ViT (Ours), which directly predicts at 0.25° resolution.

Fig. 11: Global visualization of 6-hour arbitrary-resolution prediction from ERA5 at 1.40625° to 0.25° resolution. Each row corresponds to surface-level variables T2M, U10, and V10. The first column shows the ground truth (ERA5 at 0.25°). Subsequent columns present results from different methods, including MetaSR, LIIF, MINet, Stormer (Bicubic), Stormer (Bilinear), and GSSA-ViT (Ours). Stormer (Bicubic) and Stormer (Bilinear) denote interpolations of Stormer predictions at native 1.40625° resolution to 0.25° using bicubic and bilinear schemes, respectively. The final column shows GSSA-ViT (Ours), which directly predicts at 0.25° resolution.

Fig. 12: Regional visualization of 6-hour arbitrary-resolution prediction from ERA5 at 1.40625° to 0.3515625° resolution. Each row corresponds to surface-level variables T2M, U10, and V10. The first column shows the ground truth (ERA5 at 0.3515625°). Subsequent columns present results from different methods, including MetaSR, LIIF, MINet, Stormer (Bicubic), Stormer (Bilinear), and GSSA-ViT (Ours). Stormer (Bicubic) and Stormer (Bilinear) denote interpolations of Stormer predictions at native 1.40625° resolution to 0.3515625° using bicubic and bilinear schemes, respectively. The final column shows GSSA-ViT (Ours), which directly predicts at 0.3515625° resolution.

Fig. 13: Regional visualization of 6-hour arbitrary-resolution prediction from ERA5 at 1.40625° to 0.3515625° resolution. Each row corresponds to surface-level variables T2M, U10, and V10. The first column shows the ground truth (ERA5 at 0.3515625°). Subsequent columns present results from different methods, including MetaSR, LIIF, MINet, Stormer (Bicubic), Stormer (Bilinear), and GSSA-ViT (Ours). Stormer (Bicubic) and Stormer (Bilinear) denote interpolations of Stormer predictions at native 1.40625° resolution to 0.3515625° using bicubic and bilinear schemes, respectively. The final column shows GSSA-ViT (Ours), which directly predicts at 0.3515625° resolution.

Method	Pos. Fixed	Gaussian Params Fixed	Decoder	Gaussian Num	Z500 $\downarrow$	T850 $\downarrow$	T2M $\downarrow$	U10 $\downarrow$	V10 $\downarrow$
(1) w/o Fixed Pos.	✗	✗	2 heads	2048	874.90	3.93	3.68	4.84	4.79
(2) w/ Fixed Gaussian Params	✓	✓	2 heads	2048	930.29	4.05	3.88	4.76	4.81
(3) w/o Gaussian Head	✓	✗	1 head	2048	678.44	3.32	2.90	3.82	3.91
(4) w/ More Gaussians	✓	✗	2 heads	8192	718.36	3.29	2.89	3.80	3.93
(5) w/ Fewer Gaussians	✓	✗	2 heads	1024	758.94	3.57	3.01	3.88	3.95
GSSA-ViT	✓	✗	2 heads	2048	658.84	3.20	2.83	3.71	3.87

Table 6: Ablation study on atmospheric downscaling from low-resolution CMIP data (5.6 ∘ ) to high-resolution ERA5 data (1.4 ∘ ). We report LRMSE (lower is better) across five meteorological variables, including Z500, T850, T2M, U10, and V10).

Weather Forecasting

Weather forecasting predicts future atmospheric conditions, from local storms to global patterns, using physical models, observations, and machine learning.

See all 175 papers

Downscaling weather forecasts from Low- to High-Resolution with Diffusion Models
Mar 30, 2026Joffrey Dumont Le Brazidec, Simon Lang, Martin Leutbecher +9Weather Forecasting Probabilistic Forecasting
European Centre for Medium-Range Weather Forecasts (ECMWF)
MR-GNF: Multi-Resolution Graph Neural Forecasting on Ellipsoidal Meshes for Efficient Regional Weather Prediction
Mar 13, 2026Andrii Shchur, Inna Skarga-BandurovaWeather Forecasting Graph Neural Networks
Kyiv School of Economics · Oxford Brookes University
AirCast-SR: A Foundation Model for Kilometer-Scale Atmospheric Super-Resolution via Latent Consistency Diffusion
May 20, 2026Somnath Luitel, Manmeet Singh, Joshua Durkee +11Geospatial Foundation Models Spatiotemporal Forecasting
Department of Earth, Environmental, and Atmospheric Sciences, Western Kentucky University, Bowling Green, KY, USA · NASA Goddard Space Flight Center, Greenbelt, MD, USA · The University of Texas at Austin, Austin, TX, USA +3

Mar 30, 2026J/K move · Enter open · S save

Downscaling weather forecasts from Low- to High-Resolution with Diffusion Models

Joffrey Dumont Le Brazidec, Simon Lang, Martin Leutbecher, Baudouin Raoult+8

European Centre for Medium-Range Weather Forecasts (ECMWF)

Weather Forecasting Probabilistic Forecasting Diffusion Models

Open paper AskPDF arXiv

We introduce a probabilistic diffusion-based method for global atmospheric downscaling implemented within the Anemoi framework. The approach transforms low-resolution ensemble forecasts into high-resolution ensembles by learning the conditional distribution of finer-scale residuals, defined as the difference between the high-resolution fields and the interpolated low-resolution inputs. The system is trained on reforecast pairs from ECMWF IFS, using coarse fields at 100 km to reconstruct fine-scale variability at 30 km resolution. The bulk of the training focuses on recovering small-scale structures, while fine-tuning in high-noise regimes enables the generation of extremes. Evaluation against the medium-range IFS ensemble target shows that the model increases probabilistic skill (FCRPS) for surface variables, reproduces target power spectra at small scales, captures physically consistent multivariate relationships such as wind-pressure coupling, and generates extreme values consistent with those of the target ensemble in tropical cyclones.

Abstract

Weather Forecasting

Explore similar work

Downscaling weather forecasts from Low- to High-Resolution with Diffusion Models

MR-GNF: Multi-Resolution Graph Neural Forecasting on Ellipsoidal Meshes for Efficient Regional Weather Prediction

AirCast-SR: A Foundation Model for Kilometer-Scale Atmospheric Super-Resolution via Latent Consistency Diffusion