cs.LGNov 17, 2026

An AI-Based OSSE Framework to Evaluate Learned Physical Consistency: A Multi-Model Study Using FourCastNet, Aurora, Prithvi WxC, and GraphCast

Authors: Yash Diggikar, Milton Halem, Aishwarya Raman

Organizations: University of Maryland, Baltimore County, Baltimore, MD, USA · Earth Sciences Division, NASA Ames Research Center, Moffett Field, CA, USA

Abstract

Observing System Simulation Experiments (OSSEs) are a central methodology for evaluating future atmospheric observing systems. We present the first AI-based OSSE framework to test whether different AI weather models can reproduce comparable physical impact from new observations that traditional physics based models generate. We evaluate four distinct architectural AI models on the Charney conjecture that incomplete historical observations can infer the present state of the atmospheric. We considner- FourCastNet V2, GraphCast, Aurora and Prithvi WxC and integrate each autoregressively over a 60-day forecast horizon. After a 30-day spin-up, we insert ERA5 temperature fields to infer winds and related atmospheric fields. Similar OSSE experiments are conducted for wind insertion. We measure the performance of each model's response to observational information on a 30-day data assimilation cycle forecast with and without data insertion. Distinct architectural AI weather models respond differently to the same OSSE observations. Foundation models (i.e. Prithvi WxC and Aurora), produce the strongest impact. For example, while GraphCast is the weakest, all three of the four models capture the temperature-to-geopotential height response. The two foundational models retain a positive, non-trivial, tropical zonal wind at 500 hPa response to temperature insertion. Various data insertion experiments can vary significantly even within a given architecture. We identify an architecture-specific numerical instability in GraphCast's full 37-level configuration that produces a runaway feedback concentrated at the uppermost levels with ERA5 temperature insertion correctable by excluding levels at or below 100 hPa.

Explore similar work

CardsList
  1. The physics of AI weather models

    May 22, 2026George Craig, Tobias Selz, Matthias Beylich +1AI for ScienceRepresentational Similarity Analysis

  2. Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts

    May 28, 2026Fanny Lehmann, Firat Ozdemir, Yun Cheng +4Forecasting BenchmarksWeather Forecasting

  3. PhysXAI: A Theoretical Framework for Diagnosing Physical Hallucinations in Weather Foundation Models

    Nov 11, 2026Anjan Mahapatra, Nikhil RajHallucination DetectionWeather Forecasting