FujiView: Multimodal Late-Fusion for Predicting Scenic Visibility

Aurora visibility at a given location requires two physically distinct conditions to hold at once: aurora occurring overhead, governed by solar wind-magnetosphere coupling, and observing conditions that permit detection, governed by cloud cover and moonlight. Approaches that conflate the two weaken the space-weather signal and limit cross-site generalizability. We develop Aurora Hunter, a two-stage cascade that separates occurrence prediction from observing-condition assessment. Stage 1 uses 43 physics-driven features to predict P(aurora identifiable in all-sky images) via gradient-boosted trees trained on joint Tromso+Kiruna data (about 16,600 hours, 2015-2023). Stage 2 models P(unobscured image class | identifiable) with logistic regression on 15 observing-condition features, trained on hours with identifiable aurora. The cascade P(visible) = P(identifiable) x P(unobscured | identifiable) achieves retrospective ROC-AUC of 0.958 (Tromso test, 2019-2020) and 0.933 (independent Kiruna, 2024), improving on the occurrence stage used alone by +0.095 and +0.097. Transfer to Skibotn, the one station withheld entirely (2022-2025), is limited. SHAP analysis identifies magnetic local time, the Kp x nightside interaction and the three-hour mean Kp as the dominant features (44% of attribution), consistent with auroral oval physics. Hemisphere-wide occurrence maps combine the Stage 1 amplitude, on a clear-sky basis, with the Feldstein oval parameterization. By providing location-specific visibility probabilities with measured reliability rather than coarse geomagnetic indices, the framework links space weather research to practical observation planning. An operational proof of concept is available at https://aurora-hunter.onrender.com

The bottleneck of Earth Observation processing chains is not the arrival of new imagery but whether the surface is actually visible when the image arrives. We study this as an observability forecasting problem on EarthNet2021. Given recent multispectral imagery and exogenous weather drivers, the goal is to predict whether the next acquisition will be usable and, if not, when a usable view is likely to return. To do this, we adapt LeWorldModel, a joint-embedding predictive architecture world model, to cloud-aware Earth Observation sequences. The final pipeline converts raw minicubes into episodic HDF5 sequences with five image channels (blue, green, red, near-infrared, cloud mask) and eight meteorological and calendar covariates. The resulting model has 18.0M trainable parameters and is trained from scratch on 23,904 training episodes. The trained leWorldModel is evaluated under a locked protocol: linear probes are fit on train only, calibration choices are set on an internal validation split, and the fitted heads are then frozen for valsplit, IID, OOD, and extreme evaluation. On the full frozen-bundle observability benchmark, LeWorldModel consistently outperforms persistence. For next-step usability, balanced accuracy ranges from 0.769 to 0.887, compared with 0.493 to 0.556 for persistence. For exact first-usable-horizon prediction, accuracy ranges from 0.602 to 0.806, compared with 0.120 to 0.369 for persistence. Against a frozen LightGBM baseline fit on the same training windows, LeWorldModel is better on continuous clear/cloud regression and on exact recovery timing on valsplit, IID, and extreme, while LightGBM is stronger on the simpler binary any-usable-within-six task and is more robust on OOD. In separate sampled diagnostic analyses, LeWM also produces strong ranking-based anomaly signals under synthetic temporal inconsistencies.

Sky-image irradiance studies often compare forecasting systems in which the image encoder, temporal model, fusion block, target definition, and training recipe all change together. We use a narrower protocol: the multimodal forecasting pipeline is fixed, and only the visual backbone is varied. The shared setup keeps preprocessing, clear-sky-index normalization, weather-history encoding, fusion, regression head, loss, optimizer schedule, seed, and chronological split policy unchanged. We compare ConvNeXt, Swin Transformer, VMamba, Spatial Mamba, and MambaVision backbones for 10min-ahead forecasting on Folsom and a strict matched NREL split. Forecast skill is measured against clear-sky-index smart persistence, and temporal-only rows are reported as weather-history diagnostics rather than as the main ranking criterion. On the Folsom strict split, all evaluated visual-backbone runs improve over smart persistence. In the evaluated single-seed strict runs, VMamba Small and Swin Base reach matched Folsom RMSE values of 65.39 W/m^2 and 65.50 W/m^2; the temporal-only diagnostic reaches 69.51 W/m^2. On the 313-sample NREL strict split, smart persistence remains strongest at 17.48 W/m^2, while the lowest visual RMSE is obtained by Swin Tiny at 23.76 W/m^2. These results provide a reproducible encoder comparison under one fixed multimodal operating point rather than establishing architecture-level dominance, statistically resolved ranking, or fully optimized forecasting performance. Code available here: https://github.com/Oshadha345/irradiance_benchmark