We investigate three distinct methods of incorporating all-sky imager (ASI) images into deep learning (DL) irradiance nowcasting. The first method relies on a convolutional neural network (CNN) to extract features directly from raw RGB images. The second method uses state-of-the-art algorithms to engineer 2D feature maps informed by domain knowledge, e.g., cloud segmentation, the cloud motion vector, solar position, and cloud base height. These feature maps are then passed to a CNN to extract compound features. The final method relies on aggregating the engineered 2D feature maps into time-series input. Each of the three methods were then used as part of a DL model trained on a high-frequency, 29-day dataset to generate multi-horizon forecasts of global horizontal irradiance up to 15 minutes ahead. The models were then evaluated using root mean squared error and skill score on 7 selected days of data. Aggregated engineered ASI features as model input yielded superior forecasting performance, demonstrating that integration of ASI images into DL nowcasting models is possible without complex spatially-ordered DL-architectures and inputs, underscoring opportunities for alternative image processing methods as well as the potential for improved spatial DL feature processing methods.
Figures & tables
Acronym
Description
PV
Photovoltaics
GHI
Global Horizontal Irradiance
ASI
All-Sky Imager
FOV
Field of View
NWP
Numerical Weather Prediction
DL
Deep Learning
TABLE I: List of acronyms used in the present work
Fig. 1: Map showing the locations of ASI 1 and ASI 2 and the pyranometer measuring GHI. The distance between the two cameras is highlighted, and the inset shows example time series data measured by the pyranometer from 13:30 CET to 14:20 CET on June 15th, 2023.
Fig. 2: Diagram of the three feature engineering and prediction methods A, B, and C. The output from either Method A, B, or C is concatenated with past GHI values as inputs into the LSTM-DNN bulk of the architecture, which is common for all three methods.
Fig. 3: Example image and feature maps extracted for ASI 1 at 2023-08-04 12:46:10 CET.
Fig. 4: Left panel shows the distribution of training and testing days with respect to time of year and the maximum GHI clear . The gray shaded line indicates days where the maximum solar elevation is below 15 ∘ . The right panel shows the distribution of training and testing days variability and clearness index with respect to 465 other days between 2022-06-09 and 2024-07-07.
Fig. 5: A comparison of RMSE and SS for models trained using the three methods shown in Fig. 2 .
Fig. 6: From the left: The first column shows GHI for each test day, along with forecasts made by Model A, B, and C shown in Fig. 2 for a 15-minute forecast horizon, the second column shows the absolute error (AE) of each forecast made at each timestep. The third column shows the RMSE, while the fourth column shows the SS as defined in Eq. ( 9 ). Note that SP refers to the smart persistence model.
Day
Vˉ
kˉ
CC
04-05
0.0061
0.56
morning: As
midday: As
evening: As
04-14
0.050
0.74
morning: Cs, Ac
midday: St
evening: As, Cu into Cl
TABLE II: Test set daily characteristics: k , V , and Cloud Classification (CC) based on the WMO genera (See Table III ). Morning is from 6–10, midday from 10–15, evening from 15–20. Slashes between CC genera indicate uncertainty in the determination of CC genera, which were determined using the WMO cloud classification guide [ 43 ] .
Shorthand/CC
Genera
Cb
Cumulunimbus
Ns
Nimbostratus
As
Altostratus
St
Stratus
Cu
Cumulus
Ac
Altocumulus
TABLE III: WMO cloud genera used in cloud classification [ 44 ] .
Fig. 7: PI calculated from Eq. 11 for the features extracted by Method C, aggregated by forecast horizon, where a higher Δ MAE indicates high feature importance. GHI clear is as calculated in Eq. 4 . The numbers ”1” and ”2” refer to the ASI from which the features were extracted (see Fig. 1 ).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 8: Example of cloud segmentation using RGB-space k-means clustering.
Fig. 9: A corellogram showing the absolute Pearson correlation between the current value of a feature and the future value of the GHI, at a time denoted on the x-axis. Calculated from the 120 339 samples in the training/validation data.
Method
A
B
C
Input features
RGB
Feature maps
Feature series
Parameters
85,228
85,300
42,640
Lookback
150 s
150 s
150 s
Horizon
15 min
15 min
15 min
Frequency
10 s
10 s
10 s
Conv. layers
6
6
-
Appendix
TABLE IV: DL Architecture Parameters for the three methods.
Fig. 10: Distribution and plots of CBH estimates in an image taken on 2023-06-15 13:32:40 CET, relating to image resolution. An area with an image glare is marked in the dashed yellow area, showing how the misidentification of this area reduces with smaller image sizes.
Sky-image irradiance forecasting is dominated by a two-stage design: a video-prediction model forecasts future sky frames, and a separately trained regressor maps those frames to irradiance. We propose a single-stage alternative that co-denoises the future sky-image and clear-sky-index sequences in one diffusion process, with an irradiance head that reads a denoised preview of the future sky rather than the noised diffusion state. The forecaster needs no regression stage and runs from sky images and deterministic covariates alone, with no measured irradiance at its input --- the situation at a site with a camera but no pyranometer. On 2023 SIRTA data the model reaches a pooled RMSE of 104.1~W,m−2 and a skill score of +0.34 on the continuous ranked probability score (CRPS) against smart persistence, higher than any of twelve baseline configurations, and retains that skill when retrained on 2021--2022. On the model's own generated frames, a separately trained regressor reaches 150.6~W,m−2 against the joint head's 104.1: roughly 46~W,m−2 that joint training recovers and a cascade discards. Point accuracy is competitive but not best: a deterministic network that also reads measured irradiance attains 97.6~W,m−2. The model's CRPS skill leads that network's in every sky-condition tertile, by 0.26 to 0.30.
Kanawut Suwandee, Suwichaya Suwanwimolkul
Chulalongkorn University, 254 Phaya Thai Rd, Wang Mai, Pathum Wan, Bangkok, 10330, Thailand
Accurate solar irradiance forecasting is essential for the reliable integration of photovoltaic power into modern electricity grids. All-sky imagers (ASI) provide high-resolution observations of clouds, making them well suited for intra-hour forecasting. Recent deep learning approaches have substantially improved forecast accuracy but are often limited by deterministic predictions and a reduced capability to anticipate ramp events. This work proposes FarSky, a generative forecasting framework that leverages latent-space coupling to learn task-aware representations of sky images. A multi-task autoencoder first learns a shared latent representation for image reconstruction and irradiance estimation. A latent diffusion model then generates future latent states conditioned on recent observations, from which irradiance forecasts are directly decoded. Probabilistic forecasts are inherently obtained through stochastic sampling. The framework is developed using a multi-year ASI dataset acquired at the Plataforma Solar de Almería, Spain, and evaluated on two independent test datasets against persistence, state-of-the-art end-to-end, and generative forecasting approaches. FarSky achieves the best overall deterministic and probabilistic forecasting performance, improving forecast skill by up to 11 percentage points. Furthermore, it substantially improves ramp event detection over existing methods, achieving F1-scores above 60%. These results demonstrate the potential of combining generative models with task-aware latent-space coupling for solar forecasting.
Yann Fabel, Bijan Nouri, Milon Miah +4
1DLR, Institute of Solar Research · 2RWTH Aachen University · 3CIEMAT, Energy Department, Renewable Energy Division
Ultra-short-term solar irradiance prediction is critical for photovoltaic system dispatch and power grid stability. Existing approaches suffer from three key shortcomings: single time-series models cannot capture the spatial dynamics of clouds under complex conditions, standard convolutions inadequately represent multi-scale cloud features, and fixed low-frequency compensation strategies fail to adapt to different prediction steps. To address these issues, this proposes a multi-source data fusion model for ultra-short-term irradiance prediction. The model first employs InceptionNeXt to extract multi-scale, multi-directional spatial features from ground-based cloud images. A step-adaptive low-frequency compensation unit is then introduced to dynamically modulate global low-frequency information based on the prediction step. Eventually, the enhanced image features are combined with meteorological time-series features, and a TempAttnLSTM network captures global temporal dependencies for multi-step prediction. Experiments on the public NREL dataset and practical photovoltaic stations in Shandong illustrate the effectiveness of the proposed method compared with several state-of-the-art approaches.
Jingxin Zhang Xiaoqin Wang
School of Automation, Southeast University, Nanjing, 210096, China