Organizations: State Key Laboratory of Dark Matter Physics, Tsung-Dao Lee Institute & School of Physics and Astronomy, Shanghai Jiao Tong University, Shanghai 201210, China · National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, China · School of Computer Technology and Application, Qinghai University, Xining 810016, China
Ground-based time-domain observatories require minute-by-minute, site-scale awareness of cloud cover, yet existing all-sky datasets are short, daylight-biased, or lack astrometric calibration. We present LenghuSky-8, an eight-year (2018-2025) all-sky imaging dataset from a premier astronomical site, comprising 429,620 512×512 frames with 81.2% night-time coverage, star-aware cloud masks, background masks, and per-pixel altitude-azimuth (Alt-Az) calibration. For robust cloud segmentation across day, night, and lunar phases, we train a linear probe on DINOv3 local features and obtain 93.3% ± 1.1% overall accuracy on a balanced, manually labeled set of 1,111 images. Using stellar astrometry, we map each pixel to local alt-az coordinates and measure calibration uncertainties of approximately 0.37 deg at zenith and approximately 1.34 deg at 30 deg altitude, sufficient for integration with telescope schedulers. Beyond segmentation, we introduce a short-horizon nowcasting benchmark over per-pixel three-class logits (sky/cloud/contamination) with four baselines: persistence (copying the last frame), optical flow, ConvLSTM, and VideoGPT. ConvLSTM performs best but yields only limited gains over persistence, underscoring the difficulty of near-term cloud evolution. We release the dataset, calibrations, and an open-source toolkit for loading, evaluation, and scheduler-ready alt-az maps to boost research in segmentation, nowcasting, and autonomous observatory operations.
Figures & tables
Figure 1: Samples from the dataset. First column represents the raw images that is augmented for cloud segmentation; Second column is the annotation of clear sky (blue), cloudy region (orange) and contamination region (pink); Third column is the mask for background; Forth column is the overlay of the first three columns; Last two columns are the astrometric calibration results for altitude and azimuth of the image.
Figure 2: Workflow of this paper. Solid arrows denote dependencies among product data; dashed arrows denote potential dependencies not considered in our experiments.
Figure 3: Daily number of captured frames in the dataset.
Duration of persistence [days]
Total time (days)
Total number of frames
Proportion of time
Proportion of frames
5
2176.20
388586
0.7996
0.9045
10
1540.03
273491
0.5658
0.6366
30
1000.95
179256
0.3678
0.4172
60
625.85
117792
0.2300
0.2742
Table 1: Summary of persistence statistics for continuous observations.
Start
End
Duration (days)
Num Frames
2018-12-10 13:32:44
2019-04-19 10:02:24
129.85
24747
2020-12-06 07:22:28
2021-03-04 01:50:38
87.77
18034
2020-01-26 19:52:42
2020-04-12 06:07:00
76.43
14002
2021-05-31 22:22:44
2021-08-14 16:32:27
74.76
12206
2023-01-20 06:53:42
2023-03-27 16:43:51
66.41
12267
2019-11-20 17:12:31
2020-01-25 17:26:05
66.01
13083
Table 2: Observations with hour-level persistence for more than 1 month
Start time (UTC+8)
New astrometry
Roof position
Number of images
Duration [day]
Background proportion
2018-05-01 00:02:44
Yes
–
22300
140.7
0.1917
2018-09-27 19:19:49
Yes
–
34772
203.6
0.1357
2019-04-24 15:39:36
Yes
–
9493
60.9
0.1702
2019-06-26 18:23:18
Yes
–
1343
8.7
0.1875
2019-07-05 11:59:14
Yes
–
70477
418.2
0.1902
2020-08-26 17:24:23
No
–
3124
20.9
0.2093
Table 3: Change of background during the observation. Background proportion is the ratio of the spherical FoV and area of the mask of background.
Start time (UTC+8)
σa [deg]
σr [pixel]
2018-05-01 00:02:44
0.569
4.494
2018-09-27 19:19:49
1.382
9.016
2019-04-24 15:39:36
0.774
4.298
2019-06-26 18:23:18
0.814
5.239
2019-07-05 11:59:14
1.978
5.958
2023-09-27 18:09:48
2.256
15.086
Table 4: Residual of astrometric calibration. σa is the residual in the azimuth; σr is the residual in radial, which is measured in pixel.
Baseline
Accuracy
Macro Precision
Macro Recall
Macro F1-score
DINOv3 local
0.933−0.011+0.011
0.894−0.028+0.020
0.867−0.037+0.026
0.873−0.019+0.028
DINOv3 local + CLS
0.930−0.011+0.013
0.890−0.021+0.013
0.853−0.029+0.045
0.868−0.021+0.022
DINOv3 local + CLS + register
0.924−0.010+0.020
0.897−0.030+0.008
0.841−0.031+0.046
0.860−0.023+0.033
DINOv3 local + mean( CLS + register)
0.934−0.012+0.010
0.892−0.020+0.017
0.866−0.036+0.036
0.872−0.014+0.020
Encoder-decoder
0.804−0.027+0.014
0.771−0.025+0.037
0.655−0.016+0.033
0.686−0.033+0.022
U-Net
0.851−0.020+0.031
0.839−0.080+0.030
0.699−0.034+0.022
0.728−0.048+0.015
Table 5: Overall metrics for cloud segmentation(median with 16th-83rd percentiles)
Baseline
Accuracy
Precision
Recall
F1-score
Trivial
0.888
0.859
0.858
0.858
Optical Flow
0.888
0.858
0.860
0.859
ConvLSTM
0.890
0.871
0.849
0.859
VideoGPT-1
0.871
0.841
0.823
0.831
VideoGPT-2
0.872
0.840
0.825
0.831
VideoGPT-7
0.870
0.841
0.818
0.827
Table 6: Overall metrics for weather nowcast. VideoGPT- n refers to using VideoGPT to predict the next frame according to previous n frames; Accuracy, precision, recall, and F1-score are measure by the macro average among sky, cloud and contamination class.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Failure cases of the all-sky camera. Top row (left to right): (1) Covered by dust or sand; (2) Covered by dew or ice; (3) Scattered light caused by mud coverage; (4) Covered by snow. Bottom row (left to right): (5) Obstruction by an external object; (6) Strong nearby light source; (7) Strong distant light source; (8) Camera malfunction.
Reason
# occur
# frame
Down time [days]
cover
181
30111
169.79
stronglight
388
5275
23.54
cameradown
56
2167
20.75
object
40
735
4.66
Total
665
38288
218.75
Appendix
Table 7: Statistics of failure case of all-sky camera. The events that occurs within 12 hours are merged together as one event.
Name
Formula
Gnomonic projection
r=ftanθ
Stereographic projection
r=2ftan2θ
Equidistant projection
r=2fθ
Equisolid angle projection
r=2fsin2θ
Orthographic projection
r=fsinθ
Appendix
Table 8: Optional projection types of full-sky camera, which is provided by [ 19 ] .
Figure 5: Manual annotation for cloud segmentation. Blue represents cloud regions; orange represents sky regions; Pink represents contamination regions; Columns represents images taken around given time in UTC+8; Rows represents images taken in different moon phase condition. Nearby frames are used by human to determine whether some regions are scatter light or cloud.
Figure 6: Altitude–Azimuth fitting results for different time slots. Red boxes means that the WCS of the corresponding HEALPix cell is not resolved by Astrometry.net for any image in the ensemble, and the corresponding altitude and azimuth is obtained by fitting results.
Figure 7: Astrometric calibration fitting residual for different time slots in radial and azimuthal direction.
Figure 8: Annotation of all background
Stage
Layer (in → out)
Operation
k/s/p
Channel out
Size (H × W)
Enc-1
3 → 64
Conv2d + ReLU + BN
3/1/1
64
512×512
Enc-1
64 → 64
Conv2d + ReLU + BN
3/1/1
64
512×512
Enc-1 ↓
–
MaxPool2d
2/2/0
64
256×256
Enc-2
64 → 128
Conv2d + ReLU + BN
3/1/1
128
256×256
Enc-2
128 → 128
Conv2d + ReLU + BN
3/1/1
128
256×256
Enc-2 ↓
–
MaxPool2d
2/2/0
128
128×128
Appendix
Table 9: Layer-wise specification of Encoder-Decoder (CloudSegNet) for a 3×512×512 input. “k/s/p” denotes kernel/stride/padding.
Stage
Layer (in → out)
Operation
k/s/p
Ch. out
Size (H × W)
Enc-0
3 → 64
Conv2d + ReLU + BN
3/1/1
64
512×512
Enc-0
64 → 64
Conv2d + ReLU + BN
3/1/1
64
512×512
Down-1 ↓
–
MaxPool2d
2/2/0
64
256×256
Down-1
64 → 128
Conv2d + ReLU + BN
3/1/1
128
256×256
Down-1
128 → 128
Conv2d + ReLU + BN
3/1/1
128
256×256
Down-2 ↓
–
MaxPool2d
2/2/0
128
128×128
Appendix
Table 10: Layer-wise specification of the U-Net used in our experiments (bilinear upsampling variant) for a 3×512×512 input.
Stage
Layer (in → out)
Operation
k/s/p
Ch. out
Size (H × W)
LSTM-1
3 → 64
ConvLSTM2D + Dropout
3/1/1
64
H×W
LSTM-2
64 → 64
ConvLSTM2D + Dropout
3/1/1
64
H×W
Head
64 → 3
Conv2d (readout)
1/1/0
3
H×W
Appendix
Table 11: Layer-wise specification of the ConvLSTM next-frame baseline. Input is a [B,C=3,Tin=2,H,W] clip; output is a single frame [B,3,H,W] . “k/s/p” denotes kernel/stride/padding.
VQ-VAE Encoder
Enc-1
3 → 64
Conv2d + BN + ReLU + ResidualBlock
4/2/1
64
32×32
Enc-2
64 → 128
Conv2d + BN + ReLU + ResidualBlock
4/2/1
128
16×16
Enc-3
128 → 256
Conv2d + BN + ReLU + ResidualBlock
4/2/1
256
8×8
Bottleneck
256 → 256
Conv2d + BN + ReLU
3/1/1
256
8×8
Vector Quantiser
VQ
256 →K
VectorQuantizer ( K=512 , D=256 )
–
K
T×8×8
Appendix
Table 12: Architecture of the VideoGPT nowcaster (VQ-VAE + GPT). For a [B,3,T,64,64] input, the encoder downsamples to [B,256,T,8,8] , which is vector-quantised (codebook size K ). Tokens are modeled autoregressively by a Transformer and decoded back to frames.
Class
Baseline
Precision
Recall
F1-score
Sky
Trivial
0.914
0.913
0.914
Optica lFlow
0.913
0.913
0.913
ConvLSTM
0.922
0.905
0.913
VideoGPT-1
0.920
0.882
0.900
VideoGPT-2
0.915
0.887
0.901
VideoGPT-7
0.919
0.883
0.900
Appendix
Table 13: Per-class metrics for nowcasting
Class
Baseline
Precision
Recall
F1-score
Sky
DINOv3 local
0.920−0.022+0.028
0.945−0.009+0.018
0.936−0.015+0.006
DINOv3 local + CLS
0.917−0.025+0.037
0.941−0.014+0.019
0.932−0.013+0.012
DINOv3 local + CLS + register
0.913−0.031+0.038
0.942−0.025+0.018
0.926−0.012+0.015
DINOv3 local + mean(CLS + register)
0.921−0.019+0.031
0.944−0.019+0.018
0.936−0.016+0.009
Encoder-Decoder
0.764−0.027+0.061
0.868−0.034+0.056
0.816−0.009+0.021
U-Net
0.869−0.039+0.023
0.912−0.047+0.025
0.882−0.016+0.017
Appendix
Table 14: Per-class metrics for cloud segmentation(median with 16th-83rd percentiles)
Accurate ground-based cloud classification is important for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate observation systems. However, reliable sky-image annotation is time-consuming, especially when cloud types are visually similar or mixed. We study the label efficiency of deep learning for ground-based cloud classification using the Ground-based Cloud Dataset (GCD). Rather than proposing a new architecture, we benchmark three practical strategies under limited annotation budgets: supervised transfer learning, uncertainty-based active learning, and high-confidence pseudo-labeling. An ImageNet-pretrained ResNet50 is used as a common frozen backbone, with experiments repeated over five random seeds for label budgets from 1% to 100% of the training labels. Supervised transfer learning is already highly label-efficient: test accuracy increases from 0.635±0.018 with 1% labels to 0.730±0.002 with 40% labels, approaching the full-label result of 0.735±0.003. Active learning and pseudo-labeling are competitive with supervised sampling and provide small improvements for some metrics and budgets, but neither gives a large or consistent aggregate gain. Diagnostic analyses show that accepted pseudo-labels are reliable, with accuracy from 0.946 to 0.977, but biased toward easier high-confidence sky-type groups. In contrast, uncertainty sampling preferentially queries visually challenging groups, including Mixed and the confusable Stratocumulus and Cumulonimbus groups, but these targeted acquisitions yield only modest gains. Overall, transfer learning substantially reduces annotation requirements for GCD, while simple active and semi-supervised strategies provide limited additional benefit over a strong supervised baseline.
CEREMADE, Universit´e Paris Dauphine-PSL, Paris, France. · School of Computing and Information Science, Anglia Ruskin University, Cambridge, United Kingdom. · Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland.
Earth observation imagery plays a critical role in environmental monitoring, urban planning, disaster assessment, and climate analysis. While multi-spectral sensors are increasingly available, true-color (RGB) imagery remains widely used due to the power, cost, and deployment constraints of many satellite and aerial platforms. However, existing land-cover segmentation datasets are often limited in geographic coverage, scale, or public accessibility. To bridge this gap, we introduce BELDE (Building a Large-scale Earth-observation Land-cover Dataset for Europe), a publicly available dataset tailored for RGB-based remote sensing semantic segmentation. Constructed from Sentinel-2 true-color images and ESA WorldCover data annotations, BELDE contains 1,088,385 curated image-segmentation map pairs spanning Europe with 7 land-cover classes at 10 m spatial resolution, making it one of the largest publicly available RGB land-cover segmentation datasets for Earth observation. To facilitate cross-region generalization studies, we additionally introduce BELDE-K (16,607 pairs) covering the Republic of Korea and BELDE-CA-NV (88,155 pairs) covering California and Nevada in the United States. We establish baseline results using multiple semantic segmentation architectures and evaluate both in-domain and cross-domain performance. Models trained on BELDE achieve an F1 score of 83.0% on the European test set, while performance decreases to 66.4% on BELDE-CA-NV and 58.3% on BELDE-K, highlighting the challenges posed by out-of-distribution geographic domain shift. By providing a continental-scale RGB segmentation and evaluation benchmark, BELDE supports the development of robust and transferable Earth observation models. The dataset and benchmark resources will be publicly released.
Ümit Mert Çağlar, Alptekin Temizel
Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
We introduce Sky2Ground, a three-view dataset designed for varying altitude camera localization, correspondence learning, and reconstruction. The dataset combines structured synthetic imagery with real, in-the-wild images, providing both controlled multi-view geometry and realistic scene noise. Each of the 51 sites contains thousands of satellite, aerial, and ground images spanning wide altitude ranges and nearly orthogonal viewing angles, enabling rigorous evaluation across global-to-local contexts. We benchmark state of the art pose estimation models, including MASt3R, DUSt3R, Map Anything, and VGGT, and observe that the use of satellite imagery often degrades performance, highlighting the challenges under large altitude variations. We also examine reconstruction methods, highlighting the challenges introduced by sparse geometric overlap, varying perspectives, and the use of real imagery, which often introduces noise and reduces rendering quality. To address some of these challenges, we propose SkyNet, a model which enhances cross-view consistency when incorporating satellite imagery with a curriculum-based training strategy to progressively incorporate more satellite views. SkyNet significantly strengthens multi-view alignment and outperforms existing methods by 9.6% on RRA@5 and 18.1% on RTA@5 in terms of absolute performance. Sky2Ground and SkyNet together establish a comprehensive testbed and baseline for advancing large-scale, multi-altitude 3D perception and generalizable camera localization. Code and models will be released publicly for future research.Project page: https://sky2ground2026.github.io/sky2ground/
Zengyan Wang, Sirshapan Mitra, Rajat Modi +2
Center for Research In Computer Vision, University of Central Florida