LenghuSky-8: An 8-Year All-Sky Cloud Dataset with Star-Aware Masks and Alt-Az Calibration for Segmentation and Nowcasting

Yicheng Rui; Xiao-Wei Duan; Licai Deng; Fan Yang; Zhengming Dang; Zhengjun Du; Junhao Peng; Wenhao Chu

astro-ph.IMMar 17, 2026

LenghuSky-8: An 8-Year All-Sky Cloud Dataset with Star-Aware Masks and Alt-Az Calibration for Segmentation and Nowcasting

Authors: Yicheng Rui, Xiao-Wei Duan, Licai Deng, Fan Yang, Zhengming Dang, Zhengjun Du, Junhao Peng, Wenhao Chu,

Organizations: State Key Laboratory of Dark Matter Physics, Tsung-Dao Lee Institute & School of Physics and Astronomy, Shanghai Jiao Tong University, Shanghai 201210, China · National Astronomical Observatories, Chinese Academy of Sciences, Beijing 100101, China · School of Computer Technology and Application, Qinghai University, Xining 810016, China

Abstract

Ground-based time-domain observatories require minute-by-minute, site-scale awareness of cloud cover, yet existing all-sky datasets are short, daylight-biased, or lack astrometric calibration. We present LenghuSky-8, an eight-year (2018-2025) all-sky imaging dataset from a premier astronomical site, comprising 429,620 $512 \times 512$ frames with 81.2% night-time coverage, star-aware cloud masks, background masks, and per-pixel altitude-azimuth (Alt-Az) calibration. For robust cloud segmentation across day, night, and lunar phases, we train a linear probe on DINOv3 local features and obtain 93.3% $\pm$ 1.1% overall accuracy on a balanced, manually labeled set of 1,111 images. Using stellar astrometry, we map each pixel to local alt-az coordinates and measure calibration uncertainties of approximately 0.37 deg at zenith and approximately 1.34 deg at 30 deg altitude, sufficient for integration with telescope schedulers. Beyond segmentation, we introduce a short-horizon nowcasting benchmark over per-pixel three-class logits (sky/cloud/contamination) with four baselines: persistence (copying the last frame), optical flow, ConvLSTM, and VideoGPT. ConvLSTM performs best but yields only limited gains over persistence, underscoring the difficulty of near-term cloud evolution. We release the dataset, calibrations, and an open-source toolkit for loading, evaluation, and scheduler-ready alt-az maps to boost research in segmentation, nowcasting, and autonomous observatory operations.

Figures & tables

Figure 1: Samples from the dataset. First column represents the raw images that is augmented for cloud segmentation; Second column is the annotation of clear sky (blue), cloudy region (orange) and contamination region (pink); Third column is the mask for background; Forth column is the overlay of the first three columns; Last two columns are the astrometric calibration results for altitude and azimuth of the image.

Figure 2: Workflow of this paper. Solid arrows denote dependencies among product data; dashed arrows denote potential dependencies not considered in our experiments.

Figure 3: Daily number of captured frames in the dataset.

Duration of persistence [days]	Total time (days)	Total number of frames	Proportion of time	Proportion of frames
5	2176.20	388586	0.7996	0.9045
10	1540.03	273491	0.5658	0.6366
30	1000.95	179256	0.3678	0.4172
60	625.85	117792	0.2300	0.2742

Table 1: Summary of persistence statistics for continuous observations.

Start	End	Duration (days)	Num Frames
2018-12-10 13:32:44	2019-04-19 10:02:24	129.85	24747
2020-12-06 07:22:28	2021-03-04 01:50:38	87.77	18034
2020-01-26 19:52:42	2020-04-12 06:07:00	76.43	14002
2021-05-31 22:22:44	2021-08-14 16:32:27	74.76	12206
2023-01-20 06:53:42	2023-03-27 16:43:51	66.41	12267
2019-11-20 17:12:31	2020-01-25 17:26:05	66.01	13083

Table 2: Observations with hour-level persistence for more than 1 month

Start time (UTC+8)	New astrometry	Roof position	Number of images	Duration [day]	Background proportion
2018-05-01 00:02:44	Yes	–	22300	140.7	0.1917
2018-09-27 19:19:49	Yes	–	34772	203.6	0.1357
2019-04-24 15:39:36	Yes	–	9493	60.9	0.1702
2019-06-26 18:23:18	Yes	–	1343	8.7	0.1875
2019-07-05 11:59:14	Yes	–	70477	418.2	0.1902
2020-08-26 17:24:23	No	–	3124	20.9	0.2093

Table 3: Change of background during the observation. Background proportion is the ratio of the spherical FoV and area of the mask of background.

Start time (UTC+8)	$\sigma_{\text{a}}$ [deg]	$\sigma_{\text{r}}$ [pixel]
2018-05-01 00:02:44	0.569	4.494
2018-09-27 19:19:49	1.382	9.016
2019-04-24 15:39:36	0.774	4.298
2019-06-26 18:23:18	0.814	5.239
2019-07-05 11:59:14	1.978	5.958
2023-09-27 18:09:48	2.256	15.086

Table 4: Residual of astrometric calibration.

\sigma_{a}

is the residual in the azimuth;

\sigma_{r}

is the residual in radial, which is measured in pixel.

Baseline	Accuracy	Macro Precision	Macro Recall	Macro F1-score
DINOv3 local	$0.933_{-0.011}^{+0.011}$	$0.894_{-0.028}^{+0.020}$	$0.867_{-0.037}^{+0.026}$	$0.873_{-0.019}^{+0.028}$
DINOv3 local + CLS	$0.930_{-0.011}^{+0.013}$	$0.890_{-0.021}^{+0.013}$	$0.853_{-0.029}^{+0.045}$	$0.868_{-0.021}^{+0.022}$
DINOv3 local + CLS + register	$0.924_{-0.010}^{+0.020}$	$0.897_{-0.030}^{+0.008}$	$0.841_{-0.031}^{+0.046}$	$0.860_{-0.023}^{+0.033}$
DINOv3 local + mean( CLS + register)	$0.934_{-0.012}^{+0.010}$	$0.892_{-0.020}^{+0.017}$	$0.866_{-0.036}^{+0.036}$	$0.872_{-0.014}^{+0.020}$
Encoder-decoder	$0.804_{-0.027}^{+0.014}$	$0.771_{-0.025}^{+0.037}$	$0.655_{-0.016}^{+0.033}$	$0.686_{-0.033}^{+0.022}$
U-Net	$0.851_{-0.020}^{+0.031}$	$0.839_{-0.080}^{+0.030}$	$0.699_{-0.034}^{+0.022}$	$0.728_{-0.048}^{+0.015}$

Table 5: Overall metrics for cloud segmentation(median with 16th-83rd percentiles)

Baseline	Accuracy	Precision	Recall	F1-score
Trivial	0.888	0.859	0.858	0.858
Optical Flow	0.888	0.858	0.860	0.859
ConvLSTM	0.890	0.871	0.849	0.859
VideoGPT-1	0.871	0.841	0.823	0.831
VideoGPT-2	0.872	0.840	0.825	0.831
VideoGPT-7	0.870	0.841	0.818	0.827

Table 6: Overall metrics for weather nowcast. VideoGPT-

n

refers to using VideoGPT to predict the next frame according to previous

n

frames; Accuracy, precision, recall, and F1-score are measure by the macro average among sky, cloud and contamination class.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Failure cases of the all-sky camera. Top row (left to right): (1) Covered by dust or sand; (2) Covered by dew or ice; (3) Scattered light caused by mud coverage; (4) Covered by snow. Bottom row (left to right): (5) Obstruction by an external object; (6) Strong nearby light source; (7) Strong distant light source; (8) Camera malfunction.

Reason	# occur	# frame	Down time [days]
cover	181	30111	169.79
stronglight	388	5275	23.54
cameradown	56	2167	20.75
object	40	735	4.66
Total	665	38288	218.75

Appendix

Table 7: Statistics of failure case of all-sky camera. The events that occurs within 12 hours are merged together as one event.

Name	Formula
Gnomonic projection	$r=f\tan\theta$
Stereographic projection	$r=2f\tan\frac{\theta}{2}$
Equidistant projection	$r=2f\theta$
Equisolid angle projection	$r=2f\sin\frac{\theta}{2}$
Orthographic projection	$r=f\sin\theta$

Appendix

Table 8: Optional projection types of full-sky camera, which is provided by [ 19 ] .

Figure 5: Manual annotation for cloud segmentation. Blue represents cloud regions; orange represents sky regions; Pink represents contamination regions; Columns represents images taken around given time in UTC+8; Rows represents images taken in different moon phase condition. Nearby frames are used by human to determine whether some regions are scatter light or cloud.

Panel of Figure 6 — Figure 6: Altitude–Azimuth fitting results for different time slots. Red boxes means that the WCS of the corresponding HEALPix cell is not resolved by Astrometry.net for any image in the ensemble, and the corresponding altitude and azimuth is obtained by fitting results.

Panel of Figure 7 — Figure 7: Astrometric calibration fitting residual for different time slots in radial and azimuthal direction.

Stage	Layer (in $\to$ out)	Operation	k/s/p	Channel out	Size (H $\times$ W)
Enc-1	3 $\to$ 64	Conv2d + ReLU + BN	3/1/1	64	$512\times 512$
Enc-1	64 $\to$ 64	Conv2d + ReLU + BN	3/1/1	64	$512\times 512$
Enc-1 $\downarrow$	–	MaxPool2d	2/2/0	64	$256\times 256$
Enc-2	64 $\to$ 128	Conv2d + ReLU + BN	3/1/1	128	$256\times 256$
Enc-2	128 $\to$ 128	Conv2d + ReLU + BN	3/1/1	128	$256\times 256$
Enc-2 $\downarrow$	–	MaxPool2d	2/2/0	128	$128\times 128$

Appendix

Table 9: Layer-wise specification of Encoder-Decoder (CloudSegNet) for a

3\times 512\times 512

input. “k/s/p” denotes kernel/stride/padding.

Stage	Layer (in $\to$ out)	Operation	k/s/p	Ch. out	Size (H $\times$ W)
Enc-0	3 $\to$ 64	Conv2d + ReLU + BN	3/1/1	64	$512\times 512$
Enc-0	64 $\to$ 64	Conv2d + ReLU + BN	3/1/1	64	$512\times 512$
Down-1 $\downarrow$	–	MaxPool2d	2/2/0	64	$256\times 256$
Down-1	64 $\to$ 128	Conv2d + ReLU + BN	3/1/1	128	$256\times 256$
Down-1	128 $\to$ 128	Conv2d + ReLU + BN	3/1/1	128	$256\times 256$
Down-2 $\downarrow$	–	MaxPool2d	2/2/0	128	$128\times 128$

Appendix

Table 10: Layer-wise specification of the U-Net used in our experiments (bilinear upsampling variant) for a

3\times 512\times 512

input.

Stage	Layer (in $\to$ out)	Operation	k/s/p	Ch. out	Size (H $\times$ W)
LSTM-1	3 $\to$ 64	ConvLSTM2D + Dropout	3/1/1	64	$H\times W$
LSTM-2	64 $\to$ 64	ConvLSTM2D + Dropout	3/1/1	64	$H\times W$
Head	64 $\to$ 3	Conv2d (readout)	1/1/0	3	$H\times W$

Appendix

Table 11: Layer-wise specification of the ConvLSTM next-frame baseline. Input is a

[B,\,C{=}3,\,T_{\text{in}}{=}2,\,H,\,W]

clip; output is a single frame

[B,3,H,W]

. “k/s/p” denotes kernel/stride/padding.

VQ-VAE Encoder
Enc-1	3 $\to$ 64	Conv2d + BN + ReLU + ResidualBlock	4/2/1	64	$32\times 32$
Enc-2	64 $\to$ 128	Conv2d + BN + ReLU + ResidualBlock	4/2/1	128	$16\times 16$
Enc-3	128 $\to$ 256	Conv2d + BN + ReLU + ResidualBlock	4/2/1	256	$8\times 8$
Bottleneck	256 $\to$ 256	Conv2d + BN + ReLU	3/1/1	256	$8\times 8$
Vector Quantiser
VQ	256 $\to K$	VectorQuantizer ( $K{=}512$ , $D{=}256$ )	–	$K$	$T\times 8\times 8$

Appendix

Table 12: Architecture of the VideoGPT nowcaster (VQ-VAE + GPT). For a

[B,3,T,64,64]

input, the encoder downsamples to

[B,256,T,8,8]

, which is vector-quantised (codebook size

K

). Tokens are modeled autoregressively by a Transformer and decoded back to frames.

Class	Baseline	Precision	Recall	F1-score
Sky	Trivial	0.914	0.913	0.914
	Optica lFlow	0.913	0.913	0.913
	ConvLSTM	0.922	0.905	0.913
	VideoGPT-1	0.920	0.882	0.900
	VideoGPT-2	0.915	0.887	0.901
	VideoGPT-7	0.919	0.883	0.900

Appendix

Table 13: Per-class metrics for nowcasting

Class	Baseline	Precision	Recall	F1-score
Sky	DINOv3 local	$0.920_{-0.022}^{+0.028}$	$0.945_{-0.009}^{+0.018}$	$0.936_{-0.015}^{+0.006}$
	DINOv3 local + CLS	$0.917_{-0.025}^{+0.037}$	$0.941_{-0.014}^{+0.019}$	$0.932_{-0.013}^{+0.012}$
	DINOv3 local + CLS + register	$0.913_{-0.031}^{+0.038}$	$0.942_{-0.025}^{+0.018}$	$0.926_{-0.012}^{+0.015}$
	DINOv3 local + mean(CLS + register)	$0.921_{-0.019}^{+0.031}$	$0.944_{-0.019}^{+0.018}$	$0.936_{-0.016}^{+0.009}$
	Encoder-Decoder	$0.764_{-0.027}^{+0.061}$	$0.868_{-0.034}^{+0.056}$	$0.816_{-0.009}^{+0.021}$
	U-Net	$0.869_{-0.039}^{+0.023}$	$0.912_{-0.047}^{+0.025}$	$0.882_{-0.016}^{+0.017}$

Appendix

Table 14: Per-class metrics for cloud segmentation(median with 16th-83rd percentiles)

Image Segmentation

Image segmentation assigns pixels or voxels to objects and regions, supporting measurement, scene understanding, and decisions across image domains.

See all 399 papers

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD
Sep 22, 2026Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw ZegarlinskiActive Learning Transfer Learning
CEREMADE, Universit´e Paris Dauphine-PSL, Paris, France. · School of Computing and Information Science, Anglia Ruskin University, Cambridge, United Kingdom. · Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland.
BELDE: Building a Large-scale Earth-observation Land-cover Dataset for Europe
Jun 18, 2026Ümit Mert Çağlar, Alptekin TemizelRemote Sensing Image Segmentation Distribution Shift
Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
Sky2Ground: A Benchmark for Site Modeling under Varying Altitude
Mar 14, 2026Zengyan Wang, Sirshapan Mitra, Rajat Modi +2Cross-View Image Matching Cross-View Geo-Localization
Center for Research In Computer Vision, University of Central Florida

Sep 22, 2026J/K move · Enter open · S save

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

Esther Bou Dagher, Viktoriya Bu-Dager, Boguslaw Zegarlinski

CEREMADE, Universit´e Paris Dauphine-PSL, Paris, France. · School of Computing and Information Science, Anglia Ruskin University, Cambridge, United Kingdom. · Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland.

Active Learning Transfer Learning Pseudo-Labeling Data-Efficient Learning

Open paper AskPDF arXiv

Accurate ground-based cloud classification is important for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate observation systems. However, reliable sky-image annotation is time-consuming, especially when cloud types are visually similar or mixed. We study the label efficiency of deep learning for ground-based cloud classification using the Ground-based Cloud Dataset (GCD). Rather than proposing a new architecture, we benchmark three practical strategies under limited annotation budgets: supervised transfer learning, uncertainty-based active learning, and high-confidence pseudo-labeling. An ImageNet-pretrained ResNet50 is used as a common frozen backbone, with experiments repeated over five random seeds for label budgets from

1\%

100\%

of the training labels. Supervised transfer learning is already highly label-efficient: test accuracy increases from

0.635 \pm 0.018

with

1\%

labels to

0.730 \pm 0.002

with

40\%

labels, approaching the full-label result of

0.735 \pm 0.003

. Active learning and pseudo-labeling are competitive with supervised sampling and provide small improvements for some metrics and budgets, but neither gives a large or consistent aggregate gain. Diagnostic analyses show that accepted pseudo-labels are reliable, with accuracy from

0.946

0.977

, but biased toward easier high-confidence sky-type groups. In contrast, uncertainty sampling preferentially queries visually challenging groups, including Mixed and the confusable Stratocumulus and Cumulonimbus groups, but these targeted acquisitions yield only modest gains. Overall, transfer learning substantially reduces annotation requirements for GCD, while simple active and semi-supervised strategies provide limited additional benefit over a strong supervised baseline.

Abstract

Image Segmentation

Explore similar work

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

BELDE: Building a Large-scale Earth-observation Land-cover Dataset for Europe

Sky2Ground: A Benchmark for Site Modeling under Varying Altitude