cs.CVSep 26, 2026
SaveGeoCR: Learning a Generalist Cloud Removal Prior from Heterogeneous Observations
Organizations: Korea Advanced Institute of Science and Technology (KAIST)
Abstract
Cloud removal methods are typically specialized to individual datasets and input configurations, limiting reuse across sensors, spectral bands, and observation settings. We introduce GeoCR, a generalist model that unifies RGB-only-based CR and multispectral-based CR from single- or multi-temporal cloudy observations, with optional SAR guidance, within a single network. To accommodate different spectral and sensing domains, compact input and output stems extend a pretrained RGB autoencoder while keeping its encoder and decoder trunks frozen. This shared latent interface enables a single flow transformer to jointly model clean RGB and non-RGB latents, conditioned on separate cloudy-observation streams and optional SAR tokens. Through joint pretraining on the training splits of ten datasets comprising 883,331 cloud-free target images, GeoCR learns a shared cloud removal prior across these heterogeneous configurations. The same pretrained checkpoint supports direct inference without dataset-specific fine-tuning and efficient adaptation through low-rank adaptation (LoRA). We evaluate GeoCR against general image restoration and cloud removal methods on test splits of the contributing datasets under full-band and RGB-only settings. GeoCR achieves the best FID and DISTS on full-band SEN12MS-CR and Sen2_MTC_New and RGB-only CUHK-CR2, outperforming existing models and demonstrating the effectiveness of a reusable generative model across diverse settings.
Figures & tables
Figure 2: Cross-dataset comparison of cloud removal (CR) methods. We report FID and DISTS on five cloud removal benchmarks, independently normalized for each dataset–metric pair as .
Figure 3: GeoCR framework. A shared latent interface enables joint cloud removal across heterogeneous spectral, temporal, and SAR configurations.
| Source | Imagery source | Spectral bands | Delivered tile | GSD (m) | Cloudy cond. | SAR cond. | Clean targets |
| AllClear ( Zhou et al., 2024 ) | Sentinel-2 + Sentinel-1 | 13 (L1C, TOA) | 10 | 1–3 | ✓ | 662,022 | |
| SEN12MS-CR ( Ebel et al., 2020 ) | Sentinel-2 + Sentinel-1 | 13 (L1C, TOA) | 10 | 1 | ✓ | 107,143 | |
| WHUS2-CRv ( Li et al., 2022 ) | Sentinel-2 | 13 (BOA; B10: L1C, TOA) | 10 | 1 | ✗ | 18,816 | |
| Sen2_MTC_Old ( Sarukkai et al., 2020 ) | Sentinel-2 | 3 (RGB; +NIR for cloudy inputs) | 10 | 3 | ✗ | 88,874 | |
| Sen2_MTC_New ( Huang and Wu, 2022 ) | Sentinel-2 | 4 (RGB+B8, BOA) | 10 | 3 | ✗ | 2,380 | |
| T-CLOUD ( Ding et al., 2022 ) | Landsat-8 | 3 (RGB, 8-bit) | 30 | 1 | ✗ | 2,234 |
Table 1: Overview of the GeoCR pretraining corpus. Counts refer to cloud-free target images in the training splits. Spectral bands describe the targets unless otherwise noted; cloudy conditions indicate the number of optical input frames. Tile sizes are in pixels. L1C: Level-1C; TOA/BOA: top-/bottom-of-atmosphere reflectance; RGB: red, green, and blue; NIR: near-infrared; SAR: synthetic aperture radar; GSD: ground sampling distance. B8 and B10 denote Sentinel-2 band identifiers.
| Method | Venue | FID | DISTS | KID | DINO | LPIPS | SSIM | PSNR |
| General image translation/restoration methods | ||||||||
| pix2pix ( Isola et al., 2017 ) | CVPR’17 | 194.8 | 0.241 | 0.1201 | 0.538 | 0.319 | 0.452 | 19.46 |
| pix2pixHD ( Wang et al., 2018 ) | CVPR’18 | 257.1 | 0.262 | 0.2130 | 0.379 | 0.304 | 0.401 | 19.34 |
| BBDM ( Li et al., 2023 ) | CVPR’23 | 514.6 | 0.542 | 0.5827 | 0.131 | 0.737 | 0.326 | 17.68 |
| HI-Diff ( Chen et al., 2023 ) | NeurIPS’23 | 165.7 | 0.192 | 0.0769 | 0.664 | 0.253 | 0.638 | 23.54 |
| Cloud removal methods | ||||||||
Table 2: Quantitative comparison on CUHK-CR2. Its RGB evaluation setting supports both RGB-only image translation/restoration methods and specialized cloud removal methods, enabling comparison across all ten baselines. Bold and underlined values indicate the best and second-best results, respectively. Additional results are provided in the Appendix .
Figure 4: Qualitative comparison with general restoration methods under RGB-only settings.
Figure 5: Cloud removal across input cloud coverage on SEN12MS-CR. Left: DISTS and LPIPS grouped by input cloud coverage estimated with s2cloudless ; lower is better. Input Cloudy denotes the unrestored observation. Right: qualitative comparisons across the same coverage groups.
| Recon. PSNR | RGB (w/o training) | Non-RGB (stem training) | SAR (stem training) | ||||||
| All | Clear | Cloudy | All | Clear | Cloudy | All | VV | VH | |
| Before | 45.78 | 46.77 | 45.11 | 18.84 | 20.60 | 18.20 | 18.40 | 18.57 | 18.24 |
| After | 45.78 | 46.77 | 45.11 | 40.08 | 41.41 | 38.34 | 29.14 | 28.54 | 30.21 |
| Gain | +0.00 | +0.00 | +0.00 | +21.24 | +20.81 | +20.14 | +10.74 | +9.97 | +11.98 |
Table 5: Effect of Stage 1 stem training. Reconstruction PSNR (dB) on AllClear validation samples before and after stem training, with the pretrained encoder and decoder frozen. VV and VH denote SAR polarizations.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Section | Contents |
| Section A | Stem adaptation |
| Section B | Implementation details |
| Section C | Additional quantitative results |
| Section D | Additional qualitative results |
| Section E | Failure cases and limitations |
| Section F | Datasets and preprocessing |
Table 8: Overview of the Appendix .
Figure 8: Reconstruction through the shared autoencoder. The pretrained RGB route and initialized NIR route remain fixed during Stage 1. Ten-band optical and SAR routes use learned stems with frozen encoder–decoder trunks. Ten-band visualizations use B12/B8/B5 false color, and SAR uses a VV/VH/VV composite.
| Route | Stem parameters | Trained | Reconstruction PSNR |
| 10-band TOA | 23,178 | Yes | AllClear 38.23; SEN12MS-CR 37.69 |
| 10-band BOA | 23,178 | Yes | WHUS2-CRv 37.89 |
| Single-band NIR | 2,433 | No | Sen2_MTC_New 42.06; CUHK-CR 40.41; Sen2_MTC_Old 49.51 |
| SAR VV/VH | 4,738 | Yes | AllClear 28.51; SEN12MS-CR 31.19 |
Table 9: Reconstruction by configuration. PSNR (dB) is computed from pooled reconstruction error on 64 training images per source, without clipping. These diagnostics differ from the held-out before/after comparison in the main paper and do not measure cloud removal.
| Method | FID | DISTS | KID | DINO | LPIPS | SSIM | PSNR |
| (a) T-CLOUD: RGB display imagery | |||||||
| pix2pix ( Isola et al., 2017 ) | 107.6 | 0.254 | 0.0509 | 0.450 | 0.330 | 0.635 | 20.13 |
| pix2pixHD ( Wang et al., 2018 ) | 60.8 | 0.169 | 0.0173 | 0.616 | 0.177 | 0.741 | 24.75 |
| BBDM ( Li et al., 2023 ) | 186.6 | 0.422 | 0.1179 | 0.259 | 0.546 | 0.541 | 22.69 |
| HI-Diff ( Chen et al., 2023 ) | 40.2 | 0.124 | 0.0070 | 0.726 | 0.114 | 0.874 | 30.54 |
| UnCRtainTS ( Ebel et al., 2023 ) | 63.3 | 0.180 | 0.0188 | 0.640 | 0.178 | 0.809 | 26.28 |
Table 10: Additional quantitative comparisons. Feature metrics use RGB views; PSNR/SSIM follow Table 12 .
Figure 9: Representative cloud removal results. Rows show native and RGB-only SEN12MS-CR, Sen2_MTC_New, WHUS2-CRv, CUHK-CR1, and T-CLOUD. Baselines are selected by dataset-level LPIPS; numbers denote per-image LPIPS. The first two rows show the same scene.
Figure 10: Challenging cloud removal cases. Examples cover severe occlusion, the same scene with multispectral and SAR inputs, occlusion across all temporal observations, temporal surface change, fine urban structure, and photometric mismatch. Cases are selected using occlusion, change, texture, or error criteria. Numbers indicate per-image LPIPS. Rows (a) and (b) change both optical bands and SAR availability.
| Source | Optical representation | SAR | Mix (%) | Training targets | |
| AllClear ( Zhou et al., 2024 ) | 13 bands, L1C TOA | VV/VH | 1–3 | 53.14 | 662,022 |
| SEN12MS-CR ( Ebel et al., 2020 ) | 13 bands, L1C TOA | VV/VH | 1 | 32.93 | 107,143 |
| WHUS2-CRv ( Li et al., 2022 ) | 13 bands, BOA; B10 TOA | – | 1 | 8.81 | 18,816 |
| Sen2_MTC_Old ( Sarukkai et al., 2020 ) | RGB; NIR in cloudy inputs only | – | 3 | 2.00 | 88,874 |
| Sen2_MTC_New ( Huang and Wu, 2022 ) | RGB+B8, BOA | – | 3 | 1.10 | 2,380 |
| T-CLOUD ( Ding et al., 2022 ) | RGB, 8-bit | – | 1 | 1.10 | 2,234 |
Table 11: Dataset representations and pretraining mixture. Bands describe optical targets; is the number of cloudy observations. Mix denotes the source sampling probability.
| Setting | PSNR/SSIM bands | |
| SEN12MS-CR | 13 bands | 7,899 |
| Sen2_MTC_New | RGB+B8 | 687 |
| WHUS2-CRv | 13 bands | 3,746 |
| T-CLOUD | RGB | 588 |
| CUHK-CR1 | RGB+NIR | 98 |
| CUHK-CR2 | RGB | 111 |
Table 12: Evaluation settings. denotes the size of each evaluated test split. CUHK-CR1 counts reflect the duplicate exclusions described in Section F .
Figure 11: Qualitative comparison on SEN12MS-CR ( Ebel et al., 2020 ) . Four examples from the native multispectral setting, shown as RGB views with paired clear references. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 12: Qualitative comparison on Sen2_MTC_New ( Huang and Wu, 2022 ) . Four examples from the native multi-temporal setting, shown as RGB views with paired clear references. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 13: Qualitative comparison on WHUS2-CRv ( Li et al., 2022 ) . Four examples from the native multispectral setting, shown as RGB views with paired clear references. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 14: Qualitative comparison on T-CLOUD ( Ding et al., 2022 ) . The two panels show the same four scenes with different groups of competing methods. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 15: Qualitative comparison on CUHK-CR1 ( Sui et al., 2024 ) . Four examples from the RGB+NIR setting, shown as RGB views with paired clear references. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 16: Qualitative comparison on CUHK-CR2 ( Sui et al., 2024 ) . The two panels show the same four scenes with different groups of competing methods under the RGB evaluation setting. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 17: Qualitative comparison on SEN12MS-CR ( Ebel et al., 2020 ) (RGB-only). Four examples from the RGB-only setting, with one cloudy RGB input and no SAR guidance. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 18: Qualitative comparison on Sen2_MTC_New ( Huang and Wu, 2022 ) (RGB-only). Four examples from the RGB-only setting, with one cloudy RGB input and no SAR guidance. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.
Figure 19: Qualitative comparison on WHUS2-CRv ( Li et al., 2022 ) (RGB-only). Four examples from the RGB-only setting, with one cloudy RGB input and no SAR guidance. GeoCR is evaluated without fine-tuning (w/o FT) and with LoRA adaptation.