Satellite Pretraining Is Not One Thing: A Matched-Pair Study of Generic and Earth-Observation Foundation Models for Scene Classification
Organizations: SatSure Analytics India Pvt. Ltd., Bengaluru, India
Abstract
Geospatial foundation models are pretrained on satellite and aerial imagery, but whether such an encoder beats a strong generic one for few-label land-use/land-cover mapping remains unclear, as prior comparisons rarely isolate the pretraining data. We study five matched pairs of Earth-observation (EO) and generic encoders sharing objective and architecture (DINO/SatDINO, DINOv3/DINOv3-SAT, MAE/SatMAE, MAE/Scale-MAE, CLIP/RemoteCLIP) within a benchmark of 14 frozen encoders on five RGB scene-classification datasets (0.06-30 m GSD). With native preprocessing, we evaluate k-NN, linear and attentive probes and image retrieval over six low-label splits, using paired per-image significance tests. EO pretraining is not uniformly helpful: on most datasets and metrics, it significantly improves three pairs (SatDINO, Scale-MAE, RemoteCLIP, the latter evaluated only on unseen datasets) and significantly hurts two (linear probe: DINOv3-SAT by up to 22.5 points, SatMAE by up to 8.8), with opposite outcomes even for two MAEs pretrained on the same fMoW-RGB imagery. These effects are robust to patch-token pooling and attentive probing, concentrate on small man-made objects, and are largest with 1-5% labels. Across encoders, the silhouette score of frozen embeddings predicts k-NN accuracy and retrieval mAP almost perfectly (Spearman on four of five datasets). Pretraining on satellite imagery is thus not a reliable selection criterion; candidate checkpoints must be validated on target data.
Figures & tables
| Encoder | Params | Objective | Pretraining data | Feature | Normalisation |
| Supervised | |||||
| ResNet-18 | 11M | supervised | ImageNet-1k (labels) | GAP | ImageNet |
| Generic (natural-image) pretraining | |||||
| DINO ViT-B/16 | 86M | self-distillation | ImageNet-1k | CLS | ImageNet |
| DINOv3 ViT-B/16 | 86M | self-distillation † | LVD-1689M (web) | CLS | ImageNet |
| DINOv3 ViT-L/16 | 303M | self-distillation † | LVD-1689M (web) | CLS | ImageNet |
| PatternNet | RSICD | EuroSAT | UC Merced | RESISC45 | |||||||||||
| Encoder | -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP |
| ResNet-18 | 92.9 | 96.6 | 62.7 | 70.0 | 76.7 | 35.1 | 86.9 | 90.9 | 56.8 | 73.5 | 85.0 | 57.6 | 71.6 | 77.7 | 29.8 |
| DINO ViT-B/16 | 95.3 | 98.8 | 72.4 | 76.5 | 87.2 | 43.1 | 90.7 | 96.2 | 62.8 | 78.4 | 92.7 | 64.5 | 78.7 | 89.0 | 38.1 |
| DINOv3 ViT-B/16 | 96.3 | 98.6 | 78.1 | 82.8 | 89.7 | 58.5 | 89.7 | 95.4 | 60.1 | 82.3 | 94.3 | 77.3 | 83.1 | 89.9 | 47.9 |
| DINOv3 ViT-L/16 | 96.8 | 98.6 | 84.4 | 84.5 | 91.1 | 64.3 | 92.0 | 95.9 | 63.7 | 84.9 | 95.2 | 82.9 | 87.0 | 92.1 | 58.6 |
| I-JEPA ViT-H/14 | 84.8 | 97.8 | 47.0 | 60.2 | 82.4 | 27.0 | 86.4 | 95.2 | 54.4 | 53.5 | 85.0 | 40.4 | 60.1 | 83.5 | 21.7 |
| EO model | PatternNet | RSICD | EuroSAT | UC Merced | RESISC45 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP | -NN | LP | mAP | |
| SatDINO | +1.0 ▲ | +0.3 ▲ | +4.8 ▲ | +6.8 ▲ | +1.9 ▲ | +11.7 ▲ | +1.5 ▲ | +0.1 | +4.4 ▲ | +2.1 | +0.8 | +3.8 ▲ | +5.2 ▲ | +1.7 ▲ | +6.2 ▲ |
| DINOv3-SAT | 10.2 ▽ | 1.9 ▽ | 31.3 ▽ | 28.7 ▽ | 13.3 ▽ | 37.9 ▽ | 5.3 ▽ | 1.6 ▽ | 11.2 ▽ | 41.8 ▽ | 22.5 ▽ | 47.6 ▽ | 27.4 ▽ | 12.4 ▽ | 33.7 ▽ |
| SatMAE | 4.7 ▽ | 2.0 ▽ | 4.4 ▽ | +9.3 ▲ | 2.6 ▽ | +5.7 ▲ | +0.2 | 1.2 ▽ | +6.4 ▲ | 5.9 ▽ | 8.8 ▽ | 3.5 ▽ | +0.0 | 6.5 ▽ | 0.8 ▽ |
| Scale-MAE | +3.5 ▲ | +0.1 | +8.3 ▲ | +20.7 ▲ | +4.7 ▲ | +16.7 ▲ | +3.7 ▲ | +0.8 ▲ | +6.8 ▲ | +2.6 | 2.2 | +1.7 ▲ | +14.0 ▲ | +1.9 ▲ | +8.7 ▲ |
| RemoteCLIP | +1.6 ▲ | +0.4 ▲ | +10.2 ▲ | +7.1 ▲ | +2.5 ▲ | +14.2 ▲ | +6.6 ▲ | +1.5 ▲ | +11.1 ▲ | +11.9 ▲ | +4.2 ▲ | +17.4 ▲ | +4.9 ▲ | +1.7 ▲ | +16.1 ▲ |
| EO model | Largest per-class gains of the EO model | Largest per-class losses of the EO model |
|---|---|---|
| SatDINO | railway (+7.0), tennis court (+5.7), basketball court (+5.1) | river ( 1.8), roundabout ( 1.3), cloud ( 1.2) |
| DINOv3-SAT | mountain (+5.9), chaparral (+0.5), desert ( 0.4) | basketball court ( 34.7), palace ( 34.4), tennis court ( 33.0) |
| SatMAE | freeway (+3.7), intersection (+0.1), sea ice (+0.0) | palace ( 19.9), airport ( 19.5), storage tank ( 17.4) |
| Scale-MAE | baseball diamond (+8.5), ground track field (+7.7), freeway (+7.3) | rectangular farmland ( 3.4), lake ( 1.9), island ( 1.8) |
| RemoteCLIP | tennis court (+13.0), basketball court (+7.7), baseball diamond (+6.6) | wetland ( 2.7), commercial area ( 2.6), palace ( 1.6) |
| -NN | Trained probe | ||||
|---|---|---|---|---|---|
| EO model | native | patch | LP | LP patch | EP |
| SatDINO | +3.3 (5/5) | +2.6 (5/5) | +1.0 (5/5) | +1.1 (5/5) | +1.1 (5/5) |
| DINOv3-SAT | 22.7 (0/5) | 11.7 (0/5) | 10.3 (0/5) | 5.0 (0/5) | 4.6 (0/5) |
| SatMAE | 0.2 (3/5) | +12.0 (5/5) | 4.2 (0/5) | 2.6 (0/5) | 2.9 (0/5) |
| Scale-MAE | +8.9 (5/5) | +11.4 (5/5) | +1.0 (4/5) | +0.8 (4/5) | +0.8 (4/5) |
| RemoteCLIP | +4.4 (3/3) | +2.8 (3/3) | +1.2 (3/3) | +0.8 (3/3) | +0.9 (3/3) |
| UC Merced | RSICD | ||||||
|---|---|---|---|---|---|---|---|
| DINOv3-L | Feature | -NN | LP | mAP | -NN | LP | mAP |
| web | CLS | 84.9 | 95.2 | 82.9 | 84.5 | 91.1 | 64.3 |
| web | patch mean | 85.9 | 95.2 | 84.4 | 86.4 | 91.4 | 68.3 |
| web | CLS patch | 85.6 | 95.6 | 83.9 | 85.4 | 91.8 | 66.0 |
| SAT | CLS | 43.2 | 72.7 | 35.3 | 55.8 | 77.8 | 26.3 |
| SAT | patch mean | 60.5 | 84.9 | 51.7 | 76.1 | 85.6 | 47.5 |
Geospatial Foundation Models
Geospatial foundation models turn satellite, map, and Earth-system data into reusable representations for spatial analysis and prediction.
See all 134 papers