Accuracy and efficiency of compact foundation models for surface-water mapping during floods
Organizations: Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, 95616, USA
Abstract
Pretrained Earth observation models may reduce the effort needed to map surface water, but their value depends on the alternative against which they are compared. We evaluated Prithvi-EO-2.0-Tiny using the 446 hand-labeled Sentinel-2 chips from 11 flood events in Sen1Floods11. A fine-tuned model, a frozen-encoder model, and an identical randomly initialized architecture were compared with a similarly sized U-Net, a larger Prithvi model, and spectral-index thresholds. Experiments varied the number of training chips, excluded events from model fitting, and measured computational cost and prediction reliability. With 252 training chips, U-Net achieved a water intersection-over-union (IoU) of 0.856 on the official test set, compared with 0.833 for fine-tuned Prithvi-Tiny. Pretraining improved the tiny transformer over random initialization by a reported 0.046 IoU with 25 training chips and 0.023 with 252 chips, but did not reverse the U-Net ranking. Across 11 event-exclusion evaluations, mean IoU was 0.768 for fine-tuned Prithvi-Tiny and 0.777 for U-Net; the paired interval included zero. Fine-tuned Prithvi-Tiny processed a 512x512-pixel chip in 16.1 ms on a GPU and 258 ms on a laptop CPU, compared with 30.6 and 665 ms for U-Net. Errors were concentrated near water boundaries, and high predictive entropy identified many, but not all, erroneous pixels. These results support a specific role for compact pretrained models: lower measured computational demand with competitive accuracy, rather than consistent superiority. Event-dependent performance, shared hyperparameter selection, and the distinction between observed water and newly inundated land delimit the conclusions.
Figures & tables
| Event | S2 date | Chips | A: train / valid / test | B: train / valid | Water (%) | Unlabeled (%) | Permanent water (%) |
| Ghana | 2018-09-19 | 53 | 31 / 11 / 11 | 221 / 78 | 6.3 | 33.8 | 1.4 |
| India | 2016-08-12 | 68 | 40 / 14 / 14 | 212 / 75 | 12.4 | 16 | 2.7 |
| Cambodia | 2018-08-04 | 30 | 18 / 6 / 6 | 234 / 83 | 24.1 | 9.3 | 7.6 |
| Nigeria | 2018-09-20 | 18 | 10 / 4 / 4 | 242 / 85 | 23.5 | 14.3 | 1.9 |
| Pakistan | 2017-06-28 | 28 | 16 / 6 / 6 | 236 / 83 | 7.5 | 26.7 | 2.5 |
| Paraguay | 2018-10-31 | 67 | 39 / 14 / 14 | 213 / 75 | 8.4 | 5 | 2.9 |
| Model | Encoder initialization | Updated components | Total (M) | Trainable (M) | Learning rate |
| U-Net | Random | All | 7.76 | 7.76 | |
| Tiny frozen | Pretrained | Neck + decoder + head | 7.79 | 2.15 | |
| Tiny fine-tuned | Pretrained | All | 7.79 | 7.79 | |
| Tiny random | Random | All | 7.79 | 7.79 | |
| Prithvi-100M | Pretrained | All | 91.97 | 91.97 |
| Method | Water IoU mean SD | IoU 95% CI | F1 | Precision | Recall | ECE |
| U-Net | 0.856 0.001 | 0.796–0.897 | 0.922 | 0.936 | 0.909 | 0.014 |
| Tiny frozen | 0.832 0.004 | 0.772–0.877 | 0.908 | 0.925 | 0.892 | 0.002 |
| Tiny fine-tuned | 0.833 0.001 | 0.772–0.879 | 0.909 | 0.932 | 0.887 | 0.004 |
| Tiny random | 0.809 0.003 | 0.748–0.859 | 0.895 | 0.926 | 0.865 | 0.003 |
| Prithvi-100M | 0.841 0.002 | 0.783–0.886 | 0.914 | 0.936 | 0.892 | 0.005 |
| MNDWI fitted | 0.787 0.000 | 0.711–0.846 | 0.881 | 0.869 | 0.893 | — |
| Method | Water IoU mean SD | IoU 95% CI | F1 | ECE |
| U-Net | 0.795 0.014 | 0.685–0.867 | 0.886 | 0.015 |
| Tiny frozen | 0.689 0.028 | 0.470–0.796 | 0.815 | 0.036 |
| Tiny fine-tuned | 0.752 0.031 | 0.552–0.844 | 0.859 | 0.025 |
| Tiny random | 0.706 0.022 | 0.427–0.826 | 0.828 | 0.021 |
| Prithvi-100M | 0.794 0.007 | 0.651–0.863 | 0.885 | 0.016 |
| MNDWI fitted | 0.704 0.000 | 0.529–0.806 | 0.827 | — |
| A. Seen-event test | ||||
| Method | 25 chips ( 10%) | 63 chips (25%) | 126 chips (50%) | 252 chips (100%) |
| U-Net | 0.831 0.014 | 0.844 0.006 | 0.857 0.005 | 0.856 0.001 |
| Tiny frozen | 0.784 0.052 | 0.822 0.003 | 0.822 0.010 | 0.832 0.004 |
| Tiny fine-tuned | 0.814 0.013 | 0.834 0.003 | 0.834 0.005 | 0.833 0.001 |
| Tiny random | 0.768 0.011 | 0.811 0.008 | 0.813 0.007 | 0.809 0.003 |
| Prithvi-100M | 0.820 0.005 | — | — | 0.841 0.002 |
| Method | Parameters total / trainable (M) | GFLOPs / 256-px tile | Training (min) | Training GPU memory (GB) | GPU (ms/chip) | CPU (ms/chip) |
| U-Net | 7.8 / 7.8 | 24.3 | 5.6 | 2.5 | 30.6 | 665 |
| Tiny frozen | 7.8 / 2.2 | 9.6 | 2.7 | 1.8 | 15.9 | 258 |
| Tiny fine-tuned | 7.8 / 7.8 | 9.6 | 3.3 | 2.0 | 16.1 | 258 |
| Tiny random | 7.8 / 7.8 | 9.6 | 3.3 | 2.2 | 16.8 | 258 |
| Prithvi-100M | 92.0 / 92.0 | 53.0 | 8.7 | 4.7 | 50.4 | 1386 |
| MNDWI fitted | 0 | — | — | — | — | 4.9 |
| Method | Mean event IoU (95% CI) | Worst event (IoU) | Events above U-Net | Unseen - seen IoU (95% CI) | Mean ECE |
| U-Net | 0.777 (0.705–0.843) | Paraguay (0.522) | — | -0.009 (-0.020, -0.001) | 0.030 |
| Tiny frozen | 0.765 (0.705–0.823) | Paraguay (0.564) | 3/11 | -0.011 (-0.018, -0.006) | 0.010 |
| Tiny fine-tuned | 0.768 (0.703–0.828) | Paraguay (0.573) | 3/11 | -0.004 (-0.014, +0.006) | 0.009 |
| Tiny random | 0.725 (0.657–0.792) | Paraguay (0.519) | 1/11 | -0.008 (-0.028, +0.010) | 0.013 |
| Prithvi-100M | not run | not run | not run | not run | not run |
| MNDWI fitted | 0.730 (0.671–0.792) | Pakistan (0.602) | 2/11 | -0.002 (-0.008, +0.003) | — |
Flood Inundation Mapping
Flood inundation mapping uses satellite imagery, terrain data, and models to locate flooded areas and estimate water depth for response and planning.
See all 27 papers