ShadowTransfer: A Geographic Transfer Benchmark for Overhead Shadow Detection
Abstract
Overhead aerial imagery feeds dense-prediction pipelines that are routinely trained in one geographic region and deployed in another, yet no benchmark measures how shadow detection models transfer across regions. Existing overhead shadow datasets either lack a geographic transfer protocol or rely on automatically generated labels. We introduce ShadowTransfer, a benchmark of 4,500 human-verified shadow masks across three U.S. cities at two native resolutions, paired with a leave-one-city-out evaluation protocol. Under vanilla training, holding out the test city costs 3.5 to 7.8 mIoU across four architectures, the gap remains significant when the training set grows from 225 to 900 images, and six of eight adaptation methods degrade more than vanilla training on average. We propose a three-probe framework that diagnoses where and how the transfer breaks: the failure operates in feature space, beyond the reach of pixel-histogram matching, shows up as a confidence collapse on shadow pixels while background confidence stays intact, and is recoverable by retraining different architectural components depending on the model: encoder for some, decoder for others. These findings replicate on INRIA building footprints, suggesting they describe geographic transfer beyond shadows. Guided by the diagnosis, we build Spectral Information Bottleneck (SIB), a plug-in module with one component per finding. Across the 9 (architecture, held-out city) cells, SIB has the smallest mean gap to the within-city upper bound of the ten methods we evaluate (6.42 mIoU), level with FDA. The benchmark, LOCO protocol, code, and trained models are released to support reproducible evaluation of geographic transfer in overhead dense prediction.
Figures & tables
| Dataset | Domain | Scale | Human QC | Cities | Multi-res | Transfer protocol |
| ISTD [ 12 ] | Ground-level | 1,870 | – | × | × | |
| AISD [ 10 ] | Aerial 0.3 m | 514 | 5 | × | × | |
| Airborne-Shadow [ 13 ] | Aerial | n/r | DE only | × | × | |
| S-EO [ 4 ] | Satellite 0.3 m | 19,000 | × (auto) | 3 (US) | × | × |
| ShadowTransfer (ours) | Aerial 0.3/0.6 m | 4,500 | 3 (US) | LOCO |
| LOCO mIoU | (mIoU) | Sig. | Pooled | |||||||||||
| Method | D | M | O | C † | D | M | O | C † | D | M | O | C † | Sig./18 | |
| Upper | — | — | — | — | — | — | — | — | — | — | ||||
| Vanilla | 70.0 | 77.1 | 67.0 | 75.3 | 6 | 6 | 6 | 3/3 | 18 | |||||
| FDA | 69.4 | 77.0 | 67.0 | 75.1 | 6 | 6 | 6 | 3/3 | 18 | |||||
| SegDeSiC | 73.0 | 76.1 | 65.0 | – | – | 5 | 6 | 6 | – | 17 | ||||
| IIM | 71.4 | 71.9 | 65.5 | – | – | 6 | 6 | 6 | – | 18 | ||||
| Variant | MAM | OGL | DIN |
| SAG | N/A | ||
| FDA, contrast | N/A | ||
| Haar | |||
| VIB on | |||
| VIB on (inv.) |
Appendix figures & tables47 assets
Supplementary material from the paper’s appendix.
Appendix
| Status (color) | Trigger | Kept segment | Annotator action | Share |
| Correct (green) | , IoU | Verify, usually skip | ||
| Potential_Annotated (yellow) | , IoU | (edit shape) | Adjust boundary | |
| Potential_Model (blue) | has no overlapping | if validated | Confirm or reject; assign type | |
| Incorrect (red) | has no overlapping | if validated | Verify segment is shadow |
| Phase | Sessions | Mean | Range across annotators | First-submission accept |
| Phase 1 (manual) | ||||
| Phase 2 (model-in-the-loop) | ||||
| Phase 3 (post-adjudication) | — | — |
| Category | Definition and inclusion rule |
| Building / canyon | Cast shadows from buildings and built structures, including narrow inter-building canyons. |
| Under-structure | Persistent occlusion shadows beneath elevated structures (bridges, overpasses, awnings, carports). |
| Tree-canopy dapple | Shadow patterns cast by tree foliage, including dappled shadow on the ground beneath the canopy. |
| Topography-cast | Shadows produced by terrain elevation changes (hills, embankments, cuttings). |
| Vehicle-cast | Shadows cast by cars, trucks, and other ground vehicles. |
| Thin linear | Shadows from thin vertical structures (poles, fences, antennas, wires) whose width is on the order of a few pixels. |
| City | Res. | Mean | AUC | MMD | Spat. |
| Chicago | High ( m) | ||||
| Chicago | Mid ( m) | ||||
| Miami | High ( m) | ||||
| Miami | Mid ( m) | ||||
| Phoenix | High ( m) | ||||
| Phoenix | Mid ( m) |
| City | Res. | Max | Mean | Sig./ |
| Chicago | High | |||
| Chicago | Mid | |||
| Miami | High | |||
| Miami | Mid | |||
| Phoenix | High | |||
| Phoenix | Mid |
| Hyperparameter | Value |
| Optimisation | |
| Optimizer | Adam |
| Learning rate | † |
| Weight decay | |
| Batch size | 8 |
| Epochs | 100 |
| Hyperparameter | Value |
| FDA low-freq. ratio | 0.01 |
| Target pool size | images |
| Target domain | Held-out city train/images (unlabelled) |
| Application point | Input (pre-resize, train only) |
| Target sampling | Uniform random per iteration |
| Early-stopping patience | 15 epochs |
| Hyperparameter | Value |
| Geographic branch | |
| Attachment point | feat5 ( ) |
| GRID scales / range | 10 / log-spaced |
| Encoding dim | 40 ( ) |
| MLP hidden dim / dropout | 256 / 0.3 (both layers) |
| Loss | |
| Hyperparameter | Value |
| IIM module | |
| Num. kernels | 8 |
| Kernel size | |
| Cross-channel pairs | (R,B), (R,G), (G,B) 24 features |
| Zero-mean projection | After every optimizer step |
| Loss | |
| Hyperparameter | Value |
| ISW configuration | |
| (ISW weight) | 0.6 |
| Hooked layers | feat1, feat2, feat3 |
| -means / | 3 / 1 (bottom 1 cluster content) |
| Photometric aug. (mask precomp.) | ColorJitter(0.4,0.4,0.4,0.1) |
| + GaussianBlur | |
| Hyperparameter | Value |
| Perturbation modules (training only) | |
| HRFP attach point | After stem (64 ch, ) |
| NP+ attach point | feat4 (512 ch, ) |
| HRFP+ attach point | Decoder dec3 (64 ch) |
| Inst. Norm layers | feat1 , feat2 , feat3 (affine=False) |
| HRFP / NP+ / HRFP+ prob. | 0.5 / 0.5 / 0.5 (independent) |
| Hyperparameter | Value |
| FADA configuration | |
| Adapter stages | feat3, feat4, feat5 |
| LoRA rank | 16 |
| Token length | 100 |
| LF branch | LoRA tokens on LL (content) |
| HF branch | LoRA tokens + IN on similarity map (LH/HL/HH) |
| Hyperparameter | Value |
| Optimisation | |
| Optimizer | Adamax |
| Learning rate | |
| Weight decay | Adamax default |
| Batch size | 8 (script override; train.py default 12) |
| Epochs | 100 |
| Hyperparameter | Value |
| FDA low-freq. ratio | 0.01 |
| Target pool size | images |
| Target domain | Held-out city train/images (unlabelled) |
| Target sampling | Uniform random per iteration |
| Application point | Input (dataset, pre-augmentation) |
| Modes run | LOCO only |
| Hyperparameter | Value |
| SegDeSiC module | |
| Attachment point | feat5 ( ) |
| GRID scales / range | 10 / log-spaced |
| Encoding dim | 40 |
| MLP hidden dim / dropout | 256 / 0.3 |
| Loss | |
| Hyperparameter | Value |
| IIM module | |
| Num. kernels | 8 |
| Kernel size | |
| Cross-channel pairs | (R,B), (R,G), (G,B) 24 features |
| Zero-mean projection | After every optimizer step |
| Loss | |
| Hyperparameter | Value |
| (ISW weight) | 0.6 |
| Hooked layers | glam1, glam2, glam3 |
| -means / | 3 / 1 |
| Photometric aug. (mask precomp.) | ColorJitter(0.4,0.4,0.4,0.1) |
| + GaussianBlur | |
| Loss reduction | Per-image mean, then layer mean |
| Hyperparameter | Value |
| Perturbation modules (training only) | |
| HRFP attach point | After stem (64 ch, ) |
| NP+ attach point | glam4 (512 ch, ) |
| HRFP+ attach point | Decoder s1_d_up (64 ch, ) |
| Inst. Norm layers | Removed (Option A) |
| HRFP / NP+ / HRFP+ prob. | 0.5 / 0.5 / 0.5 |
| Hyperparameter | Value |
| FADA configuration | |
| Adapter stages | GLAM 3, 4 (LFEM output, pre-GFEM) |
| LoRA rank | 16 |
| Token length | 100 |
| Decomposition | Haar DWT |
| Optimisation overrides | |
| Hyperparameter | Value |
| Optimisation | |
| Optimizer | AdamW ( , ) |
| Learning rate | (CLI; argparse default ) |
| Weight decay | 0.05 |
| Batch size | 8 |
| Epochs (max) | 100 |
| Hyperparameter | Value |
| FDA low-freq. ratio | 0.01 |
| Target pool size | images |
| Target domain | Held-out city train/images (unlabelled) |
| Application point | Input (pre-tensor, pre-normalisation) |
| Early-stopping patience | 10 (all modes) |
| Modes run | LOCO only |
| Hyperparameter | Value |
| SegDeSiC module | |
| Attachment point | feat_block11 ( ) |
| GRID scales / range | 10 / |
| Encoding dim | 40 |
| MLP hidden dim / dropout | 256 / 0.3 |
| Loss | |
| Hyperparameter | Value |
| IIM module | |
| Num. kernels | 8 |
| Kernel size | |
| Cross-channel pairs | (R,B), (R,G), (G,B) 24 features |
| Zero-mean projection | After every optimizer step |
| Loss | |
| Hyperparameter | Value |
| (ISW weight) | 0.6 |
| Hooked blocks | ViT blocks 3, 6, 9 |
| Token reshape | Strip CLS; |
| -means / | 3 / 1 |
| Photometric aug. (mask precomp.) | ColorJitter(0.4,0.4,0.4,0.1) |
| + GaussianBlur |
| Hyperparameter | Value |
| Perturbation modules (training only) | |
| HRFP attach point | feat_block3 (384 ch, ) |
| NP+ attach point | feat_block11 (384 ch, ) |
| HRFP+ attach point | Decoder up3 (64 ch, ) |
| HRFP+ projection | Frozen conv |
| Inst. Norm layers | Removed |
| Hyperparameter | Value |
| FADA configuration | |
| Adapter blocks | ViT blocks 3, 6, 9, 11 |
| LoRA rank | 16 |
| Token length | 100 |
| LF branch | LoRA tokens on LL |
| HF branch | LoRA tokens + IN on similarity map (LH/HL/HH) |
| Hyperparameter | Value |
| Architecture | |
| Backbone | DINOv2 ViT-L/14 at patch 16, frozen ( M parameters) |
| Adapters | Rein, token length 100, LoRA rank 16 |
| Decoder | Mask2Former, 2 classes |
| Trainable parameters | M |
| Optimisation | |
| Hyperparameter | Value |
| Style network | AdaIN: VGG-19 encoder to relu4_1 + mirror decoder, frozen |
| Style-network weights | Public pretrained AdaIN release; 0 trainable parameters |
| Style source | Permuted training batch |
| Mixing coefficient | 0.5 (fixed; checked for non-degeneracy on source-city validation) |
| Application probability | 1.0 per training batch |
| Batch size | 8 (CrossEarth: 4) |
| Hyperparameter | Value |
| Adapted parameters | Affine of BatchNorm/LayerNorm/GroupNorm layers |
| Excluded (upstream) | layer4 , ViT blocks 9–11, final norm |
| Normalisation statistics | Source running statistics (evaluation mode) |
| Optimizer | SAM ( ) over SGD (momentum 0.9) |
| Learning rate | (upstream ViT default, not batch-scaled) |
| Steps per image | 1 (MAMNet, OGLANet, DINOv3); 10 (CrossEarth) |
| Method | City | Res. | Upper | LOCO | 95% CI | ||
| Vanilla | Chicago | HR | 86.31 | 80.45 | |||
| Chicago | MR | 86.35 | 83.22 | ||||
| Miami | HR | 79.11 | 74.67 | ||||
| Miami | MR | 78.89 | 76.75 | ||||
| Phoenix | HR | 73.75 | 64.23 | ||||
| Phoenix | MR | 85.64 | 83.40 |
| Method | City | Res. | Upper | LOCO | 95% CI | ||
| Vanilla | Chicago | HR | 79.12 | 68.47 | |||
| Chicago | MR | 80.77 | 65.08 | ||||
| Miami | HR | 73.89 | 66.18 | ||||
| Miami | MR | 74.19 | 70.73 | ||||
| Phoenix | HR | 66.15 | 61.18 | ||||
| Phoenix | MR | 75.39 | 70.32 |
| Method | City | Res. | Upper | LOCO | 95% CI | ||
| Vanilla | Chicago | HR | 80.27 | 67.26 | |||
| Chicago | MR | 83.63 | 75.23 | ||||
| Miami | HR | 76.00 | 72.61 | ||||
| Miami | MR | 75.76 | 68.11 | ||||
| Phoenix | HR | 67.82 | 65.24 | ||||
| Phoenix | MR | 81.85 | 71.52 |
| Arch. | Method | City | Upper | LOCO | 95% CI | ||
| MAMNet | SAR | Chicago | 86.31 | 80.45 | |||
| Miami | 79.11 | 74.66 | |||||
| Phoenix | 73.75 | 64.22 | |||||
| OGLANet | SAR | Chicago | 79.12 | 68.70 | |||
| Miami | 73.89 | 66.36 | |||||
| Phoenix | 66.15 | 61.18 |
| Held-out city | Sig./42 | |||
| Chicago | ||||
| Miami | ||||
| Phoenix |
| Resolution | Sig./63 | |||
| High-res ( m/px) | ||||
| Mid-res ( m/px) |
| Method | Sig./18 | |||
| Vanilla | ||||
| FDA | ||||
| SegDeSiC | ||||
| IIM | ||||
| MRFP+ | ||||
| FADA |
| Architecture | Exp. A: decoder | Exp. B: encoder |
| DINOv3 | ||
| MAMNet | ||
| OGLANet |
| Rung 225 | Rung 450 | Rung 900 | |
| MAMNet | (9) | (9) | (9) |
| OGLANet | (8) | (9) | (9) |
| DINOv3 | (7) | (9) | (9) |
| All | (24) | (27) | (27) |
| Arch. | City | Vanilla gap | SD | Aug. gap | Threshold | Seeds | Changed | |
| MAMNet | Chicago | 1/1 | no | |||||
| MAMNet | Miami | 2/1 | no | |||||
| MAMNet | Phoenix | 3/1 | no | |||||
| OGLANet | Chicago | 3/2 | no | |||||
| OGLANet | Miami | 1/1 | no | |||||
| OGLANet | Phoenix | 1/1 | yes |
| Group | Parameter | Value |
| Content VIB | (constant) | (MAMNet, OGLANet); (DINOv3) |
| VIB hidden width | ||
| Edge VIB | ( on MAMNet) | |
| VIB hidden width | ||
| SAG (U-Net) | gate bias init | |
| CNN preproc | FDA spectral radius |
| Method | Access | Mean [95% CI] | Worst (cell) | Harm/benefit | Largest harm | SD | SIB method [95% CI] | Holm | |
| SIB | S/U | 9 | (OGLANet/Mia) | 2/2 | – | – | |||
| FDA | U | 9 | (DINOv3/Phx) | 1/4 | |||||
| SegDeSiC | U | 9 | (OGLANet/Mia) | 2/3 | |||||
| SAR | D | 9 | (DINOv3/Chi) | 2/1 | |||||
| Vanilla | S | 9 | (DINOv3/Chi) | – | – | ||||
| BSM | S | 8 | (OGLANet/Chi) | 6/2 |
| Variant | MAMNet | OGLANet | DINOv3 |
| Reference (mIoU) | |||
| VIB on | – | ||
| All VIB | – | – | |
| Edge VIB | – | – | |
| Uniform | |||
| Symmetric VIB |
| Arch | Holdout | val err. at | val err. at | reduction | |
| MAMNet | Phoenix | ||||
| MAMNet | Miami | ||||
| MAMNet | Chicago | ||||
| OGLANet | Phoenix | ||||
| OGLANet | Miami | ||||
| OGLANet | Chicago |
| Mean err. | Wilcoxon | Cells improving | Median | |
| Arch | Holdout | |||
| MAMNet | Phoenix | |||
| MAMNet | Miami | |||
| MAMNet | Chicago | |||
| OGLANet | Phoenix | |||
| OGLANet | Miami | |||
| OGLANet | Chicago |
Cross-Domain Transfer Learning
Cross-domain transfer learning reuses knowledge across datasets, populations, devices, or tasks when their distributions and feature spaces differ.
See all 218 papers