SARVIA: Fast-Adapting the Strong SAR Foundation Models That Are Already in Your MLLMs
Organizations: Singapore Management University
Abstract
Synthetic aperture radar (SAR) foundation models provide transferable representations for diverse perception tasks in all-weather Earth observation. However, their development often relies on costly large-scale SAR pretraining, raising the question of whether such extensive domain-specific pretraining is necessary. We find that modern multimodal large language models (MLLMs) already provide visual encoders that can outperform dedicated SAR foundation models as downstream initializations. This motivates a new paradigm: reuse general visual capabilities and devote domain training to brief specialization. We therefore introduce SARVIA, an adaptive post-pretraining method that specializes modern MLLMs using self-supervised visual questions with computable answers. These questions are constructed from unlabeled observations through general image tasks that capture spatial relations and SAR-specific tasks that exploit native measurements and SAR--optical correspondence. A shared answer-prediction objective jointly trains low-rank adapters across visual and language components, with loss-driven task sampling. We then export the adapted visual encoder as a standalone backbone. With 600 adaptation steps and 2 GPU-hours, SARVIA outperforms the compared foundation models on all six SAR detection and segmentation benchmarks. Compared with dedicated SAR pretraining, SARVIA reduces domain-training costs by orders of magnitude while improving visual transfer across architectures and scales. This new paradigm shifts the emphasis from repeated, costly pretraining to a reusable specialization process that can benefit from advances in open MLLMs. Dataset and code: https://github.com/zcablii/SARVIA.
Figures & tables
| Dataset | Task | Categories | Metric | Train | Val. | Test | Prediction head | Input / crop | Schedule |
|---|---|---|---|---|---|---|---|---|---|
| RSAR | Oriented det. | 6 | AP 50 | 78,837 | – | 8,538 | Oriented R-CNN [ 48 ] | 12 epochs | |
| M4-SAR | Oriented det. | 6 | AP 50 | 56,116 | 22,112 | 33,946 | Oriented R-CNN [ 48 ] | 12 epochs | |
| SARDet-100K | Horizontal det. | 6 | mAP | 94,493 | 10,492 | 11,613 | Cascade R-CNN [ 4 ] | 12 epochs | |
| OpenEarthMap-SAR | Segmentation | 8 | mIoU | 4,333 | 210 | 490 | UPerNet [ 46 ] | 40k steps | |
| AIR-PolSAR-Seg-2.0 | Segmentation | 6 | mIoU | 1,233 | 309 | – | UPerNet [ 46 ] | 40k steps | |
| WHU-OPT-SAR | Segmentation | 7 | mIoU | 17,640 | 5,880 | 5,880 | UPerNet [ 46 ] | 40k steps |
| Encoder source | Backbone | Vanilla | +SARVIA | GPU-hours |
|---|---|---|---|---|
| SAR / Earth-observation foundation models | ||||
| TerraMind [ 20 ] | ViT-L | 34.69 | — | 5,122 |
| SARMAE [ 31 ] | ViT-L | 33.07 | — | 320 |
| MLLM visual encoders | ||||
| Qwen3-VL-2B [ 1 ] | ViT-L | 28.42 | 32.66 (+4.24) | 1.87 |
| InternVL2.5-2B [ 8 ] | ViT-L | 33.79 | 35.37 (+1.58) | 2.71 |
| Method | Backbone | AP 50 |
| O-RCNN [ 48 ] | ResNet-50 | 64.82 |
| SARMAE [ 31 ] | ViT-B | 66.80 |
| RoI Transformer [ 12 ] | ResNet-50 | 66.95 |
| GSDet [ 13 ] | ResNet-50 | 68.00 |
| SARVIA (Ours) | ViT-B | 70.29 |
| OlmoEarth [ 18 ] | ViT-L | 57.93 |
| Method | Backbone | mAP |
| SUMMIT [ 15 ] | ViT-B | 50.86 |
| SARMAE [ 31 ] | ViT-B | 52.00 |
| SARATR-X [ 22 ] | HiViT-B | 54.20 |
| DINO (UniSAR-7M) [ 5 , 41 ] | ViT-B | 54.22 |
| SARVIA (Ours) | ViT-B | 55.10 |
| SatMAE [ 10 ] | ViT-L | 40.98 |
| Method | Backbone | AP 50 |
| OlmoEarth [ 18 ] | ViT-B | 21.86 |
| TerraMind [ 20 ] | ViT-B | 39.12 |
| SUMMIT [ 15 ] | ViT-B | 40.18 |
| SARMAE [ 31 ] | ViT-B | 40.73 |
| SARVIA (Ours) | ViT-B | 41.23 |
| SatMAE [ 10 ] | ViT-L | 23.26 |
| Method | Backbone | mIoU |
| SARATR-X-v2 [ 21 ] | iTPN-B | 31.34 |
| VMamba [ 35 ] | VSS | 34.74 |
| U-Net [ 40 ] | EfficientNet-B4 | 35.13 |
| SARVIA (Ours) | ViT-B | 35.76 |
| OlmoEarth [ 18 ] | ViT-L | 27.85 |
| SatMAE [ 10 ] | ViT-L | 27.91 |
| Method | Backbone | mIoU |
| SARATR-X-v2 [ 21 ] | iTPN-B | 83.43 |
| TerraSegNet [ 44 ] | EfficientNetV2-S | 92.10 |
| SARVIA (Ours) | ViT-B | 93.37 |
| SatMAE [ 10 ] | ViT-L | 78.75 |
| OlmoEarth [ 18 ] | ViT-L | 79.25 |
| TerraMind [ 20 ] | ViT-L | 88.12 |
| Method | Backbone | mIoU |
| PIDNet [ 49 ] | PIDNet-S | 44.30 |
| OCRNet [ 50 ] | HRNetV2-W48 | 44.80 |
| SegFormer [ 47 ] | MiT-B4 | 45.30 |
| SARATR-X-v2 [ 21 ] | iTPN-B | 46.10 |
| Mask2Former [ 9 ] | ResNet-101 | 46.80 |
| TransUNet [ 6 ] | ResNet-50 + ViT | 47.10 |
| Sampling | Task families | mIoU |
| Base encoder | — | 36.29 |
| Static | General image only | 37.33 |
| SAR-specific only | 37.94 | |
| Both | 38.21 | |
| TSCL [ 37 ] | Both | 37.75 |
| DWA [ 33 ] | 37.33 |
| Update | QA draws | Last-used TV | Budget shifted |
|---|---|---|---|
| 400 | 51,200 | 21.85% | 5.46% |
| 600 | 76,800 | 22.97% | 10.55% |
| 800 | 102,400 | 24.53% | 13.04% |
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
| Source | Train | Eval |
|---|---|---|
| TerraMesh | 703,460 | 257,276 |
| SARLO-80 | 101,699 | 8,700 |
| KuroSiwo | 53,889 | 51,152 |
| Total | 859,048 | 317,128 |
| Task group | Task and answer | Var. | T | K | S |
| General image tasks | |||||
| Basic spatial tasks | Recover SAR patch order, match crops, identify flips or missing patches, and locate polygonal crops. Return a permutation, candidate index, flip descriptor, or box. | 6 | 6 | 6 | 6 |
| Single-modality ordering | Reorder SAR or optical patches using a same-modality reference, one fixed anchor, or no reference. Return the permutation restoring a or grid. | 12 | 6 | 6 | 12 |
| Reference flips | Compare a SAR image with its unchanged reference to identify the flipped cell in a or grid. Return its index and flip direction. | 2 | 2 | 2 | 2 |
| Multiscale matching | Match a SAR or optical query resized by or to three same-modality candidate regions. Return the matching index. | 4 | 2 | 2 | 4 |
| Subtotal | 24 | 16 | 16 | 24 | |
| Setting | Main experiment |
|---|---|
| MLLM | Qwen3.5-2B [ 39 ] |
| LoRA rank / / dropout | 16 / 32 / 0 |
| Adapter placement | All linear layers |
| Learning rate: vision / merger | / |
| Learning rate: language | |
| Schedule / warmup | Constant / none |
Vision Foundation Model Adaptation
Vision foundation model adaptation tailors pretrained image and video models to new tasks and domains while balancing accuracy, efficiency, and generalization.
See all 352 papers