cs.CVJan 30, 2026

How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models

Authors: Leonard Hackel, Tom Burgert, Begüm Demir

Organizations: Big Data Analytics for Earth Observation BIFOLD / TU Berlin 10587 Berlin, Germany

Abstract

Large-scale foundation models (FMs) in remote sensing (RS) (denoted as RS FMs) are developed following paradigms established in computer vision (CV), yet the validity of transferring CV scaling laws to RS has not been systematically examined. We hypothesize that RS FMs enter an overparameterized regime at substantially smaller scales than their CV counterparts, with task-relevant information encoded redundantly across model dimensions. To test this hypothesis, we apply post-hoc slimmability, uniform width reduction of pretrained encoder transformer blocks, to measure representational redundancy across ten state-of-the-art RS FMs on classification, segmentation, and change detection. RS FMs retain on average over 85% relative accuracy on RS classification datasets under aggressive width reduction, while masked autoencoder (MAE) and DINOv2 pretrained on natural images (denoted as CV MAE and CV DINOv2) degrade sharply on ImageNet subsets of matched class count over the same range of computational requirements. A CV MAE evaluated directly on the same RS dataset narrows but does not close the gap, indicating that both dataset characteristics and domain-specific pretraining contribute to the differences between the models. Analyses of the representation space of the models indicate that task-relevant variance concentrates in few principal components and is redundantly encoded across model dimensions. We further show that learned slimmable training improves over post-hoc slimmability for contrastive objectives, but reconstruction-based objectives do not benefit consistently from current slimmable training protocols. Our findings establish post-hoc slimming as a practical deployment strategy for resource-constrained RS applications and as a diagnostic tool for representational redundancy in RS FMs. Code is available at https://github.com/hackelle/slimmable-remote-sensing-foundation-models.

Figures & tables

Explore similar work

CardsList
  1. REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent

    Nov 21, 2025Binger Chen, Tacettin Emre Bök, Behnood Rasti +2Model SelectionGeospatial Foundation Models

  2. Multimodal Foundation Models Adaptation based on Domain-Aware Relaxed Orthogonal Subspace for Remote Sensing

    Sep 12, 2026Han Luo, Ruoyu Yang, Yinhe Liu +1Vision Foundation Model AdaptationParameter-Efficient Fine-Tuning

  3. Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models

    Jun 24, 2026Qinzhe Yang, Keyan Chen, Jia Xu +2Remote Sensing Image SegmentationInstance Segmentation