cs.CVNov 10, 2026

CroCoMAE: Cross-modal self-supervised collaborative representation learning for SAR-optical remote sensing data

Authors: Vito Recchia, Giuseppe Guarino, Cássio Fraga Dantas, Giuseppina Andresini, Annalisa Appice, Dino Ienco

Organizations: Department of Computer Science, University of Bari Aldo Moro, Bari, Italy · INRIA, EVERGREEN, Univ. of Montpellier, Montpellier, France · Consorzio Interuniversitario Nazionale per l’Informatica (CINI), Bari, Italy

Abstract

Remote sensing increasingly captures the Earth's surface through complementary sensors such as SAR and optical, yet ground-truth annotations remain scarce, limiting supervised approaches. Self-supervised learning (SSL) addresses the label scarcity, and cross-modal SSL further exploits the complementarity between co-located sensors, but existing cross-modal frameworks align modalities mostly through global image-level representations or treat them as largely redundant, leaving their fine-grained complementary structure underexploited. We propose CroCoMAE, a cross-modal SSL framework for SAR-optical remote sensing that unifies contrastive learning and masked image modeling within a ViT-based architecture, operating at two complementary levels. At the image level, an intra-modality contrastive loss on the class token yields globally discriminative, augmentation-invariant representations. At the patch level, masked image modeling drives fine-grained per-modality representations, while a cross-modal contrastive objective aligns a shared subspace across SAR and optical patch tokens, leaving the complementary subspace unconstrained to retain modality-specific information. On two SAR-optical benchmarks covering scene classification and semantic segmentation, CroCoMAE improves over uni-modal SSL baselines and outperforms state-of-the-art cross-modal frameworks in most configurations, with gains in both single-modality and multi-modal evaluation across varying label regimes and the largest improvements on semantic segmentation.

Figures & tables

Explore similar work

CardsList
  1. TAR: Text Semantic Assisted Cross-modal Image Registration Framework for Optical and SAR Images

    May 12, 2026Zhuoyu Cai, Dou Quan, Ning Huyan +3Remote SensingSynthetic Aperture Radar

  2. MMATR-X: High-Resolution Optical-SAR Joint Pretraining via Conditioned Alignment and Degraded-Target Prediction

    Apr 18, 2026Bowen Peng, Yongxiang Liu, Jie Zhou +4Cross-Modal LearningMasked Autoencoders

  3. OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding

    Mar 25, 2026Xiaoyu Tang, Jun Dong, Jintao Cheng +1Remote SensingVision-Language Grounding