cs.CVNov 4, 2026

CAPE-JEPA - Cross-modal Adversarial Probabilistic Embedding JEPA

Authors: Aniket Junghare, Biplab Banerjee

Organizations: Indian Institute of Technology Bombay, Mumbai, India

Abstract

Cross-modal remote sensing image retrieval (RS-CMIR) requires representations that are simultaneously modality-invariant and semantically discriminative, despite the profound physical gap between heterogeneous sensors such as Synthetic Aperture Radar (SAR) and multispectral imagery. Recent JEPA-based Predictive self-supervised approaches such as X-JEPA formulate cross-modal alignment as a deterministic latent forecasting task, implicitly assuming uniform patch predictability and leaving cross-modal distributional alignment unaddressed. We propose CAPE-JEPA, a probabilistic cross-modal Joint Embedding Predictive Architecture with three contributions: (i) a cross-attentional Gaussian predictor outputting per-patch parameters (mu, sigma^2) conditioned on opposite-modality context, trained via negative log-likelihood to model inherent cross-sensor prediction uncertainty; (ii) adversarial distributional regularization combining unimodal isotropic Gaussian regularization with a novel cross-modal adversarial variant that aligns modality distributions via worst-case Epps-Pulley characteristic function tests without explicit sample pairing; and (iii) in-training contrastive learning with attention-pooled heads for scene-level discriminative grounding. Evaluated on BigEarthNet-14K (BEN-14K) and fMoW across SAR, multispectral, and RGB modalities, CAPE-JEPA surpasses X-JEPA by up to 13.74 points on BEN-14K and 15.59 points on fMoW, outperforming all baselines including SkySense and CROMA across both unimodal and cross-modal directions while using fewer parameters (407.30M vs. 432.72M).

Figures & tables

Explore similar work

CardsList
  1. CR-JEPA: Cross-Modal Joint-Embedding Predictive Learning for Remote Sensing Image Retrieval

    May 30, 2026Md Aminur Hossain, Ayush V. Patel, Nitant Dube +1Cross-Modal Representation LearningJoint-Embedding Predictive Architecture

  2. RS-JEPA: Remote Sensing Joint-Embedding Predictive Architecture with Scan-Path Views for Cross-Sensor Image Retrieval

    Nov 10, 2026Tanishq Rachamalla, Srishti Kaushik, Muhammad Attique Khan +2Image RetrievalJoint-Embedding Predictive Architecture

  3. HQ-JEPA: Hybrid Quantum Joint-Embedding Predictive Architecture for Cross-Modal Remote Sensing Representation Learning

    May 29, 2026Md Aminur Hossain, Ayush V. Patel, Sanjay K. Singh +1Representation LearningCross-Modal Representation Learning