Embedding-Only Uplink for Onboard Retrieval Under Shift in Remote Sensing

Sangcheol Sim

cs.CVMar 30, 2026

Embedding-Only Uplink for Onboard Retrieval Under Shift in Remote Sensing

Authors: Sangcheol Sim

Organizations: Telepix

Abstract

Downlink bottlenecks motivate onboard systems that prioritize hazards without transmitting raw pixels. We study a strict setting where a ground station uplinks only compact embeddings plus metadata, and an onboard system performs vector search to triage new captures. We ask whether this embedding-only pipeline remains useful under explicit remote-sensing shift: cross-time (pre/post-event), cross-event/location (different disasters), cross-site cloud (15 geographic sites), and cross-city AOI holdout (buildings). Using OlmoEarth embeddings on a scaled public multi-task benchmark (27 Sentinel-2 L2A scenes, 15 cloud sites, 5 SpaceNet-2 AOIs; 10 seeds), we find that all effective methods rely on the same uplinked embeddings, but the optimal decision head is task-dependent: kNN retrieval is significantly superior for cloud classification (0.92 vs. centroid 0.91; p<0.01, Wilcoxon), while class centroids dominate temporal change detection (0.85 vs. retrieval 0.48; p<0.01). These results show that embedding-only uplink is the key enabler--once embeddings are onboard, the system can select the best head per task at no additional uplink cost, with all telemetry under 1 KB per query.

Figures & tables

Figure 1: Embedding-only uplink pipeline. The ground station computes hint embeddings and uplinks (embedding, metadata) tuples—no imagery. The onboard system indexes hints in a vector database and retrieves top- $k$ matches for each new capture via cosine similarity, emitting compact JSON telemetry ( ${\sim}$ 700 B at $k{=}5$ ) as the downlink product.

Figure 2: Qualitative retrieval example (hazard task). A Derna flood query retrieves same-event crops (sim $>$ 0.96) before a cross-event match (Maui wildfire, sim $=$ 0.91), demonstrating embedding-level hazard grouping.

Task	Metric	Retrieval	Random	Centroid	Lin. probe	No-retr.	Oracle
Hazard (S2)	Recall@5	$\mathbf{1.00}{\scriptstyle\pm.00}$	$.15{\scriptstyle\pm.09}^{**}$	$.99{\scriptstyle\pm.04}$	$\mathbf{1.00}{\scriptstyle\pm.00}$	$.00{\scriptstyle\pm.00}^{**}$	$1.00{\scriptstyle\pm.00}$
Hazard (S2)	Top-1	$\mathbf{.91}{\scriptstyle\pm.08}$	$.03{\scriptstyle\pm.03}^{**}$	$.86{\scriptstyle\pm.12}$	$.90{\scriptstyle\pm.08}$	$.00{\scriptstyle\pm.00}^{**}$	$1.00{\scriptstyle\pm.00}$
Change (S2)	Time-pref	$.48{\scriptstyle\pm.08}$	$.00{\scriptstyle\pm.00}^{**}$	$\mathbf{.85}{\scriptstyle\pm.15}^{**}$	$.24{\scriptstyle\pm.19}^{**}$	$.00{\scriptstyle\pm.00}^{**}$	$1.00{\scriptstyle\pm.00}$
Cloud (S2)	Bal. acc	$\mathbf{.92}{\scriptstyle\pm.04}$	$.50{\scriptstyle\pm.05}^{**}$	$.91{\scriptstyle\pm.05}^{**}$	$.92{\scriptstyle\pm.04}$	$.50{\scriptstyle\pm.00}^{**}$	$1.00{\scriptstyle\pm.00}$
Build. (SN2)	Macro-F1	$.51{\scriptstyle\pm.26}$	$.42{\scriptstyle\pm.10}$	$\mathbf{.70}{\scriptstyle\pm.12}$	$.64{\scriptstyle\pm.17}$	$.40{\scriptstyle\pm.03}$	$1.00{\scriptstyle\pm.00}$

Table 1: Multi-task results at

k{=}5

(mean

\pm

std, 10 seeds, xlarge benchmark). Bold: best non-oracle per row. Significance markers from paired Wilcoxon signed-rank test vs. retrieval:

{}^{**}p{<}0.01

{}^{*}p{<}0.05

Figure 3: $k$ -sweep: task metric vs. telemetry bytes ( $k\in\{1,5,10\}$ ). Dashed horizontal lines show $k$ -independent baselines (centroid, linear probe). Buildings favors small $k$ ; change improves with larger $k$ . Error bars: $\pm$ 1 std over 10 seeds.

Domain Generalization

Domain generalization develops models that perform reliably in new environments without training on target-domain data.

See all 375 papers

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing
Dec 12, 2025Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan +2Multimodal Understanding Cross-Modal Learning
Linköping University, Sweden
Zero-Shot Satellite Image Retrieval through Joint Embeddings: Application to Crisis Response
May 6, 2026James Walsh, William Fawcett, Grace Colverd +1Geospatial Foundation Models Disaster Response
1Trillium Technologies · University of Cambridge · 3Universidad de Antioquia
Earth Embeddings
Aug 4, 2026Adam J. Stewart, Heng Fang, Isaac A. Corley +1Geospatial Representation Learning Land Cover Classification
Chair of Data Science in Earth Observation, Technical University of Munich, Munich, Germany · Munich Center for Machine Learning, Munich, Germany · KTH Royal Institute of Technology, Stockholm, Sweden +1

Dec 12, 2025J/K move · Enter open · S save

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

Emanuel Sánchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan, Yonghao Xu+1

Linköping University, Sweden

Multimodal Understanding Cross-Modal Learning Multimodal Embedding Cross-Modal Retrieval Vision-Language Grounding Remote Sensing Benchmarks+2

Open paper AskPDF arXiv

Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both region-level spatial reasoning and holistic scene understanding. Existing remote-sensing approaches are fragmented: dual-encoder retrieval models scale well but cannot interleave modalities, whereas generative assistants support grounding, yet are inefficient for retrieval. Benchmarks mirror this split: interleaved evaluations mainly target generative assistants, while cross-modal retrieval benchmarks target dual encoders. To bridge this gap, we introduce RSMEB, a unified remote sensing benchmark that evaluates cross-modal and interleaved retrieval across 21 tasks under a single ranking protocol, enabling comprehensive comparison of retrieval models on region- and geo-aware capabilities as well as conventional retrieval. As a strong reference baseline, we present VLM2GeoVec, an instruction-conditioned, single-encoder interleaving formulation tailored to remote sensing that packs image, text, bounding-box, and geo-coordinate tokens into one sequence and learns a unified embedding via contrastive training. Across RSMEB, VLM2GeoVec achieves

\textbf{26.6\%}

P@1 in region-caption retrieval (

\textbf{+25}

percentage points),

\textbf{32.5\%}

in referring-expression retrieval (

\textbf{+19}

), and

\textbf{17.8\%}

in semantic geo-aware retrieval (

\textbf{>3}$$\times

prior best), while remaining competitive in conventional scene classification and text--image retrieval in zero-shot settings. Together, the proposed suite and reference baseline standardize evaluation and deliver a unified embedder for scalable retrieval and region-/geo-aware grounding. The code, the model checkpoints, and the data are available at https://github.com/emasa/VLM2GeoVec.

Abstract

Domain Generalization

Explore similar work

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

Zero-Shot Satellite Image Retrieval through Joint Embeddings: Application to Crisis Response

Earth Embeddings