Overview

Remote sensing benchmarks provide shared datasets, splits, metrics, and protocols for testing satellite, aerial, SAR, hyperspectral, and LiDAR models on tasks such as land-cover mapping, change detection, geo-localization, and vision-language reasoning. They compare CNNs and transformers with foundation models using supervised, zero-shot, frozen-encoder, and fine-tuning evaluations, often with cross-region, cross-year, or cross-sensor splits. Results show that rankings can change across tasks and conditions: geographic and sensor shifts, fine details, temporal reasoning, and complex grounding expose weaknesses that clean in-distribution scores may hide. A central challenge is to make comparisons reproducible and realistic while measuring calibration, geometric accuracy, efficiency, robustness, and downstream utility rather than relying on a single aggregate score.

Latest papers

218 papers22 papers in the last four weeks, down 12% on the four weeks before. 0.2% of all new papers.

Jan 26, 2026

DisasterInsight: A Building-Centric Benchmark for Evaluating Vision--Language Models in Disaster Response

Vision--language models (VLMs) show promise for disaster-response remote sensing, but existing benchmarks mainly emphasize scene-level or damage-centric assessment. To study this building-centric gap, we introduce \method{}, a diagnostic benchmark built on xBD, a pre/post-disaster satellite dataset with building-level damage labels. \method{} enriches building instances with OpenStreetMap-derived functional labels and contains 134{,}108 task-specific instruction records across 15 task types, spanning instance-level assessment, scene-level counting, multi-instance reasoning, and structured report generation. The benchmark supports RGB pre/post-disaster imagery, single- and multi-view instance formulations, and scene-level RGB/SAR diagnostic inputs. Experiments with general-domain and remote-sensing VLMs show that models perform better on visible damage cues than on building-function understanding, multi-instance reasoning, counting, and grounded reporting. Instruction tuning improves performance on several tasks but does not close this building-centric gap.
Jan 19, 2026

SQuID: A Benchmark for Quantitative Spatial Reasoning on Satellite Imagery

Vision-Language Models (VLMs) describe satellite images fluently but struggle with the quantitative spatial reasoning that remote sensing (RS) analysts perform routinely, such as counting buildings, measuring forest area in hectares and computing distances in meters. We introduce SQuID (Satellite Quantitative Intelligence Dataset), a benchmark of 2,000 questions over 587 images from four sources at 0.3 to 1.0 m ground sampling distance, covering 24 question types in three difficulty tiers. Answers are computed from curated segmentation masks or, for 50 questions, from human consensus, and numeric answers are scored against tolerance ranges informed by human annotations. SQuID also releases the reference masks behind its answers, so that modular systems can be evaluated using reference rather than predicted masks, and 75,687 training question-answer pairs on disjoint images (SQuID-Train). Among 15 evaluated systems, the best zero-shot end-to-end VLM reaches 28.1%, and RS-specialized VLMs score lower. As a baseline, we propose QASR (Quantitative Agentic Spatial Reasoning), in which an LLM writes a program that segments the image and measures the masks with geometric operators. QASR reaches 42.0%. On the 1,950 mask-derived questions, running its programs on the reference masks instead of predicted masks raises accuracy from 42.2% to 82.6%, so segmentation is its main source of error. Fine-tuning Qwen2.5-VL-7B on SQuID-Train raises its accuracy from 16.1% to 40.7%, close to QASR. Multi-condition questions (Tier 3) remain the hardest in every setting, at 62.0% for QASR with reference masks and 13.2% after fine-tuning. These results leave clear room for end-to-end models that learn metric spatial reasoning, and SQuID provides the data to train and evaluate them.
Dec 12, 2025

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both region-level spatial reasoning and holistic scene understanding. Existing remote-sensing approaches are fragmented: dual-encoder retrieval models scale well but cannot interleave modalities, whereas generative assistants support grounding, yet are inefficient for retrieval. Benchmarks mirror this split: interleaved evaluations mainly target generative assistants, while cross-modal retrieval benchmarks target dual encoders. To bridge this gap, we introduce RSMEB, a unified remote sensing benchmark that evaluates cross-modal and interleaved retrieval across 21 tasks under a single ranking protocol, enabling comprehensive comparison of retrieval models on region- and geo-aware capabilities as well as conventional retrieval. As a strong reference baseline, we present VLM2GeoVec, an instruction-conditioned, single-encoder interleaving formulation tailored to remote sensing that packs image, text, bounding-box, and geo-coordinate tokens into one sequence and learns a unified embedding via contrastive training. Across RSMEB, VLM2GeoVec achieves 26.6%\textbf{26.6\%} P@1 in region-caption retrieval (+25\textbf{+25} percentage points), 32.5%\textbf{32.5\%} in referring-expression retrieval (+19\textbf{+19}), and 17.8%\textbf{17.8\%} in semantic geo-aware retrieval (\textbf{>3}$$\times prior best), while remaining competitive in conventional scene classification and text--image retrieval in zero-shot settings. Together, the proposed suite and reference baseline standardize evaluation and deliver a unified embedder for scalable retrieval and region-/geo-aware grounding. The code, the model checkpoints, and the data are available at https://github.com/emasa/VLM2GeoVec.
Nov 13, 2025

OpenSR-SRGAN: A Flexible Super-Resolution Framework for Multispectral Earth Observation Data

Satellite images often contain useful measurements beyond visible red, green and blue light, but at a spatial resolution that is too coarse for some applications. Super-resolution methods estimate a finer-resolution image from these coarse observations. OpenSR-SRGAN is a modular, configuration-driven framework for training and benchmarking such models on multispectral Earth-observation data. The software enables users to train and evaluate configurable generator-discriminator architectures on arbitrary sensor band setups using concise YAML configurations, without modifying model code. All components, such as generators, discriminators, loss functions, training schedules, normalizations, and stability heuristics, are exposed through a common YAML interface. OpenSR-SRGAN makes it straightforward to reproduce experiments, compare architectures, and deploy super-resolution pipelines across diverse remote-sensing datasets. OpenSR-SRGAN supports complete end-to-end workflows with minimal setup: selecting architectures, scaling factors, band combinations, and training strategies entirely from configuration files. Although designed for remote-sensing super-resolution, its tensor-based model components can be adapted to other paired imaging data when suitable preprocessing and validation are provided.
Nov 4, 2025

GeoCrossBench: Cross-Band Generalization for Remote Sensing

The data for remote sensing is constantly acquired, and new data comes from a growing number and diversity of satellites, while the vast majority of labeled data comes from older satellites. As remote-sensing foundation models for Earth observation scale up, the cost of (re-)training to support new satellites grows too, so cross-band generalization across sensors and satellites is increasingly important. We introduce GeoCrossBench, an extension of the popular GeoBench benchmark with a new evaluation protocol for cross-band generalization across sensors and satellites: it tests standard in-distribution performance with the same bands for train and test, generalization to inputs with no intersection between train and test; and generalization to test inputs containing a superset of the training bands. We develop χχViT, a self-supervised extension of the band-agnostic ChannelViT, as a supporting baseline for cross-band generalization. We evaluate a representative set of remote-sensing-specific and general-purpose vision models, characterize current performance, and identify directions for improvement through 11,900 H100 GPU-hours of experiments. When averaging dataset-specific metric scores, DOFA leads the in-distribution setting (61.30), frozen Panopticon leads the no-overlap setting (22.75), and ImageNet-pretrained ViT-B leads both the superset setting (56.19) and the overall average across settings (45.27). While top rankings in each setting are close, we clearly see that all models suffer significant performance losses when evaluated on unseen bands. We will publicly release the code and datasets to support the development of more future-proof remote sensing models with stronger cross-band generalization.
Jun 4, 2025

Intra- and Cross-Scale Feature Interaction Network for Referring Remote Sensing Image Segmentation

Referring remote sensing image segmentation (RRSIS) is a challenging multimodal task that requires accurate localization and pixel-level segmentation of targets specified by natural-language expressions. Existing RRSIS datasets remain limited in image resolution, scene diversity, and category coverage, which restricts the generalization ability and practical applicability of current methods. To advance this field, we introduce NWPU-Refer, the largest and most diverse bilingual RRSIS dataset to date. It contains 15,003 high-resolution images (1024--2048 px) collected from more than 30 countries across five continents, together with 49,745 annotated targets covering single-object, multi-object, and no-target scenarios. We further propose the Multi-scale Referring Segmentation Network (MRSNet), a framework tailored to the characteristics of remote sensing imagery. Specifically, MRSNet incorporates an Intra-scale Feature Interaction Module (IFIM) to refine fine-grained features within each encoder stage and a Hierarchical Feature Interaction Module (HFIM) to integrate cross-scale information while preserving spatial structure. Extensive experiments on NWPU-Refer and RRSIS-D demonstrate that MRSNet achieves strong and competitive performance across multiple metrics, especially under strict IoU thresholds and IoU-based evaluation, validating the effectiveness of the proposed dataset and model. The dataset and code are publicly available at https://github.com/CVer-Yang/NWPU-Refer.
May 21, 2025

The P3^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

We present the P3^3 dataset, a large-scale multimodal benchmark for building vectorization, constructed from aerial LiDAR point clouds, high-resolution aerial imagery, and vectorized 2D building outlines, collected across three continents. The dataset contains over 10 billion LiDAR points with decimeter-level accuracy and RGB images at a ground sampling distance of 25 centimeter. While many existing datasets primarily focus on the image modality, P3^3 offers a complementary perspective by also incorporating dense 3D information. We demonstrate that LiDAR point clouds serve as a robust modality for predicting building polygons, both in hybrid and end-to-end learning frameworks. Moreover, fusing aerial LiDAR and imagery further improves accuracy and geometric quality of predicted polygons. The P3^3 dataset is publicly available, along with code and pretrained weights of three state-of-the-art models for building polygon prediction at https://github.com/raphaelsulzer/PixelsPointsPolygons .
Mar 19, 2025

EarthScape: A Multimodal Benchmark for Geologic Mapping and Surface-Aware Learning under Geographic Domain Shift

Understanding natural Earth surface processes requires integrating topography, imagery, hydrology, and human infrastructure while remaining robust to geographic variation, yet few benchmarks support both challenges. We introduce EarthScape, a multimodal benchmark for geologic mapping and surface-aware learning under geographic domain shift. EarthScape combines aerial imagery, LiDAR-derived elevation, multi-scale terrain derivatives, and contextual vector data into 37 co-registered input channels per sample, together with pixel-level geologic masks. Its geographically separated regions share a common label space, enabling controlled evaluation of in-domain and cross-region generalization under realistic class imbalance and spatial complexity. Benchmark experiments show that engineered terrain representations transfer more robustly than raw elevation, while spatial support and multimodal terrain inputs strongly influence performance. Fusion effectiveness depends on backbone and evaluation domain, with more complex cross-attention providing no consistent advantage over simpler strategies. These representation trends extend across geospatial foundation models and semantic segmentation, while class-level results show that no single terrain feature or scale is optimal across geologic classes. Together, EarthScape provides a benchmark for studying representation design, multimodal fusion, and geographic robustness in surface-aware geospatial learning. Our source code is available at https://github.com/masseygeo/earthscape and the dataset at https://doi.org/10.13023/kgs.data.05.01.2025.