SpaceSense-Bench: A Large-Scale Multi-Modal Benchmark for Spacecraft Perception and Pose Estimation

Aodi Wu; Jianhong Zuo; Zeyuan Zhao; Xubo Luo; Ruisuo Wang; Xue Wan

cs.CVMar 10, 2026

SpaceSense-Bench: A Large-Scale Multi-Modal Benchmark for Spacecraft Perception and Pose Estimation

Authors: Aodi Wu, Jianhong Zuo, Zeyuan Zhao, Xubo Luo, Ruisuo Wang, Xue Wan

Organizations: University of Chinese Academy of Sciences, Beijing, China · Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China · Nanjing University of Aeronautics and Astronautics, Nanjing, China

Abstract

Autonomous space operations such as on-orbit servicing and active debris removal demand robust part-level semantic understanding and precise relative navigation of target spacecraft, yet collecting large-scale real data in orbit remains impractical due to cost and access constraints. Existing synthetic datasets, moreover, suffer from limited target diversity, single-modality sensing, and incomplete ground-truth annotations. We present SpaceSense-Bench, a large-scale multi-modal benchmark for spacecraft perception encompassing 136~~satellite models with approximately 70~~GB of data. Each frame provides time-synchronized 1024 $\times$ 1024 RGB images, millimeter-precision depth maps, and 256-beam LiDAR point clouds, together with dense 7-class part-level semantic labels at both the pixel and point level as well as accurate 6-DoF pose ground truth. The dataset is generated through a high-fidelity space simulation built in Unreal Engine~5 and a fully automated pipeline covering data acquisition, multi-stage quality control, and conversion to mainstream formats. We benchmark five representative tasks (object detection, 2D semantic segmentation, RGB--LiDAR fusion-based 3D point cloud segmentation, monocular depth estimation, and orientation estimation) and identify two key findings: (i)~perceiving small-scale components (e.g., thrusters and omni-antennas) and generalizing to entirely unseen spacecraft in a zero-shot setting remain critical bottlenecks for current methods, and (ii)~scaling up the number of training satellites yields substantial performance gains on novel targets, underscoring the value of large-scale, diverse datasets for space perception research. The dataset, code, and toolkit are publicly available at https://github.com/wuaodi/SpaceSense-Bench.

Figures & tables

Dataset	Sat	Img	RGB	Depth	Part Seg	Pose
SPEED [ 7 ]	1	15k	✓			✓
SPEED+ [ 8 ]	1	70k	✓			✓
URSO [ 9 ]	2	5k	✓			✓
Hoang [ 10 ]	–	3k	✓		✓(3)
UESD [ 11 ]	33	10k	✓		✓(5)
SPARK [ 12 ]	10	150k	✓	✓		✓

TABLE I: Comparison with existing spacecraft perception datasets. Sat: number of spacecraft models. Img: number of frames (SpaceSense-Bench reports sparse / dense). Part Seg classes in parentheses.

Fig. 2: Overall data collection pipeline of SpaceSense-Bench. The pipeline consists of four stages: (1) 3D asset library construction and part decomposition, (2) high-fidelity space scene setup, (3) adaptive trajectory planning and multi-sensor synchronized capture, and (4) automated ground truth generation, quality control, and mainstream format export.

Fig. 3: Multi-modal data and ground truth examples from SpaceSense-Bench. Each column shows one satellite. From top to bottom: RGB image with 6D pose axes overlay, seven-class semantic segmentation mask, LiDAR point cloud with per-point semantic labels, and colorized depth map.

Fig. 4: Dataset statistics. (a) Per-class foreground pixel ratio showing a long-tail distribution. (b) LiDAR point count versus target distance. (c) Satellite maximum dimension distribution (log scale). (d) Frame count per trajectory type for 22 approach and 5 orbit trajectories.

(a) 2D Semantic Segmentation — per-class IoU (%)
Model	Backbone	aAcc	mIoU	body	solar	dish	omni	payload	thruster	adapter
FCN [ 24 ]	ResNet-50	99.16	41.30	68.1	82.9	30.7	1.0	19.0	12.8	16.1
DeepLabV3+ [ 17 ]	ResNet-50	99.19	43.70	68.1	83.2	38.2	1.0	21.5	17.7	20.2
SegFormer [ 19 ]	MiT-B3	99.27	45.14	71.2	87.6	39.7	2.9	22.6	20.1	17.3
Mask2Former [ 20 ]	Swin-B	99.28	45.63	71.4	88.6	26.3	1.9	19.1	24.6	33.3
(b) Object Detection (YOLO26 [ 25 ] ) — per-class [email protected] (%)

TABLE II: Benchmark results on the 14-satellite zero-shot test set. Best results in bold . Task-specific metrics are detailed in each sub-table header.

Fig. 5: Effect of training set size on zero-shot generalization. mIoU and mAcc of PMFNet on the 14-satellite test set as the number of training satellites increases from

\sim

9 to 117.

Explore similar work

Jun 13, 2026

Segmentation-based Detection for Efficient Multi-Task Spacecraft Perception

Vision-based perception is fundamental to Space Situational Awareness and autonomous on-orbit operations such as rendezvous, docking, servicing, and navigation. However, progress in this area is limited by the scarcity of annotated space imagery and by challenging visual-domain characteristics including severe illumination changes, low signal-to-noise ratio, and high contrast. We address Stream 1 of the SPARK 2026 Challenge, which requires a single model for spacecraft classification, detection, and fine-grained component segmentation across multiple target types. We propose a compact architecture that integrates a MobileNetV3 encoder with a U-Net-style decoder, combining computational efficiency with accurate dense prediction. Detection is derived analytically from the union of predicted component masks, avoiding a separate bounding-box regression head in the single-spacecraft setting. Our method achieved an overall leaderboard score of 0.9482, with task-specific scores of 1.0000 in classification, 0.9788 in detection, and 0.8917 in segmentation. The proposed approach ranked second overall in the SPARK 2026 Challenge, demonstrating that lightweight encoder-decoder architectures can deliver strong multi-task performance for practical onboard space vision systems.

Sivaperuman Muniyasamy, Surendar Devasundaram

University of Arizona, Tucson, AZ, USA

Image Segmentation Multi-Task Learning Object Detection Robotic Perception Semantic Segmentation

Ask PDF

Sep 8, 2026

SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science

Lunar orbital missions, such as Lunar Reconnaissance Orbiter, Kaguya/SELENE, Gravity Recovery and Interior Laboratory, and Lunar Prospector, among others, provide rich multi-instrument observations, but their heterogeneity in sampling, projection, and conventions limits reproducible machine learning (ML). We introduce SomBench, a unified, spatially-aligned, ML-ready lunar dataset aggregating 30+ co-registered layers from ten instruments across four missions, spanning 1 meter to 20 kilometer/pixel and covering 82 degree latitude in 90 Lunar Transverse Mercator zones with two polar stereographic caps. An image-anchored tiling pipeline yields pretraining-ready multimodal tile views with leakage-safe splits, distributed as netCDF with Parquet catalogs. An application benchmark suite spans impact processes, volcanic history, and polar volatiles. Baseline experiments with ResNet-50 and SwinV2-B models confirm that each benchmark task is learnable from the released inputs, establishing reference points for future model development.

Himanshu Patil, Gabby Nyirjesy, Rachel A. Slank +19

The University of Alabama in Huntsville, Huntsville, AL, USA · IBM Research, Yorktown Heights, NY, USA · Science and Technology Institute, Universities Space Research Association (USRA), Huntsville, AL, USA +7

Benchmark Design Scientific ML Remote Sensing

Ask PDF

Mar 15, 2026

GroundSet: A Cadastral-Grounded Dataset for Spatial Understanding with Vector Data

Precise spatial understanding in Earth Observation is essential for translating raw aerial imagery into actionable insights for critical applications like urban planning, environmental monitoring and disaster management. However, Multimodal Large Language Models exhibit critical deficiencies in fine-grained spatial understanding within Remote Sensing, primarily due to a reliance on limited or repurposed legacy datasets. To bridge this gap, we introduce a large-scale dataset grounded in verifiable cadastral vector data, comprising 3.8 million annotated objects across 510k high-resolution images with 135 granular semantic categories. We validate this resource through a comprehensive instruction-tuning benchmark spanning seven spatial reasoning tasks. Our evaluation establishes a robust baseline using a standard LLaVA architecture. We show that while current RS-specialized and commercial models (e.g., Gemini) struggle in zero-shot settings, high-fidelity supervision effectively bridges this gap, enabling standard architectures to master fine-grained spatial grounding without complex architectural modifications.

Roger Ferrod, Maël Lecene, Krishna Sapkota +4

Université Paris Cité, LIPADE, F-75006 Paris, France · Google Research

VLM Evaluation Remote Sensing Image Understanding Visual Spatial Reasoning Geospatial Reasoning Vision-Language Grounding Remote Sensing Visual Grounding

Ask PDF