DreamSat-Bench: Development and Initial Testing of a Testbed for AI-Based Pose Estimation from 3D Reconstruction

Alex Posadas-Nava; August Berne; Giovanni Lavezzi; Kareena Shah; Alejandro Carrasco; Josiane Uwumukiza; Giacomo Battaglia; Paolo Panicucci

cs.ROSep 12, 2026

DreamSat-Bench: Development and Initial Testing of a Testbed for AI-Based Pose Estimation from 3D Reconstruction

Authors: Alex Posadas-Nava, August Berne, Giovanni Lavezzi, Kareena Shah, Alejandro Carrasco, Josiane Uwumukiza, Giacomo Battaglia, Paolo Panicucci, +3 more

Organizations: Massachusetts Institute of Technology · Politecnico di Milano · University of Illinois Urbana-Champaign · Universidad Polit´ecnica de Madrid

Abstract

This paper presents the development and initial testing of DreamSat-Bench, a modular rendezvous and proximity operation testbed designed to benchmark AI-based relative navigation techniques. By integrating a software- and hardware-in-the-loop robotic pipeline, the platform enables a seamless transition from digital simulation to physical reality. DreamSat-Bench unifies state-of-the-art robotic learning tools such as MuJoCo, Isaac Lab, and LeRobot into a single benchmarking platform, utilizing robotic arms to trace 3D trajectories. The platform allows for extensive customization of orbital environments and lighting to evaluate the simulation-to-reality gap. We demonstrate the testbed's utility by evaluating an end-to-end vision-based navigation pipeline that pairs DreamSat, a generative AI framework for single-view 3D reconstruction, with FoundationPose for zero-shot 6-DoF tracking of unseen spacecraft. Initial testing explores mission-representative orbital segments, including fixed-point station-keeping and fly-around characterization. Through a series of parametric studies, we quantify the impacts of reconstruction latency, mesh resolution, orbital range, and illumination geometry on pose estimation accuracy. Finally, a preliminary hardware-in-the-loop campaign qualitatively validates the physical deployment of the pipeline, identifying target symmetry and accumulated tracking drift as critical factors for robust navigation. DreamSat-Bench provides a rigorous framework for maturing autonomous navigation with unprepared space assets in the absence of prior geometric models.

Explore similar work

Jul 15, 2026cs.CV

DreamSat-Pose: Spacecraft Pose Estimation from Single-View 3D Reconstructions and Learned 2D-3D Feature Matching

6-DoF pose estimation is a critical task in autonomous rendezvous and proximity operations. In the case of an unknown target, this task becomes challenging as it shall be paired with the reconstruction of the target shape model. In this article, we propose a novel framework for single-shot shape and pose estimation of unknown spacecraft objects. Given a single image, we first reconstruct a 3D shape model of the target, then estimate the relative six-degrees-of-freedom pose by learning dense 2D-3D correspondences. The image features are extracted using a frozen DINOv3 vision transformer, while the geometric features are computed from the reconstructed point cloud using a trainable dynamic graph convolutional neural network encoder. A dual-stream transformer matcher refines descriptors through alternating self- and cross-attention, producing soft correspondences that are passed to a Perspective-

n

-Point solver for pose recovery. We evaluate the method on the SPE3R dataset and consider FoundationPose as a representative baseline for current state-of-the-art capabilities. Results show reliable pose estimates achieving 0.157 degrees mean pointing error using only a single image and reconstructed geometry, demonstrating strong generalization to unseen spacecraft.

Josiane Uwumukiza, Jocelyn Zhao, Giovanni Lavezzi +5

Undergraduate Student, Wellesley College, Wellesley, MA 02481, USA · Undergraduate Student, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Research Scientist, Massachusetts Institute of Technology, Cambridge, MA 02139, USA +5

Relative Pose Estimation Spacecraft Rendezvous and Proximity Operations Object Pose Estimation Spacecraft Pose Estimation

Ask PDF

Sep 22, 2026cs.CV

Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation

We present an improvement on previous spacecraft pose estimation architectures that results in the lowest published mean rotation errors we know of on the SPEED+ lightbox and sunlamp test sets for a known, non-cooperative spacecraft. By using a previously established heatmap-based pose estimation architecture and adapting a large self-supervised ViT foundation model (DINOv3) in place of the smaller convolutional and ViT encoders of previous work, we show that pose estimation accuracy improves from 300M to 840M parameters with no saturation yet observed. We also evaluate our 840M model on a Jetson Orin NX 16GB, measuring single-pass network inference at 133.8 ms per crop with a board draw of 32.0 W. These measurements demonstrate embedded inference feasibility on a processor family with orbital flight heritage. Our resulting model outperforms previous models across lightbox and sunlamp domains while training only on synthetic data. Our best model, using DINOv3 840M adapted with LoRA as the encoder (rank 64, three-seed ensemble with four-rotation test-time augmentation), results in

1.56^\circ

mean rotation error on sunlamp and

1.17^\circ

on lightbox, compared to the previous best mean rotation errors we know of on these test sets,

2.66^\circ

and

1.75^\circ

by EagerNet.

John Church, Vazghen Nikolian

Both authors contributed equally to this work; author order is alphabetical.

Vision Foundation Model Adaptation Object Pose Estimation Spacecraft Pose Estimation

Ask PDF

Sep 28, 2026cs.RO

MarsLab: A Martian Rover Simulator for Planetary Rover Autonomous Navigation

Future Mars missions will require rover autonomy that can operate across unstructured terrain, changing illumination, atmospheric dust, and limited communication. Simulation is a practical way to study these conditions before deployment, but existing Mars-relevant resources differ in scope, including mission-oriented simulators, fixed analog datasets, task-specific environments, and open robotics interfaces. In this context, we present MarsLab, an open-source, ROS2-native Mars rover simulator for autonomy and navigation algorithm development. MarsLab combines HiRISE-derived and procedural terrain with customizable rock, crater, solar-illumination, and atmospheric-dust settings, and runs a Perseverance-class rover model in NVIDIA Isaac Sim. The runtime publishes RGB, depth, RGB-D point clouds, LiDAR, IMU, wheel odometry, and Ground Truth (GT) pose data through standard ROS2 topics. We demonstrate MarsLab with Simultaneous Localization and Mapping (SLAM) benchmarks across sensing modalities, dust levels, scene geometry, and route length, and with Visual Place Recognition (VPR) benchmarks over repeated Mars Base traversals under illumination and dust changes. The results illustrate how controlled scene variation and shared GT trajectories can be used to compare trajectory-level estimation and image-level place recognition within the same simulator. Our Project Page: https://kimhoyun-robotair.github.io/MarsLab/.

Hoyun Kim, Beomsu Kim, Giseop Kim

Department of Robotics and Mechatronics Engineering, DGIST, Daegu, Republic of Korea

Simultaneous Localization and Mapping Off-Road Robot Navigation Space Robotics Robot Navigation

Ask PDF