Markerless Multi-Modal Autonomous Robotic Inspection of Large Space Structures

On-orbit inspection imagery is crucial as it enables characterization of non-cooperative resident space objects, providing the geometry and structural condition essential for active debris removal and on-orbit servicing mission planning. However, most existing neural implicit surface reconstruction methods have been confined to synthetic or hardware-in-the-loop data with known camera poses and controlled illumination. In this work, we present a pipeline for neural implicit surface reconstruction of non-cooperative space objects from monocular inspection imagery. We demonstrate it on publicly released ISS inspection footage from the STS-119 mission and publicly released on-orbit inspection footage of an H-IIA rocket upper stage. We find that segmentation-based background removal is essential for successful camera pose estimation from real on-orbit footage, where background variation between frames caused direct processing to fail entirely. We further incorporate photometric correction of per-frame exposure variations and analyze its behavior across datasets, finding that performance in shadowed regions varies with the illumination characteristics of the input footage.

Robotic operations in space are challenging due to the harsh environment and the high cost of failure. Fiducial markers provide visual references that aid autonomous rendezvous, proximity operations, and docking for space robots. However, existing fiducial markers are mostly single-scale and largely designed for terrestrial robotics. Such markers leave the camera's field of view at close range, precisely during the proximity and docking phases where reliable tracking is most critical. This paper presents AstraTag, a fiducial marker designed for autonomous on-orbit robotic operations. The marker template is based on a square Spidron pattern whose recursive, self-similar structure enables detection across multiple spatial scales. Marker identification uses a 48-bit signature derived from triangular sub-regions of the template and encoded with a Generalised Reed-Solomon (GRS) code. The detection pipeline performs contour-based quadrilateral localisation, perspective normalisation, and signature matching against a pre-computed dictionary. To handle markers affixed to curved spacecraft surfaces, it incorporates a Thin-Plate Spline (TPS) re-warp fallback that exploits the marker's internal rectangular borders as additional geometric correspondences. We benchmark AstraTag against three-layer Fractal ArUco and AprilTag on spacecraft mockups with flat and curved surfaces. On curved surfaces, AstraTag achieves a higher detection rate than both baselines, offering a robust recursive-marker option for space robotics.

A proposed method for the control of groups of inspection spacecraft is Multi-Agent Reinforcement Learning (MARL). While MARL has already been employed for this purpose in previous work, the reward functions used focus on reaching a finite set of predetermined inspection points around the target. In this work, we study and develop a generalized reward function for the MARL inspection task informed by the analysis of 3D reconstructions of inspected objects in orbit. Because the reward function is generalized such that any number of images at arbitrary locations may evaluated, we also allow trained agents to have complete control over when images are collected. With this approach, we gather insights into best practices for not only the specific MARL inspection task, but also gain key takeaways informative to the broader inspection task outside of a MARL context.