cs.ROOct 16, 2026
SaveFoundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation
Organizations: Astroup Lab, Beijing, China · Harbin Institute of Technology, Beijing Research Institute · Peking University
Abstract
Space robots must interpret tasks and act despite unfamiliar dynamics, difficult sensing conditions, and limited opportunities for human intervention. Foundation models offer reusable perceptual and behavioral knowledge, but their value depends on how that knowledge reaches physical execution. This critical review follows space-specific requirements through perception, prediction, planning, and action to deployment and validation, emphasizing orbital manipulation and proximity operations. Terrestrial comparisons establish benefits of particular pretraining stages and model interfaces; their transfer to space dynamics and constrained onboard operation remains insufficiently tested. Event cameras offer timely motion observations across strong illumination contrasts, while spiking neural networks provide possible routes to sparse local computation. We examine how sensor degradation, state estimation and computing choices affect feedback and recovery. Higher action throughput need not yield fresher feedback; lower processor power can also coexist with higher energy per inference. We organize validation as an iterative sim-to-ground-to-space pathway, connecting physical tests with telemetry-based diagnosis and closed-loop revalidation. A satellite-capture example links observations to approach, contact and recovery, while a corresponding protocol separates generalization from adaptation with limited target-domain interaction. These comparisons identify what models already contribute and the physical evidence needed to support autonomous space operation.
Figures & tables
| Review | Established emphasis | Focus of this review |
|---|---|---|
| Robot foundation models [ 3 , 11 , 5 ] | Applications, interfaces, model use and construction | Reuse of pretrained components and evidence of transfer |
| Space robotics [ 1 , 2 ] | Autonomy, manipulation and physical requirements | Adaptation through observations, training and execution |
| AI-SRO [ 6 ] | Capability lifecycle: physics, interfaces, resources and validation | Gains, failures and tested conditions; timing and energy demands |
| Multimodal space robots [ 7 ] | Task planning and end-to-end control (abstract inspected) | Joint tests of task transfer and physical variation |
| Neuromorphic methods [ 8 , 12 , 9 , 10 , 13 ] | Sensing, representations, training and hardware | Roles of event sensors and spiking computation; deployment cost |
Table 1: Existing reviews and the questions pursued here. Their emphases overlap; the final column identifies our focus in relation to each body of literature.
Figure 1: How space conditions change the autonomy loop. Each pathway links a physical or operational constraint to its consequences for execution and to model or control responses. The pathways illustrate mechanisms discussed in this review; their importance depends on the mission and hardware [ 1 , 2 , 8 , 14 , 15 ] .
Figure 2: The same joint rotation with a fixed or freely moving base. Dashed and solid geometry denote initial and final poses. Support reactions hold the fixed base; the free base translates and counter-rotates. This planar example starts at rest in a local free-fall frame, neglecting external disturbances and active base control during a short maneuver. The response depends on mass distribution and geometry; a learned policy must account for the resulting end-effector motion [ 16 , 1 ] .
Figure 3: A functional architecture for embodied space autonomy. Operators propose and approve tasks; local runtime supervision monitors execution, and status returns to mission control. Fast physical feedback remains local. Dashed arrows show data collection, offline adaptation, and component release. Blue boxes indicate possible pretrained modules; foundation models may span several functions [ 33 , 34 , 35 , 36 , 37 , 30 ] .
Figure 4: From light to measurements and neural computation. Left: idealized single-pixel outputs share a nonquantitative time axis. Frame-bar height represents integrated light; differential events mark log-intensity changes; integrating pixels emit spikes at an accumulated-light threshold. The passive SPAD512 example records detection within each exposure; other SPAD readouts differ. Right: suitable encodings connect measurements to either network family; a leaky integrate-and-fire neuron illustrates state accumulation and spike emission. Traces are explanatory illustrations [ 8 , 13 , 51 , 10 ] .
| Sensor | Measurement | Model input and information limit |
|---|---|---|
| Conventional frame camera [ 8 , 51 ] | Light integrated over an exposure, then read as intensity values | Image tensors; motion during exposure can blur features; saturation loses intensity detail |
| Differential event camera (DVS) [ 8 ] | Thresholded log-intensity changes with location, timestamp and polarity | Event tensors/tokens or reconstructed images; unchanged intensity is not directly measured |
| Intensity-integrating spike camera [ 13 , 52 ] | Intensity accumulated to a threshold; each spike resets the integrator | Temporal spike encoding or reconstruction; intensity requires spike history |
| Passive SPAD camera (SPAD512) [ 51 , 53 ] | Binary value per pixel/exposure: no detection or at least one | Binary-frame accumulation estimates intensity; one exposure does not resolve photon multiplicity |
Table 2: Four sensing mechanisms, input representations and information limits. Sensor outputs do not determine the neural-network implementation.
| Output and examples | Source of physical behavior | Feedback responsibility | Orbital adaptation |
|---|---|---|---|
| Numerical targets: Astra [ 73 , 74 ] | Robot API and servo; model uses observed state | Local servo/program; observations return to model | Base-aware frames; coupled limits; observation age |
| Skill or program calls; typed choices: SayCan, Code as Policies, Jev [ 34 , 37 , 38 ] | Existing skills/controllers; external simulation in Jev skill tests | Executor closes local loop and returns status | Coupled-dynamics skills; checked preconditions and recovery |
| Spatial objectives or constraints: VoxPoser, OrbitTAMP [ 65 , 17 ] | Explicit robot/orbital dynamics in planner | VoxPoser replans; cited OrbitTAMP trajectories are open-loop | Target prediction; whole-body constraints; execution feedback |
| Learned action policies: OpenVLA, Octo, [ 33 , 39 , 40 ] | Robot trajectories and lower-level control | Policy updates; lower-level control between updates | Base/contact state; adapted training; constrained execution |
| Latent/video futures: V-JEPA 2-AC, Cosmos Policy [ 62 , 42 ] | Transitions learned from robot trajectories | Planner/policy selects; observations update prediction | Momentum/contact representation; calibration under changed dynamics |
Table 3: Physical assumptions and feedback responsibilities across model interfaces. Orbital adaptations are proposed extensions of the methods discussed in Sections 5 – 6 ; prediction can support any action interface.
Figure 5: A proposed capture example linking observations to execution and recovery. The phases draw on orbital approach/contact control and supervised execution [ 82 , 43 , 18 ] ; the combined foundation-model system is a design synthesis. Image semantics can guide target selection, while geometric and contact estimates determine admissible motion. A hold or retreat must remain feasible under the current dynamics and contact constraints.
| Evidence type | Example | Quantities and units | Exclusions or conditions |
|---|---|---|---|
| Runtime profiling | Jetson AGX Orin [ 85 ] | Peak memory (GB); chunk latency; 15/30/50 W and MAX modes | Power modes specify limits, not measured draw |
| Arithmetic estimate | Manipulation SNN [ 88 ] | Operations; assumed J/operation; estimated J/inference | No measured memory, routing, or idle cost |
| Power-derived inference energy | Akida V1 / Jetson Orin Nano [ 49 ] | E2F: 57.4 / 7.63 mJ/frame, from dynamic power / FPS | AKD1000 chip / Jetson board; different precision and operating points |
| Measured board power | Nahuku, preprint Table 2 [ 93 ] | Idle: 0.935–0.938 W; running-minus-idle: 7–12 mW | Preloaded events; excludes streaming I/O and preprocessing |
| Proposed task accounting | Matched-task test | Integrated energy (J); J/completed task; completion rate | Declare sensors, host, recovery and actuators included |
Table 4: Resource and energy evidence with different quantities and measurement scopes, not a cross-platform ranking. The final row proposes complete-task accounting. Akida comparison values use dynamic power divided by throughput and exclude idle energy.
| Medium | Task or mechanism | Calibration quantities | Residual difference | Sources |
|---|---|---|---|---|
| Digital simulation | Approach, sensing, contact models | Integrator step; sensor clocks; contact parameters | Model omissions and numerical error | [ 102 , 101 , 103 ] |
| Air bearings | Planar capture or attitude tests | Support inertia; drag; level/centre-of-mass offset | Constrained degrees of freedom | [ 1 , 108 ] |
| Gravity offloading | Spatial arm motion and contact | Residual load; tracking error; bandwidth | Suspension constraints and forces | [ 109 , 110 ] |
| Robotic HIL | Relative motion and physical contact | Delay; tracking; stiffness/damping | Rendered dynamics and workspace limits | [ 113 , 112 ] |
| Processor HIL | Real inference with simulated robot | Execution latency; I/O; numerical format | Simulated embodiment and environment | [ 114 ] |
| Environmental tests | Thermal load or radiation response | Temperature/power; particle exposure/dose; recovery time | Configuration-specific, not task assurance | [ 14 , 28 , 115 ] |
Table 5: Validation media, preserved mechanisms and residual differences. Calibration quantities are proposed reporting items for the chosen task; environmental tests address separate conditions.
Figure 6: Proposed sim-to-ground-to-space workflow and illustrative test mechanisms. Planar air bearings permit translation and yaw while retaining support; cable tension counteracts weight , with residual forces [ 1 , 109 , 110 ] . Robotic HIL couples simulated motion to physical sensing or contact; processor HIL runs inference hardware with a simulated robot [ 113 , 112 , 114 ] . Ground measurements calibrate models; telemetry informs diagnosis, update choice and closed-loop revalidation. Flight evidence remains task-specific.
Figure 7: Evidence for tested model–hardware configurations. Physical manipulation, free flight and rover driving are distinguished. Mars planning ran on Earth; Space-LLaVA’s separate GPT-4o demonstration is excluded. NR denotes unreported information. Cell-level sources and qualifications accompany the manuscript [ 45 , 67 , 17 , 68 , 69 , 36 , 114 , 88 ] .
| Study | Reuse and target update | Comparison conditions | Finding and attribution |
|---|---|---|---|
| Cosmos Policy [ 42 ] | Video-model initialization; robot policy training | LIBERO; pretrained versus random initialization; same gradient steps | 98.5 versus 94.6% success. Video-pretraining benefit at the tested training budget |
| SpaceMind [ 67 ] | Frozen VLM; skill instructions updated over five rounds per group | Six task-bound groups; no cross-task skill transfer | Four groups improve; instruction-based recovery within repeated tasks |
| OrbitTAMP [ 17 ] | Frozen LLM; prompt developed on 100 commands | Held-out instance/radio-style sets, 100 commands each; same ontology | Exact partial MissionConfig: rules 92/57%; Qwen3.5-9B 78/70%; Terra 98/98%. Parser contribution |
| Spike-driven VLA [ 90 ] | Pretrained vision/language encoders; language frozen; 80k policy updates | LIBERO tasks used in training; demonstration count NR | ANN/SNN: 96.3/92.4% success; estimated 179.7/20.9 mJ/chunk. Priors and capacity differ |
| OpenVLA-OFT [ 87 ] | OpenVLA checkpoint; LoRA on filtered trajectories from 500-demo suite pools | LIBERO execution after adaptation; fixed-query A100 timing | Adding chunking: 15.9 108.8 actions/s, 62.9 73.5 ms/query. Decoding benefit |
Table 6: Within-study comparisons of reuse and measured contributions. Conditions refer to selected experiments. NR: not reported.
More in this topic
Space Robotics
Space robotics develops robots and autonomy for exploring, assembling, and maintaining spacecraft and planetary environments.
See all 71 papers