Foundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation

Zizhe Wang; Jia Deng; Xin Lu; Zehao Jia; Yi Huang; Yao Zhu; Jianing Li

cs.ROOct 16, 2026

Foundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation

Authors: Zizhe Wang, Jia Deng, Xin Lu, Zehao Jia, Yi Huang, Yao Zhu, Jianing Li

Organizations: Astroup Lab, Beijing, China · Harbin Institute of Technology, Beijing Research Institute · Peking University

Abstract

Space robots must interpret tasks and act despite unfamiliar dynamics, difficult sensing conditions, and limited opportunities for human intervention. Foundation models offer reusable perceptual and behavioral knowledge, but their value depends on how that knowledge reaches physical execution. This critical review follows space-specific requirements through perception, prediction, planning, and action to deployment and validation, emphasizing orbital manipulation and proximity operations. Terrestrial comparisons establish benefits of particular pretraining stages and model interfaces; their transfer to space dynamics and constrained onboard operation remains insufficiently tested. Event cameras offer timely motion observations across strong illumination contrasts, while spiking neural networks provide possible routes to sparse local computation. We examine how sensor degradation, state estimation and computing choices affect feedback and recovery. Higher action throughput need not yield fresher feedback; lower processor power can also coexist with higher energy per inference. We organize validation as an iterative sim-to-ground-to-space pathway, connecting physical tests with telemetry-based diagnosis and closed-loop revalidation. A satellite-capture example links observations to approach, contact and recovery, while a corresponding protocol separates generalization from adaptation with limited target-domain interaction. These comparisons identify what models already contribute and the physical evidence needed to support autonomous space operation.

Figures & tables

Review	Established emphasis	Focus of this review
Robot foundation models [ 3 , 11 , 5 ]	Applications, interfaces, model use and construction	Reuse of pretrained components and evidence of transfer
Space robotics [ 1 , 2 ]	Autonomy, manipulation and physical requirements	Adaptation through observations, training and execution
AI-SRO [ 6 ]	Capability lifecycle: physics, interfaces, resources and validation	Gains, failures and tested conditions; timing and energy demands
Multimodal space robots [ 7 ]	Task planning and end-to-end control (abstract inspected)	Joint tests of task transfer and physical variation
Neuromorphic methods [ 8 , 12 , 9 , 10 , 13 ]	Sensing, representations, training and hardware	Roles of event sensors and spiking computation; deployment cost

Table 1: Existing reviews and the questions pursued here. Their emphases overlap; the final column identifies our focus in relation to each body of literature.

Figure 1: How space conditions change the autonomy loop. Each pathway links a physical or operational constraint to its consequences for execution and to model or control responses. The pathways illustrate mechanisms discussed in this review; their importance depends on the mission and hardware [ 1 , 2 , 8 , 14 , 15 ] .

Figure 2: The same joint rotation with a fixed or freely moving base. Dashed and solid geometry denote initial and final poses. Support reactions hold the fixed base; the free base translates and counter-rotates. This planar example starts at rest in a local free-fall frame, neglecting external disturbances and active base control during a short maneuver. The response depends on mass distribution and geometry; a learned policy must account for the resulting end-effector motion [ 16 , 1 ] .

Figure 3: A functional architecture for embodied space autonomy. Operators propose and approve tasks; local runtime supervision monitors execution, and status returns to mission control. Fast physical feedback remains local. Dashed arrows show data collection, offline adaptation, and component release. Blue boxes indicate possible pretrained modules; foundation models may span several functions [ 33 , 34 , 35 , 36 , 37 , 30 ] .

Figure 4: From light to measurements and neural computation. Left: idealized single-pixel outputs share a nonquantitative time axis. Frame-bar height represents integrated light; differential events mark log-intensity changes; integrating pixels emit spikes at an accumulated-light threshold. The passive SPAD512 example records detection within each exposure; other SPAD readouts differ. Right: suitable encodings connect measurements to either network family; a leaky integrate-and-fire neuron illustrates state accumulation and spike emission. Traces are explanatory illustrations [ 8 , 13 , 51 , 10 ] .

Sensor	Measurement	Model input and information limit
Conventional frame camera [ 8 , 51 ]	Light integrated over an exposure, then read as intensity values	Image tensors; motion during exposure can blur features; saturation loses intensity detail
Differential event camera (DVS) [ 8 ]	Thresholded log-intensity changes with location, timestamp and polarity	Event tensors/tokens or reconstructed images; unchanged intensity is not directly measured
Intensity-integrating spike camera [ 13 , 52 ]	Intensity accumulated to a threshold; each spike resets the integrator	Temporal spike encoding or reconstruction; intensity requires spike history
Passive SPAD camera (SPAD512) [ 51 , 53 ]	Binary value per pixel/exposure: no detection or at least one	Binary-frame accumulation estimates intensity; one exposure does not resolve photon multiplicity

Table 2: Four sensing mechanisms, input representations and information limits. Sensor outputs do not determine the neural-network implementation.

Output and examples	Source of physical behavior	Feedback responsibility	Orbital adaptation
Numerical targets: Astra [ 73 , 74 ]	Robot API and servo; model uses observed state	Local servo/program; observations return to model	Base-aware frames; coupled limits; observation age
Skill or program calls; typed choices: SayCan, Code as Policies, Jev [ 34 , 37 , 38 ]	Existing skills/controllers; external simulation in Jev skill tests	Executor closes local loop and returns status	Coupled-dynamics skills; checked preconditions and recovery
Spatial objectives or constraints: VoxPoser, OrbitTAMP [ 65 , 17 ]	Explicit robot/orbital dynamics in planner	VoxPoser replans; cited OrbitTAMP trajectories are open-loop	Target prediction; whole-body constraints; execution feedback
Learned action policies: OpenVLA, Octo, $\pi_{0}$ [ 33 , 39 , 40 ]	Robot trajectories and lower-level control	Policy updates; lower-level control between updates	Base/contact state; adapted training; constrained execution
Latent/video futures: V-JEPA 2-AC, Cosmos Policy [ 62 , 42 ]	Transitions learned from robot trajectories	Planner/policy selects; observations update prediction	Momentum/contact representation; calibration under changed dynamics

Table 3: Physical assumptions and feedback responsibilities across model interfaces. Orbital adaptations are proposed extensions of the methods discussed in Sections 5 – 6 ; prediction can support any action interface.

Figure 5: A proposed capture example linking observations to execution and recovery. The phases draw on orbital approach/contact control and supervised execution [ 82 , 43 , 18 ] ; the combined foundation-model system is a design synthesis. Image semantics can guide target selection, while geometric and contact estimates determine admissible motion. A hold or retreat must remain feasible under the current dynamics and contact constraints.

Evidence type	Example	Quantities and units	Exclusions or conditions
Runtime profiling	Jetson AGX Orin [ 85 ]	Peak memory (GB); chunk latency; 15/30/50 W and MAX modes	Power modes specify limits, not measured draw
Arithmetic estimate	Manipulation SNN [ 88 ]	Operations; assumed J/operation; estimated J/inference	No measured memory, routing, or idle cost
Power-derived inference energy	Akida V1 / Jetson Orin Nano [ 49 ]	E2F: 57.4 / 7.63 mJ/frame, from dynamic power / FPS	AKD1000 chip / Jetson board; different precision and operating points
Measured board power	Nahuku, preprint Table 2 [ 93 ]	Idle: 0.935–0.938 W; running-minus-idle: 7–12 mW	Preloaded events; excludes streaming I/O and preprocessing
Proposed task accounting	Matched-task test	Integrated energy (J); J/completed task; completion rate	Declare sensors, host, recovery and actuators included

Table 4: Resource and energy evidence with different quantities and measurement scopes, not a cross-platform ranking. The final row proposes complete-task accounting. Akida comparison values use dynamic power divided by throughput and exclude idle energy.

Medium	Task or mechanism	Calibration quantities	Residual difference	Sources
Digital simulation	Approach, sensing, contact models	Integrator step; sensor clocks; contact parameters	Model omissions and numerical error	[ 102 , 101 , 103 ]
Air bearings	Planar capture or attitude tests	Support inertia; drag; level/centre-of-mass offset	Constrained degrees of freedom	[ 1 , 108 ]
Gravity offloading	Spatial arm motion and contact	Residual load; tracking error; bandwidth	Suspension constraints and forces	[ 109 , 110 ]
Robotic HIL	Relative motion and physical contact	Delay; tracking; stiffness/damping	Rendered dynamics and workspace limits	[ 113 , 112 ]
Processor HIL	Real inference with simulated robot	Execution latency; I/O; numerical format	Simulated embodiment and environment	[ 114 ]
Environmental tests	Thermal load or radiation response	Temperature/power; particle exposure/dose; recovery time	Configuration-specific, not task assurance	[ 14 , 28 , 115 ]

Table 5: Validation media, preserved mechanisms and residual differences. Calibration quantities are proposed reporting items for the chosen task; environmental tests address separate conditions.

Figure 6: Proposed sim-to-ground-to-space workflow and illustrative test mechanisms. Planar air bearings permit

x/y

translation and yaw while retaining support; cable tension

T

counteracts weight

mg

, with residual forces [ 1 , 109 , 110 ] . Robotic HIL couples simulated motion to physical sensing or contact; processor HIL runs inference hardware with a simulated robot [ 113 , 112 , 114 ] . Ground measurements calibrate models; telemetry informs diagnosis, update choice and closed-loop revalidation. Flight evidence remains task-specific.

Figure 7: Evidence for tested model–hardware configurations. Physical manipulation, free flight and rover driving are distinguished. Mars planning ran on Earth; Space-LLaVA’s separate GPT-4o demonstration is excluded. NR denotes unreported information. Cell-level sources and qualifications accompany the manuscript [ 45 , 67 , 17 , 68 , 69 , 36 , 114 , 88 ] .

Study	Reuse and target update	Comparison conditions	Finding and attribution
Cosmos Policy [ 42 ]	Video-model initialization; robot policy training	LIBERO; pretrained versus random initialization; same gradient steps	98.5 versus 94.6% success. Video-pretraining benefit at the tested training budget
SpaceMind [ 67 ]	Frozen VLM; skill instructions updated over five rounds per group	Six task-bound groups; no cross-task skill transfer	Four groups improve; instruction-based recovery within repeated tasks
OrbitTAMP [ 17 ]	Frozen LLM; prompt developed on 100 commands	Held-out instance/radio-style sets, 100 commands each; same ontology	Exact partial MissionConfig: rules 92/57%; Qwen3.5-9B 78/70%; Terra 98/98%. Parser contribution
Spike-driven VLA [ 90 ]	Pretrained vision/language encoders; language frozen; 80k policy updates	LIBERO tasks used in training; demonstration count NR	ANN/SNN: 96.3/92.4% success; estimated 179.7/20.9 mJ/chunk. Priors and capacity differ
OpenVLA-OFT [ 87 ]	OpenVLA checkpoint; LoRA on filtered trajectories from 500-demo suite pools	LIBERO execution after adaptation; fixed-query A100 timing	Adding chunking: 15.9 $\rightarrow$ 108.8 actions/s, 62.9 $\rightarrow$ 73.5 ms/query. Decoding benefit

Table 6: Within-study comparisons of reuse and measured contributions. Conditions refer to selected experiments. NR: not reported.

Space Robotics

Space robotics develops robots and autonomy for exploring, assembling, and maintaining spacecraft and planetary environments.

See all 71 papers

Sep 15, 2026

Artificial Intelligence-Enabled Space Robot Operations: Technologies, Challenges and Prospects

Space robots are increasingly expected to perform long-duration, contact-rich, and multi-stage operations with limited human intervention. Recent advances in artificial intelligence (AI), robot learning, and embodied foundation models provide new opportunities to improve the autonomy and adaptability of such systems, but their transfer to space is constrained by scarce mission data, space-specific dynamics and sensing conditions, limited onboard resources, and stringent safety requirements. This article reviews artificial intelligence-enabled space robot operations (AI-SRO) from a capability-building perspective. We first summarize representative operational scenarios, autonomy trends, and space-specific constraints. We then establish a three-layer technical framework comprising capability foundations, capability formation, and capability deployment/evolution. Within this framework, we review simulation environments, datasets and benchmarks; task and environment understanding, state perception, decision-making and planning, and action execution; and onboard deployment, ground-to-space adaptation, continual learning, and capability transfer. Finally, we propose key research directions toward trustworthy simulation and data, open-world multimodal cognition, long-horizon safe decision-making, physically constrained policy learning, and space computing infrastructures.

Zeyuan Huang, Gang Chen, Zixuan Hao +7

School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing, China

Space Robotics Robotics Embodied AI Robot Learning

Ask PDF

Apr 15, 2026

SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing

Autonomous on-orbit servicing demands embodied agents that perceive through visual sensors, reason about 3D spatial situations, and execute multi-phase tasks over extended horizons. We present SpaceMind, a modular and self-evolving vision-language model (VLM) agent framework that decomposes knowledge, tools, and reasoning into three independently extensible dimensions: skill modules with dynamic routing, Model Context Protocol (MCP) tools with configurable profiles, and injectable reasoning-mode skills. An MCP-Redis interface layer enables the same codebase to operate across simulation and physical hardware without modification, and a Skill Self-Evolution mechanism distills operational experience into persistent skill files without model fine-tuning. We validate SpaceMind through 192 closed-loop runs across five satellites, three task types, and two environments, a UE5 simulation and a physical laboratory, deliberately including degraded conditions to stress-test robustness. Under nominal conditions all modes achieve 90--100% navigation success; under degradation, the Prospective mode uniquely succeeds in search-and-approach tasks where other modes fail. A self-evolution study shows that the agent recovers from failure in four of six groups from a single failed episode, including complete failure to 100% success and inspection scores improving from 12 to 59 out of 100. Real-world validation confirms zero-code-modification transfer to a physical robot with 100% rendezvous success. Code: https://github.com/wuaodi/SpaceMind

Aodi Wu, Haodong Han, Xubo Luo +3

University of Chinese Academy of Sciences, Beijing, China · Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China

Tool-Using Vision-Language Agents VLMs for Robotics Space Robotics LLM Agent Skill Learning

Ask PDF

Aug 24, 2026

Reward-Free Continual Adaptation for Resilient Space Robots

Space robots operate in extreme environments where hardware degradation can critically compromise traditional control strategies. While continual reinforcement learning offers a promising mechanism for online adaptation, it inherently requires access to a reward signal during deployment. However, precise reward computation in space is often infeasible due to the lack of external tracking systems and the overall complexity of the environment. To address the challenge of unobservable rewards, we introduce a reward-free continual learning framework that leverages latent-state world models. By pre-training a model-based agent across diverse simulations, the world model learns a robust predictor of the reward structure within its latent space. Upon deployment to an environment with severe hardware degradation, we freeze the observation encoder and reward predictor to update only the transition dynamics of the world model through unsupervised rollouts. By training the policy entirely on imagined trajectories generated by this updated world model, the agent adapts to altered dynamics without receiving new rewards. We demonstrate our approach across simulated planetary traversal, orbital navigation, and precision assembly tasks subjected to severe morphological failures.

Andrej Orsula, Miguel Olivares-Mendez, Carol Martinez

University of Luxembourg

Reinforcement Learning Continual Robot Learning Reward-Free RL Space Robotics Model-Based RL Robot Dynamics Learning

Ask PDF

Abstract

Space Robotics

Explore similar work

Artificial Intelligence-Enabled Space Robot Operations: Technologies, Challenges and Prospects

SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing

Reward-Free Continual Adaptation for Resilient Space Robots