Organizations: University of Chinese Academy of Sciences, Beijing, China · Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, Beijing, China
Autonomous on-orbit servicing demands embodied agents that perceive through visual sensors, reason about 3D spatial situations, and execute multi-phase tasks over extended horizons. We present SpaceMind, a modular and self-evolving vision-language model (VLM) agent framework that decomposes knowledge, tools, and reasoning into three independently extensible dimensions: skill modules with dynamic routing, Model Context Protocol (MCP) tools with configurable profiles, and injectable reasoning-mode skills. An MCP-Redis interface layer enables the same codebase to operate across simulation and physical hardware without modification, and a Skill Self-Evolution mechanism distills operational experience into persistent skill files without model fine-tuning. We validate SpaceMind through 192 closed-loop runs across five satellites, three task types, and two environments, a UE5 simulation and a physical laboratory, deliberately including degraded conditions to stress-test robustness. Under nominal conditions all modes achieve 90--100% navigation success; under degradation, the Prospective mode uniquely succeeds in search-and-approach tasks where other modes fail. A self-evolution study shows that the agent recovers from failure in four of six groups from a single failed episode, including complete failure to 100% success and inspection scores improving from 12 to 59 out of 100. Real-world validation confirms zero-code-modification transfer to a physical robot with 100% rendezvous success. Code: https://github.com/wuaodi/SpaceMind
Figures & tables
Figure 1: Overview of SpaceMind. The framework operates as a VLM-based decision-control hub that perceives through visual sensors, reasons about the current situation, and issues motion and sensor-control commands. It supports three task types, namely rendezvous, search-and-approach, and inspection, across both high-fidelity UE5 simulation with 5 satellites and 174 runs and a physical laboratory with 2 satellites and 18 runs. Key capabilities include modular skill routing, plug-and-play tools, switchable reasoning modes, and experience-driven self-evolution.
Figure 2: SpaceMind architecture. The framework comprises four layers. The Skill Layer decomposes agent knowledge into modular skill definitions with LLM-based dynamic routing (Sec. 3.2 ). The VLM Decision Core hosts three switchable reasoning modes with hierarchical memory (Sec. 3.3 ). The MCP Tool Layer and Redis Message Bus provide an environment-agnostic interface that enables the same codebase to operate across UE5 simulation and a physical laboratory (Sec. 3.4 ). A side loop implements Skill Self-Evolution , which distills episode experience into learned skill files and injects them into subsequent runs (Sec. 3.5 ).
Campaign
Env
Runs
Tool Ablation
UE5
9
Reasoning Eval
UE5
135
Self-Evolution
UE5
30
Real-World
AGV Pro
18
Total
192
Table 1: Overview of experimental campaigns.
Task
Abbr.
Description
Metrics
Rendezvous
Rndz.
Target visible; approach to ∼ 2 m
Succ, Dist, Steps
Search-and-Approach
Search
Target not visible; find then approach to ∼ 2 m
Succ, Dist, Steps
Inspection
Insp.
Close-range 5-dim structured report
Score (0–100)
Table 2: Task definitions and evaluation metrics.
Cond.
Position
Exposure
Difficulty
C1
Nominal
Nominal
Baseline
C2
Far lateral offset
Underexposure
Degraded
C3
Reverse offset
Overexposure
Degraded
Table 3: Initial condition definitions for UE5 simulation.
Rndz.
Search
Nav. Pass
Insp.
Tool Profile
Dist (m)
Steps
Dist (m)
Steps
Rate
Score
Vision-Only
FAIL
FAIL
0/2
65
Hybrid-Nav
2.006
9
2.277
16
2/2
65
Hybrid-Nav-Code
2.048
13
2.279
15
2/2
50
Table 4: Tool profile ablation results over 9 runs.
Rndz. (pass / 5)
Search (pass / 5)
Nav. Total
Insp. (avg score)
Mode
C1
C2
C3
Sub
C1
C2
C3
Sub
Pass
Dist (m)
C1
C2
C3
Avg
Standard
5
5
3
13 (87%)
4
0
0
4 (27%)
17/30 (57%)
2.143
40.2
41.0
36.2
39.1
ReAct
5
1
2
8 (53%)
5
0
0
5 (33%)
13/30 (43%)
2.177
38.6
40.8
42.8
40.7
Prospective
5
3
2
10 (67%)
4
1
2
7 (47%)
17/30 (57%)
2.079
36.6
22.6
29.8
29.7
Table 5: Reasoning mode evaluation results across conditions, 135 runs in total. Navigation reports pass count per 5 runs for each task type; inspection reports average score per 5 runs. Best values in each column are bold .
Figure 3: Navigation pass rate across initial conditions. ReAct degrades sharply from C1 to C2, while Standard and Prospective maintain higher robustness.
Baseline
Best After Evol.
Satellite
Task
Cond
Result
Detail
Result
Detail
Verdict
CAPSTONE
Search
C1
OK
2.597 m
0/5
–
No benefit
CAPSTONE
Insp.
C2
50 pt
–
65 pt
+30%
Positive
New Horizons
Search
C1
FAIL
10 st
3/5
2.102 m
Positive
New Horizons
Rndz.
C3
FAIL
20 st
5/5
7 st
Strong +
Huygens
Rndz.
C2
OK
2.209 m
3/5
2.169 m
No benefit
Table 6: Skill self-evolution results over 30 runs across 6 groups. BL = baseline from the reasoning evaluation campaign.
Figure 4: Skill self-evolution learning curves for six groups, organized by task type in columns for Rndz., Search, and Insp. respectively. For navigation groups, bars show success and the dashed line shows step count; for inspection groups, the solid line shows score and the dashed horizontal line marks the baseline level. Green indicates clear positive benefit; gray indicates no significant improvement.
Rndz.
Search
Insp.
Satellite
Pass
Runs
Rate
Pass
Runs
Rate
Avg Score
Runs
CAPSTONE
3
3
100%
2
3
67%
41.0
3
Artemis
3
3
100%
2
3
67%
34.0
3
Total
6
6
100%
4
6
67%
36.8
6
Table 7: Real-world validation results over 18 runs.
Figure 5: Third-person view of a real-world search-and-approach episode on CAPSTONE over 14 steps. The agent searches for the target via yaw sweep in Steps 1–4, aligns and approaches in Steps 5–8, recovers from lateral drift at Step 12, and terminates at 0.8 m from the target surface at Step 14.
Figure 6: Qualitative examples from the agent’s sensor view. Top row: (a–b) Lab search, where the target is not visible at Step 1 but found and approached by Step 42; (c–d) inspection under C2, where the image is nearly black at Step 1 and clear after active exposure adjustment at Step 10. Bottom row: (e–f) Prospective search under C2, with an empty view at Step 1 and the target reached at Step 29; (g–h) self-evolution for New Horizons Rndz. C3, showing the baseline at Step 1 and successful approach at Step 8 after learned skill injection.
School of Information Science and Electronic Engineering & School of Integrated Circuits, Shanghai Jiao Tong University · Institute of Artificial Intelligence, China Telecom · 3Central South University +1