Space-X: Self-Evolving Spatial Agents Weaving Clues into Evidence Chains
Organizations: New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences (CASIA) · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences
Abstract
Spatial reasoning connects visual observations, geometric estimates, and linguistic decisions into a coherent derivation. Yet tool-augmented agents often acquire evidence without clear targets, combine outputs under inconsistent object or reference-frame semantics, and misjudge evidence sufficiency. Thus, more tools or calls do not reliably improve reasoning. We introduce Space-X, a spatial-reasoning framework that self-evolves toward complete, continuous evidence-chain construction. An Execution Agent acquires, associates, and judges evidence per problem using role-specific memory. A Control Agent diagnoses evidence-chain breaks from execution traces and jointly revises tools, interfaces, skills, memory organization, and decision rules. This loop co-evolves capability and evidence use: evidence needs guide capability development; usable results guide actions. Across multiple benchmarks, Space-X yields gains with four base models. Relative to baselines, GPT-6-sol improves answer accuracy on ViewSpatial and MMSI by 50.39% and 23.07%, respectively; Gemini-3.8-Flash gains 47.50% and 25.22% on these benchmarks. This provides a unified route from fragmented spatial clues to traceable, derivable answers.
Figures & tables
| Method | ViewSpatial | SPAR | VSI [-1pt](Only 10 frames) | |||||
|---|---|---|---|---|---|---|---|---|
| Overall | Cam. rel. dir. | Cam. obj. orient. | Pers. obj. orient. | Pers. rel. dir. | Pers. scene sim. | |||
| Base models | ||||||||
| Qwen3-VL-8B-Instruct [ 5 ] | 42.20 | 54.20 | 29.72 | 47.29 | 40.26 | 31.13 | 39.62 | – |
| Qwen2.5-VL-7B-Instruct [ 5 ] | 36.85 | 46.76 | 31.22 | 39.96 | 32.42 | 26.61 | 33.83 | – |
| GPT-6-sol | 53.19 | 59.05 | 37.75 | 44.48 | 76.25 | 47.96 | 57.15 | 48.40 |
| GPT-6-astra | 59.89 | 62.94 | 37.85 | 44.88 | 89.07 | 66.15 | 68.96 | 61.08 |
| Method | MMSI | BLINK | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Overall | C–C | O–O | R–R | C–O | O–R | C–R | Meas. | Appr. | Cam. motion | Obj. motion | MSR | ||
| Base models | |||||||||||||
| Qwen3-VL-8B-Instruct [ 5 ] | 31.10 | 27.96 | 37.23 | 32.10 | 31.40 | 35.29 | 38.55 | 37.50 | 15.15 | 27.03 | 28.95 | 29.80 | 66.70 |
| Qwen2.5-VL-7B-Instruct [ 5 ] | 26.80 | 27.96 | 26.60 | 19.75 | 32.56 | 38.82 | 28.92 | 23.44 | 21.21 | 20.27 | 30.26 | 24.75 | 55.87 |
| GPT-6-sol | 53.90 | 62.37 | 39.36 | 53.09 | 38.37 | 44.71 | 67.47 | 68.75 | 50.00 | 72.97 | 44.74 | 55.05 | 82.09 |
| GPT-6-astra | 71.60 | 73.12 | 69.15 | 75.31 | 61.63 | 65.88 | 72.29 | 73.44 | 69.70 | 83.78 | 64.47 | 75.25 | 89.49 |
| Setting | Tool evol. | Rule evol. | P/G | Ops | VS | SPAR | MMSI |
|---|---|---|---|---|---|---|---|
| Initial | No | No | 7 | 4 | 56.86 | 59.28 | 54.19 |
| Tools only | Yes | No | 39 | 4 | 68.15 | 63.96 | 60.19 |
| Rules only | No | Yes | 7 | 9 | 61.52 | 61.02 | 57.80 |
| Joint | Yes | Yes | 55 | 12 | 79.99 | 70.39 | 66.33 |
| Setting | VS | SPAR | MMSI | Mismatch | Delivery |
|---|---|---|---|---|---|
| Flat records | 67.25 | 63.03 | 59.91 | 21.70 | 71.06 |
| Association | 73.26 | 66.96 | 63.73 | 5.64 | 82.20 |
| Association + memory | 79.99 | 70.39 | 66.33 | 1.05 | 98.24 |
| Setting | VS | SPAR | MMSI | Repair | Damage |
|---|---|---|---|---|---|
| Initial answer | 74.46 | 65.27 | 59.86 | – | – |
| Standard review | 75.95 | 66.82 | 62.45 | 11.39 | 2.38 |
| Claim judgment | 79.99 | 70.39 | 66.33 | 22.19 | 1.50 |
| Model | Setting | VS | SPAR | MMSI |
|---|---|---|---|---|
| GPT-6-astra | Transferred | 85.26 | 73.82 | 77.70 |
| X-SFT | 87.24 | 76.53 | 79.17 | |
| Gemini-3.8-Flash | Transferred | 85.22 | 75.79 | 65.93 |
| X-SFT | 87.93 | 77.32 | 68.09 | |
| Gemini-4-Argon | Transferred | 87.61 | 76.45 | 78.45 |
| X-SFT | 89.33 | 77.96 | 80.48 |
| Harness | VS | SPAR | MMSI | Calls/question |
|---|---|---|---|---|
| DSH CLI | 55.25 | 58.06 | 55.01 | 0 |
| DSH CLI + spatial tools | 68.46 | 63.56 | 61.28 | 6.72 |
| Codex CLI | 56.91 | 58.96 | 56.38 | 0 |
| Codex CLI + spatial tools | 70.34 | 65.14 | 60.56 | 4.64 |
| Space-X | 79.99 | 70.39 | 66.33 | 3.21 |
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
| Evidence link | Trace evidence | Structural change | Comparison check |
|---|---|---|---|
| Task premises and source selection | Target, position, and facing acquisition | Joint-role retrieval, semantic selection, coverage completion | Required roles reach judgment |
| Object regions and geometric operands | Originals, regions, pixel transforms, coordinates | Same-source region–geometry binding and requested-frame readout | Endpoints match task objects |
| Tool outputs and API context | Result IDs, displayed content, actual attachments | Complete readback and requested-source restoration | Cited support is present in model input |
| Current observations and later reasoning | Notes, unresolved relations, memory retrieval | Observation relay and instance/event ledgers | New observations extend earlier premises |
| Candidate answers and source support | Claims, disagreements, verification traces | Claim judgment, source reconciliation, closure reserve | Final decisions have explicit support |
| Capability group | Representative operations or outputs | Semantics preserved in evidence |
|---|---|---|
| Source access and local observation | Original-image inspection, crops, attachment selection | Source identity, crop region, timestamp, pixel mapping |
| Object and body perception | Candidates, SAM3 masks, body axes and heading cues | Target role, candidate identity, visible region, estimate status |
| Shared geometry and depth | VGGT, DA3, and MapAnything geometric outputs | Input views, cameras, frame, scale, confidence information |
| Quantity and frame readouts | Source-camera depth, range, surface distance, observer relation | Operands, endpoints, units, requested frame |
| Multiview and temporal evidence | Semantic selection, coverage completion, instance/event ledgers | View identity, instance links, time and spatial scope |
| General operations and state | Memory queries, observation records, plan updates, readback | Evidence IDs, source dependencies, current premises, downstream use |
| Memory role | Stored content | Update or retrieval stage | Evidence-chain function |
|---|---|---|---|
| Working | Constraints, proposals, evidence needs, unresolved relations | Planning, association, answer formation | Track goals and premise progress |
| Observation | Originals, frames, crops, regions, time, source notes | Source registration, reinspection, local observation | Retain inspectable visual support |
| Spatial | Identity, geometry, coordinates, quantities, relations | Perception, spatial calculation, cross-view association | Unify spatial interpretation |
| Episodic | Actions, events, proposals, verification history | Each operation and stage transition | Retain execution and improvement signals |
| Procedural | Skills, applicability, interpretation, checks | Task-time retrieval; cross-iteration updates | Reuse operating experience |
| Record level | Principal fields | Upstream and downstream connections |
|---|---|---|
| Original observation | Evidence ID, artifact ID, source time, view index | Original support for crops, localization, and reinspection |
| Derived region | Parent source, candidate identity, region, pixel transform | Maps local results back to originals |
| Spatial result | Input evidence, frame, units, endpoints, status | Binds readouts to operands and quantities |
| Relation claim | Roles, relation, citations, unresolved premises | Connects tool results to candidate answers |
| Stage record | Actions, retrieval, delivery, judgment | Turns task execution into improvement feedback |
| Category | Principal changes | Evidence-chain role | Count | Share |
|---|---|---|---|---|
| T: Perception and geometry | Objects, bodies, depth, cameras, scale, retrieval, coverage | Acquire task-relevant spatial information | 194 | 33.74% |
| S: Semantics and skills | Roles, frames, answer contracts, applicability, selection | Define the premises each output supports | 108 | 18.78% |
| M: Memory and continuity | Notes, instance/event ledgers, retrieval, readback, compression | Preserve evidence state across steps | 39 | 6.78% |
| E: Association and delivery | Source identities, interfaces, binding, visual/numerical delivery | Make existing results usable downstream | 116 | 20.17% |
| C: Judgment and integration | Claim checks, closure reserves, resources, route integration | Complete grounded decisions | 118 | 20.52% |
| Code | Component | Count |
|---|---|---|
| T1 | Object/body/instance/attribute perception | 72 |
| T2 | Depth, scale, distance, extent | 28 |
| T3 | Camera frames and multiview geometry | 64 |
| T4 | Image/video retrieval and coverage | 30 |
| S1 | Object roles and body/frame rules | 26 |
| S2 | Quantities, answer contracts, scope | 48 |
| Stage | Sequence | T | S | M | E | C | Total |
|---|---|---|---|---|---|---|---|
| Build & Ground | 1–130 | 36 | 23 | 13 | 22 | 36 | 130 |
| Specialize Evidence | 131–276 | 50 | 35 | 12 | 26 | 23 | 146 |
| Connect & Complete | 277–445 | 62 | 28 | 9 | 35 | 35 | 169 |
| Integrate & Refine | 446–575 | 46 | 22 | 5 | 33 | 24 | 130 |
| Node | Capability developed | Connected components | Position |
|---|---|---|---|
| N01 | Ground the reasoning task | Typed roles; quantity semantics; claim ledger; skills; compression | 9 |
| N02 | Route by evidence needs | Task contracts; camera/body semantics; applicability; routing | 57 |
| N03 | Bind entities and real camera frames | Localization; body heading; camera projection; geometry semantics | 130 |
| N04 | Acquire question-relevant sources | Relevant frames; coverage; instance/event ledgers; packets; budgets | 234 |
| N05 | Carry observations across steps | Observation relay; original delivery; source notes; closure | 341 |
| N06 | Ground quantities and cross-view relations | Same-source geometry; surface/center semantics; roles; camera readout | 420 |
| Order | Type | Structural change | Role in the evidence chain |
|---|---|---|---|
| 01 | C | Typed task plans and tool dependencies | Represent roles, premises, and executable dependencies. |
| 02 | E | Tool outputs and crop delivery | Place generated visual evidence in the next request. |
| 03 | T | Shared geometry, scale anchors, observer transforms | Express quantities in common coordinates and question scale. |
| 04 | C | Claim ledger and disagreement reconciliation | Record inspectable claims and their sources. |
| 05 | S | Projection skills and evidence compression | Distinguish image projection, lateral position, and centers. |
| 06 | C | Task-conditioned integration | Select workflows by output and spatial semantics. |
| Order | Type | Structural change | Role in the evidence chain |
|---|---|---|---|
| 21 | T | Three-dimensional body orientation | Add body-pose and target-relation evidence. |
| 22 | M | Long-video temporal packets | Keep events associated with their sources. |
| 23 | E | Candidate identity and category checks | Preserve alternative objects during source inspection. |
| 24 | T | Segmented semantic selection and anchors | Balance relevance and coverage in a shared source bank. |
| 25 | C | Acquisition and budget integration | Combine semantic selection with batched inspection. |
| 26 | M | Conditional acquisition-history relay | Carry identity, scope, and source questions forward. |
| Sample | ViewSpatial | SPAR |
|---|---|---|
| 1 | 53.19 | 57.15 |
| 2 | 54.58 | 57.76 |
| 3 | 59.14 | 61.64 |
| 4 | 65.62 | 61.96 |
| 5 | 68.09 | 63.14 |
| 6 | 68.82 | 64.01 |
| Asset | Count | Organization |
|---|---|---|
| Perception/geometry tools | 55 | Callable object, body, geometry, frame, and spatiotemporal interfaces |
| General operations | 12 | Source access, memory queries and updates, execution support |
| Memory roles | 5 | Working, observation, spatial, episodic, procedural |
| Historical skill snapshots | 49 | Structured procedure combinations across iterations and paths |
| Distinct historical skill IDs | 76 | Reusable experience identified across the archive |
| Setting | Tool evol. | Rule evol. | P/G | Ops | ViewSpatial | SPAR | MMSI |
|---|---|---|---|---|---|---|---|
| Initial | No | No | 7 | 4 | 56.86 | 59.28 | 54.19 |
| Tools only | Yes | No | 39 | 4 | 68.15 | 63.96 | 60.19 |
| Rules only | No | Yes | 7 | 9 | 61.52 | 61.02 | 57.80 |
| Joint | Yes | Yes | 55 | 12 | 79.99 | 70.39 | 66.33 |
| Setting | VS | SPAR | MMSI | Mismatch | Delivery |
|---|---|---|---|---|---|
| Flat records | 67.25 | 63.03 | 59.91 | 21.70 | 71.06 |
| Association | 73.26 | 66.96 | 63.73 | 5.64 | 82.20 |
| Association + memory | 79.99 | 70.39 | 66.33 | 1.05 | 98.24 |
| Setting | VS | SPAR | MMSI | Repair | Damage |
|---|---|---|---|---|---|
| Initial answer | 74.46 | 65.27 | 59.86 | – | – |
| Standard review | 75.95 | 66.82 | 62.45 | 6.73 | 2.38 |
| Claim judgment | 79.99 | 70.39 | 66.33 | 13.24 | 1.50 |
| Target model | Setting | VS | SPAR | MMSI |
|---|---|---|---|---|
| GPT-6-astra | Transferred | 85.26 | 73.82 | 77.70 |
| GPT-6-astra | X-SFT | 87.24 | 76.53 | 79.17 |
| Gemini-3.8-Flash | Transferred | 85.22 | 75.79 | 65.93 |
| Gemini-3.8-Flash | X-SFT | 87.93 | 77.32 | 68.09 |
| Gemini-4-Argon | Transferred | 87.61 | 76.45 | 78.45 |
| Gemini-4-Argon | X-SFT | 89.33 | 77.96 | 80.48 |
| Harness | VS | SPAR | MMSI | Calls |
|---|---|---|---|---|
| DSH CLI | 55.25 | 58.06 | 55.01 | 0 |
| DSH + shared tools | 68.46 | 63.56 | 61.28 | 6.72 |
| Codex CLI | 56.91 | 58.96 | 56.38 | 0 |
| Codex + shared tools | 70.34 | 65.14 | 60.56 | 4.64 |
| Space-X | 79.99 | 70.39 | 66.33 | 3.21 |
| Base model | Setting | ViewSpatial | SPAR | MMSI | BLINK |
|---|---|---|---|---|---|
| GPT-6-sol | Model alone | 53.19 | 57.15 | 53.90 | 82.09 |
| GPT-6-sol | Space-X | 79.99 | 70.39 | 66.33 | 86.41 |
| GPT-6-astra | Model alone | 59.89 | 68.96 | 71.60 | 89.49 |
| GPT-6-astra | Space-X | 85.26 | 73.82 | 77.70 | 91.31 |
| Gemini-3.8-Flash | Model alone | 57.78 | 63.00 | 52.65 | 85.89 |
| Gemini-3.8-Flash | Space-X | 85.22 | 75.79 | 65.93 | 95.80 |
| Base model | Setting | Overall | Low | Middle | High |
|---|---|---|---|---|---|
| GPT-6-sol | Model alone | 57.15 | 49.49 | 45.90 | 67.71 |
| GPT-6-sol | Space-X | 70.39 | 57.81 | 78.58 | 78.84 |
| GPT-6-astra | Model alone | 68.96 | 60.44 | 62.87 | 78.57 |
| GPT-6-astra | Space-X | 73.82 | 60.05 | 83.81 | 82.73 |
| Gemini-3.8-Flash | Model alone | 63.00 | 58.67 | 43.87 | 73.23 |
| Gemini-3.8-Flash | Space-X | 75.79 | 62.49 | 77.63 | 87.01 |
| Task | Sol | Sol + X | Astra | Astra + X | Flash | Flash + X | Argon | Argon + X |
|---|---|---|---|---|---|---|---|---|
| Art style | 87.18 | 84.62 | 86.32 | 88.03 | 90.60 | 91.67 | 90.60 | 90.52 |
| Counting | 67.50 | 80.51 | 79.17 | 82.35 | 86.67 | 95.74 | 92.50 | 95.33 |
| Forensic detection | 82.58 | 85.61 | 100.00 | 100.00 | 99.24 | 100.00 | 100.00 | 100.00 |
| Functional correspondence | 70.00 | 72.31 | 75.38 | 73.85 | 70.77 | 88.24 | 71.54 | 75.20 |
| IQ test | 51.03 | 59.46 | 78.52 | 80.67 | 57.33 | 87.50 | 84.09 | 80.71 |
| Jigsaw | 93.33 | 96.67 | 100.00 | 99.33 | 96.00 | 97.00 | 100.00 | 100.00 |
| Task | Model alone | Space-X |
|---|---|---|
| Overall | 53.09 | 66.42 |
| Object counting | 45.86 | 49.54 |
| Absolute distance | 30.35 | 40.00 |
| Object size | 73.35 | 73.09 |
| Room size | 50.56 | 55.26 |
| Relative distance | 59.58 | 71.43 |
| Task | GPT-6-sol | GPT-6-astra | Gemini-3.8-Flash | Gemini-4-Argon | ||||
|---|---|---|---|---|---|---|---|---|
| Model alone | Space-X | Model alone | Space-X | Model alone | Space-X | Model alone | Space-X | |
| Overall | 48.40 | 65.34 | 61.08 | 71.99 | 53.09 | 66.42 | 63.32 | 75.40 |
| Object counting | 1.29 | 52.12 | 25.03 | 58.62 | 45.86 | 49.54 | 56.67 | 64.47 |
| Absolute distance | 43.03 | 63.79 | 48.85 | 68.32 | 30.35 | 40.00 | 50.67 | 68.59 |
| Object size | 69.62 | 70.94 | 73.36 | 74.68 | 73.35 | 73.09 | 75.50 | 83.01 |
| Room size | 44.44 | 59.58 | 54.18 | 65.66 | 50.56 | 55.26 | 51.84 | 57.66 |