SpatialBrain: Spatial Intelligence Growing from Experience

Yifan Zhou; Ziyang Gong; Xiaojun Ye; Yifei Liu; Donggang Li; Xiaolong Zhou; Zhihang Zhong; Xue Yang

cs.AIOct 21, 2026

SpatialBrain: Spatial Intelligence Growing from Experience

Authors: Yifan Zhou, Ziyang Gong, Xiaojun Ye, Yifei Liu, Donggang Li, Xiaolong Zhou, Zhihang Zhong, Xue Yang

Organizations: Shanghai Jiao Tong University · Zhejiang University

Abstract

Spatial reasoning requires more than access to geometric tools: a reasoner must decide when direct observation is sufficient and when to reconstruct, measure, or change reference frames. We present SpatialBrain, a framework that retains experience about these decisions while keeping its vision-language backbone fixed. The key distinction is between evidence about a scene and experience about how to reason with that evidence. 3DFusionMemory fuses current-scene observations into queryable spatial relations. A Spatial Skill Library provides reusable tool procedures, and Spatial Cognition associates task conditions with experience about when and how to use memory and skills. Completed trajectories are abstracted into process records and consolidated between task blocks, allowing later decisions to draw on earlier execution without transferring scene coordinates or reference answers. Across five spatial-reasoning benchmarks, SpatialBrain improves on direct answers from the same backbone on four and ties on the fifth. Component comparisons, experience-coverage curves, and traced resource decisions characterize how the framework accumulates and uses these resources.

Figures & tables

Figure 1: SpatialBrain: spatial reasoning with experience reuse. Spatial Cognition chooses how to use direct reasoning, spatial skills, and scene evidence. Completed trajectories update procedures and resource-use experience for later tasks; 3DFusionMemory is rebuilt for each scene. The scene is illustrative.

Figure 2: Self-evolution from earlier trajectories. (a) Initial reasoning leaves the reference frame unresolved. (b) Completed earlier trajectories are abstracted and consolidated between task blocks. (c) Accumulated experience guides memory access and skill selection for the same target. The trajectories and outcomes are illustrative.

Figure 3: Multi-view observations become spatial relations. Three of six input views, a saved fused 3D scene, and view support for five estimated object nodes. Colors link identities. The RGB surface uses saved depth and camera poses; edges show unambiguous stored relations. Labels give stored directions and 3D measurements derived from object centers. Coordinates use A as origin and reconstruction-world axes; lengths are in units of $d_{0}=|AB|$ , not meters.

Method	MMSI	MindCube	OmniSp.	SpaceDG	SAT	Avg
Foundation VLMs
Gemini-2.5-Pro	36.9	57.5	55.2	47.2	79.3	55.2
Qwen3-VL-8B [ 2 ]	27.4	43.4	51.5	43.1	74.7	48.0
GPT-5.5	51.0	74.3	67.8	57.5	84.7	67.1
GPT-5.6-sol	53.8	67.2	68.3	56.9	91.3	67.5
Training-based Spatial VLMs

Table 1: Spatial reasoning performance across five benchmarks. Scores are percentages; higher is better. Our SpaceDG evaluations use a 1,000-question subset. Avg is the unweighted mean across benchmarks. Bold marks the best displayed score; dashes indicate unavailable results. Evaluation settings are given in Appendix A .

Figure 4: Errors by task type and reasoning demand. All 1,050 MindCube and 1,000 MMSI items. (a) Direct GPT-5.6-sol and SpatialBrain by official category. (b) Input-derived categories for the same pair and component configurations, all using GPT-5.6-sol. Radii show pooled error rates (0–80%). Appendix B.3 gives the mapping, counts, and within-study comparisons.

Panel of Figure 5 — Figure 5: Memory-grounded reference-frame reasoning. Saved 3D anchors are shown on an RGB cutaway of the reconstructed scene. The planner sees the memory atlas with single-view support flags; a tool verifies the stair-relative direction. Coordinates use reconstruction units. Full trajectory: Appendix C.1 .

Figure 6: MindCube component comparison. Accuracy overall and by task category on 1,050 items. The y-axis starts at 40%. Evaluation settings are given in Appendix A .

Figure 7: Test-time experience and seed-skill coverage. (a) Distinct task signatures with stored experience. (b) Distinct seed skills referenced by that experience. Each curve spans the benchmark’s full evaluation scope (Section 4.1 ).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Factor	Setting	Applies to
Evaluation size	MMSI 1,000; MindCube 1,050; OmniSpatial 1,533; SpaceDG 1,000; SAT 150.	Our main-table evaluations
Metric	Accuracy (%); MRA for numeric SpaceDG questions.	Our scored comparisons
Option order	Fixed question-option order, without circular evaluation.	Our SAT evaluations
Backbone	GPT-5.6-sol.	Direct / SpatialBrain pair
Reference answers	Excluded from agent inputs and experience updates; predictions fixed before scoring.	Our agent evaluations
Unanswered items	Retained in the denominator and counted as wrong, including refusals.	Component comparisons

Appendix

Table A1 : Evaluation settings. The final column specifies which comparisons share each setting. Published baselines follow their original protocols.

Figure A1 : Stored spatial evidence across benchmarks. (a) Tasks with 3D objects or relations. (b) Snapshot availability and 3D-object presence. Coverage measures stored state, not retrieval or answer correctness.

Category	MindCube	MMSI	Pooled
Relations	178/98 (459)	279/247 (574)	457/345 (1,033)
Motion/rotation	117/7 (405)	64/63 (163)	181/70 (568)
Sequential reasoning	46/25 (153)	51/56 (113)	97/81 (266)
Measurement	—	49/44 (112)	49/44 (112)
Other attributes/actions	3/0 (33)	19/19 (38)	22/19 (71)

Appendix

Table A2: Errors within input-derived categories. Top: Direct/SpatialBrain wrong-answer counts, with category size in parentheses. Bottom: pooled error rates (%) for all radar curves. Each method uses the same 2,050 items and fixed category assignment.

Figure A2 : MMSI component errors by task family. Earlier component configurations on all 1,000 items. Bars count wrong answers, including unanswered questions. The four families partition the evaluation set; configuration and answer-availability differences are detailed in Appendix A .

Figure A3 : MMSI errors by task type. Direct GPT-5.6-sol answers and SpatialBrain on all 1,000 items. Each rate divides wrong answers by the number of items in that type. Lower is better.

Spatial Reasoning in Language Models

See all 51 papers

EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents
Oct 12, 2026Bin Yu, Shijie Lian, Zhaolong Shen +10Tool-Using Agents Spatial Reasoning
Harbin Institute of Technology · Zhongguancun Academy · DeepCybo +5
SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning
Sep 28, 2026Ruifan Zuo, Guocheng Hu, Wanshui Gan +3Visual Spatial Reasoning VLM Reasoning
Shandong University · Shanghai AI Laboratory · Zhiyang Innovation Co., Ltd.
Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning
Aug 8, 2026Shi-Yu Tian, Zhuo-Xia Wang, Xuan-Yi Zhu +6Tool Use in VLMs LLM Agent Skill Learning
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · College of Computer Science, Sichuan University

Oct 12, 2026J/K move · Enter open · S save

EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents

Bin Yu, Shijie Lian, Zhaolong Shen, Xiaopeng Lin+9

Harbin Institute of Technology · Zhongguancun Academy · DeepCybo+5

Tool-Using Agents Spatial Reasoning Visual Spatial Reasoning Agent Skill Learning Self-Evolving Agents Memory-Augmented Agents

Open paper AskPDF arXiv

Spatial reasoning agents can inspect tool outputs and repair errors, but reusing these lessons requires preserving both the computation and the conditions that make it valid. We introduce EvoSpatial, a framework that learns from execution experience by updating a library of linked skills and tools. Skills describe procedures, geometric assumptions, and verification steps; tools implement the operations those procedures require. When solving a task, the agent retrieves relevant skills and synthesizes reusable computational functions if existing tools are insufficient. After each task, execution feedback and evidence checks guide skill revision and tool consolidation, without ground-truth answers or benchmark scores. Across 20 benchmarks, EvoSpatial averages 63.9, 4.0 percentage points above the average reported by SpatialClaw. The evolved library also improves held-out and cross-benchmark performance when updates are disabled, and trajectories generated with evolved skills improve supervised fine-tuning, even without external skills at evaluation. Keeping checked procedures on record, together with the tools that support them, lets an agent carry spatial reasoning experience from one task to the next.

Abstract

Spatial Reasoning in Language Models

Explore similar work

EvoSpatial: Experience-Driven Skill and Tool Evolution for Spatial Reasoning Agents

SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning

Self-Evolving Neuro-Symbolic Skills for Tool-Augmented Spatial Reasoning