Spatial reasoning is a cornerstone capability for intelligent systems to perceive and interact with the physical world. However, multimodal large language models (MLLMs) frequently suffer from hallucinations and imprecision when parsing complex geometric layouts. As data-driven scaling struggles to internalize structured geometric priors and spatial constraints, integrating mature, specialized vision models presents a compelling alternative. Despite its promise, applying this paradigm to spatial reasoning is hindered by two key challenges: The difficulty of invoking heterogeneous, parameter-rich tools, as well as the challenge of understanding and effectively leveraging their diverse low-level outputs (e.g., segmentation masks, depth maps) in high-level reasoning. To address these challenges, we propose LAST, a unified framework for tool-augmented spatial reasoning. LAST features an extensible interactive sandbox, termed LAST-Box, which abstracts heterogeneous tool invocations into atomic instructions and reusable spatial skills, returning multimodal hints (e.g., annotated images and textual descriptions) that can be directly consumed by LLMs. We further design a three-stage progressive training strategy that guides models from understanding tool outputs to proficient and adaptive tool invocation. Experiments on four datasets show that LAST-7B achieves around 20% performance gains over its backbone and outperforms strong proprietary closed-source LLMs, substantially enhancing reasoning on complex spatial tasks.
Figures & tables
Figure 1. Visualization of preliminary experimental results for problem analysis.
Figure 2. Overview of the proposed LAST-Box Architecture diagram of LAST-Box showing two layers: atomic operations (GroundingDINO, SAM, VGGT, DepthAnything, Python, Owlv2) at the bottom, and spatial skills (EstimateSize, EstimateDepth) at the top, with example inputs and outputs.
Figure 3. Illustration of the progressive training strategy. Diagram illustrating the three-stage progressive training strategy: Stage 1 Warmup with raw and augmented image pairs, Stage 2 SFT with multi-task spatial reasoning, and Stage 3 IRL with GRPO optimization over multiple rollouts.
Models
CVbench
SPBench-SI
EmbSpatial
MSMU
Avg
2D-REL
3D-Depth
3D-Dist
Rel_Dir
Rel_dist
Size_Est
Abs_Dist
General Models
Qwen2.5-VL-3B
72.46
79.00
65.00
27.45
67.39
18.61
10.67
56.29
8.67
45.06
LLAVA-NEXT-8B
70.62
66.67
60.30
23.43
72.34
27.63
15.34
50.30
10.63
44.14
LLAVA-OneVision-8B
78.23
78.23
75.77
45.27
50.63
35.70
25.00
56.20
14.00
51.00
InternVL3-8B
73.85
83.17
67.83
32.03
23.08
24.62
20.13
58.98
13.44
44.13
Table 1. Comparison between our LAST-7B and other baseline models. Numerical estimation tasks are highlighted with a pink background. Bold indicates the best performance in each category, while underline denotes the second best.
Figure 4. Representative experimental examples of LAST-7B from CVBench, EmbSpatial, and MSMU. Four qualitative examples showing LAST-7B reasoning traces with tool calls for relative depth, spatial relationship, size estimation, and distance tasks, each with thinking steps, tool invocations, and final answers.
Models
CVBench
MSMU
2D-REL
3D-Depth
3D-Dist
Gemini-3-flash
58.64
68.90
73.20
25.07
+ LAST-Box
70.92
82.25
65.14
26.14
Δ (delta)
12.28 ↑
13.35 ↑
8.06 ↓
1.07 ↑
Gemini-2.5-Pro
67.48
79.41
80.35
27.52
+ LAST-Box
85.48
90.32
95.25
30.06
Table 2. Performance changes of closed-source models after integrating LAST-Box.
Dataset
Tool SR
Acc(w/uns)
Acc(w/suc)
Δ Acc
CVBench
88.67
87.65
91.51
↑ 3.86
SPBench-SI
95.14
40.82
69.69
↑ 28.87
EmbSpatial
99.67
60.00
80.74
↑ 20.74
Table 3. Tool success rate and accuracy under successful and unsuccessful tool usage across different datasets of LAST-7B.
Figure 5. Comparison of tool invocation behavior between After SFT (Stage 2) and After RL (Stage 3). (a) Top-4 tool usage distribution across benchmarks. (b) Multi-step invocation rate across benchmarks. Two subfigures: (a) pie charts comparing tool usage distribution after SFT vs after RL, showing more balanced usage after RL; (b) bar chart showing multi-step tool invocation rates increase after RL across all benchmarks.
Settings
CVBench
SPBench-SI
EmbSpatial
MSMU
Backbone
78.27
44.87
69.70
14.20
w/o Stage 1 (Warmup)
89.63
64.87
79.63
39.23
w/o Stage 2 (SFT)
79.30
48.26
65.45
20.63
w/o Stage 3 (RL)
90.37
65.20
80.67
38.74
LAST-7B
91.17
66.76
81.00
41.17
Table 4. Ablation on training stages. Removing any stage leads to performance degradation.
Settings
CVBench
SPBench-SI
EmbSpatial
MSMU
Backbone
78.27
44.87
69.70
14.20
No-tool SFT
83.52
63.81
78.23
39.73
Text-only hint
86.77
66.90
80.16
38.38
Image-only hint
86.24
65.11
79.56
39.21
LAST-7B (Full hint)
91.17
66.76
81.00
41.17
Table 5. Ablation on tool hint modality. “No-tool SFT” trains with the same data but without any tool outputs. “Image-only” and “Text-only” restrict the hint modality during inference.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 6. The system prompt designed to guide the model in selecting and executing visual tools.
College of Computer Science and Artificial Intelligence, Fudan University · School of Computing and Information Systems, Singapore Management University · School of Computer Science and Technology, East China Normal University +2
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy · Stevens Institute of Technology