Existing geometry-aware Multimodal Large Language Models (MLLMs) increasingly rely on external 3D encoders to provide structural priors, but executing these additional encoder paths at inference introduces substantial latency and computational overhead. We propose GeoThinker-F, an efficient framework that internalizes spatiotemporal perception by transforming causal MLLM hidden states into predictive latent world representations. Our key insight is that intermediate semantic representations contain sufficient information to predict useful representations of current geometry and future semantics, without requiring an exact reproduction of features from a dedicated 3D encoder. To realize this idea, GeoThinker-F adopts a decoupled two-stage training strategy. In the first stage, the MLLM backbone is frozen while lightweight Latent Frame Prediction (LFP) heads learn from an offline 3D geometry expert and next-frame semantic targets. In the second stage, the external expert is removed, and the model is jointly optimized to generate and selectively integrate its self-predicted priors through a specialized Spatial-Grounded Temporal (SGT) Fusion module. Extensive evaluations show that GeoThinker-F eliminates runtime dependence on external 3D geometry encoders while maintaining competitive performance across diverse spatial intelligence benchmarks. It achieves up to a 3.6× inference speedup, reducing latency from 1.45s to 0.40s while retaining strong spatial reasoning performance.
Figures & tables
Figure 2: Overview of the GeoThinker- F pipeline. The framework adopts two decoupled training stages to internalize geometric and temporal representations within the MLLM. Stage 1 (Left): With the MLLM backbone frozen, Latent Frame Prediction (LFP) heads attached to intermediate layers learn to predict latent geometry features produced by an offline 3D expert and semantic features of the next frame. Stage 2 (Right): The external geometry encoder is removed from the pipeline. The resulting geometry and future-semantic representations are selectively integrated by SGT-Fusion modules inserted at intermediate MLLM layers for spatial reasoning.
Methods
Avg.
Numerical Answer
Multiple-Choice Answer
Obj.Count
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
Baseline
Chance Level (Random)
–
–
–
–
–
25.0
36.1
28.3
25.0
Chance Level (Frequency)
34.0
62.1
32.0
29.9
33.1
25.1
47.9
28.4
25.2
Proprietary Models (API)
GPT-4o ( Hurst et al., 2024 )
34.0
46.2
5.3
43.8
38.2
37.0
41.3
31.5
28.5
Table 1: Performance comparisons on VSI-Bench (Vanilla regime). By internalizing 3D geometry priors and incorporating temporal dynamics priors, GeoThinker- F completely eliminates the external VGGT encoder while outperforming the GeoThinker.
Models
AVG.
VSI-Bench
MMSI-Bench
MindCube
ViewSpatial
SITE
CV-Bench
Human
–
79.2
97.2
94.5
–
67.5
–
Random Choice
–
34.0
25.0
33.0
26.3
0.0
–
Proprietary Models
Seed-1.6 ( ByteDance Seed, 2025 )
53.41
49.9
38.3
48.7
43.8
54.6
85.2
Gemini-2.5-Pro ( Comanici et al., 2025 )
56.33
53.5
38.0
57.6
46.0
57.0
85.9
GPT-5 ( OpenAI, 2025 )
57.50
55.0
41.8
56.3
45.5
61.8
84.6
Table 2: Cross-benchmark comparison on spatial intelligence benchmarks (Scaled regime). † indicates evaluation on reduced subsets. * indicates trained with official S1+S2 setting. Benchmarks include VSI-Bench ( Yang et al., 2025a ) , MMSI-Bench ( Yang et al., 2026c ) , MindCube ( Yin et al., 2025 ) , Viewspatial ( Li et al., 2025a ) , SITE ( Wang et al., 2025e ) , CV-Bench ( Tong et al., 2024 ) .
Figure 3: Efficiency and throughput comparison. (a) Compute vs. throughput trade-off, showing GeoThinker- F positioned in the ultra-efficient zone with minimal FLOPs and superior throughput. (b) Relative speedup over the highest-accuracy baseline among the compared methods within each backbone family, reaching an 8.8× acceleration on Qwen2.5-VL-7B compared to Cambrian-P and a 3.6× acceleration on Qwen3-VL-8B compared to GeoThinker.
Figure 4: Qualitative comparison between predicted geometry representations and VGGT feature targets. From top to bottom, we show the input RGB frames, geometry representations predicted by LFP head, and the corresponding feature targets from offline VGGT expert. The predicted features demonstrate remarkable structural alignment with the offline-expert feature targets.
Obj. Count
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Appr. Order
LFP Pretrain
Geometry
Temporal
Avg.
Numerical Answer
Multiple-Choice Answer
✗
✗
✗
49.36
67.6
36.2
58.9
60.8
42.6
45.6
26.2
56.6
✗
✓
✓
49.32
69.4
36.8
57.7
60.5
46.7
35.6
30.4
57.1
✓
✓
✗
50.17
70.0
35.8
57.0
58.9
47.1
41.5
32.5
58.4
✓
✗
✓
50.54
70.1
37.1
57.1
59.9
46.6
41.8
31.9
59.7
✓
✓
✓
50.76
69.8
36.2
57.6
58.1
47.9
41.8
30.9
63.6
Table 3: Ablation Study on VSI-Bench. All compared variants are trained on the same vanilla SFT data with Qwen2.5-VL-7B backbone, showing the individual and joint contributions of our endogenous Geometry-Prior LFP and Temporal-Prior LFP branches.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Method
FLOPs(T)
Amortized Latency
NC ↑
DAC ↑
TTC ↑
Comf. ↑
EP ↑
PDMS ↑
(ms/output token)
Constant Velocity
–
–
68.0
57.8
50.0
100
19.4
20.6
Ego Status MLP
–
–
93.0
77.3
83.6
100
62.8
65.6
ReCogDrive w/ InternVL
73.45
32.15
95.0
93.7
87.8
100
78.7
81.9
GeoThinker
122.69
42.62
98.0
93.2
94.1
100
78.4
84.4
GeoThinker- F (Ours)
88.22
37.08 (-52.91%)
96.5
94.0
90.4
100
78.7
83.3
Appendix
Table 4: Performance comparison on NAVSIM navtest . Compared with ReCogDrive, GeoThinker- F improves key driving metrics. Compared with GeoThinker, it reduces the spatial-prior integration overhead by 52.91%. Evaluation includes N ot-at-fault C ollisions (NC), D rivable A rea C ompliance (DAC), T ime- T o- C ollision within bound (TTC), Comf ort (Comf.), E go and P rogress (EP).
Model
FLOPs(T)
Latency(ms)
Steps
Spatial
Object
Goal
Long
Avg
Baseline
π0 +FAST
-
-
-
96.4
96.8
88.6
60.2
85.5
π0
-
-
-
96.8
98.8
95.8
85.2
94.1
OpenVLA-OFT-7B
-
-
175K
97.6
98.4
97.9
94.5
97.1
GR00T-N1.5
-
-
20K
92.0
92.0
86.0
76.0
86.5
Qwen3-VL-4B
Appendix
Table 5: Comparison of different VLA models on LIBERO. We train one policy for all 4 suites. All scores are averaged over 500 trials for each task suite (10 tasks × 50 episodes). Compared with GeoThinker, it reduces the spatial-prior integration overhead by 87.15%.
Model
FLOPs(T) ↓
MACs(T) ↓
Latency(s) ↓
VSI ↑
Qwen2.5VL-3B
Qwen-2.5VL-3B
95.29
47.64
0.60
28.6
3DThinker Qwen-2.5VL-3B *
95.29
47.64
0.60
59.1
VG-LLM-4B
152.92
76.43
1.43
46.7
GeoThinker 8frame
157.81
78.87
1.48
48.9
Qwen2.5VL-7B
Appendix
Table 6: Quantitative efficiency and spatial reasoning performance comparison across different model backbones. We report computational cost in FLOPs (T) and MACs (T), inference latency (s), and accuracy on VSI-Bench. * denotes leveraging estimated lower bound of latency. GeoThinker- F 32frame achieves substantial reductions in FLOPs, MACs, and inference latency compared to external 3D encoder baselines (e.g., GeoThinker), while maintaining highly competitive performance.
Extra-Module
Total
Backbone
Extra-Module
Total
Backbone
Extra-Modu
Total
Avg.
Methods
Params (M)
FLOPs (T)
Latency (ms)
ReCogDrive
0.0
7944.4
73.45
0.00
73.45
5046.78
0
5046.78
32.15
GeoThinker
2076.1
10020.5
73.45
49.23
122.69
5229.24
1461.42
6690.67
42.62
GeoThinker- F
1187.1
9131.6
73.45
14.77
88.22
5222.86
598.20
5821.07
37.08
Appendix
Table 7: Resource efficiency and latency breakdown on autonomous driving evaluation. We report extra-module and total parameter counts ( M ), computational complexity in FLOPs ( T ), latency breakdowns ( ms ), and amortized Latency ( Avg. ).
Extra-Module
Total
Backbone
Extra-Module
Total
Methods
Params (M)
FLOPs (T)
Latency (ms)
QwenOFT
0.0
4503.4
2.44
0.00
2.44
82.0
GeoThinker
1705.5
6208.9
2.44
3.40
5.84
254.8
GeoThinker- F
277.0
4780.4
2.44
0.07
2.51
104.2
Appendix
Table 8: Computational footprint and inference latency comparison on embodied robotic manipulation. Parameter size ( M ), FLOPs ( T ), and total latency ( ms ) are decomposed across the backbone and added auxiliary modules.
Model
Video Mixture
VSI
VideoMME
MVBench
Cambrian-S-7B
✗
69.2
54.1
-
3M
65.1 (-4.1)
61.9 (+7.8)
64.5
GeoThinker Qwen3vl-8B-32frame
✗
72.0
53.7
42.8
430k
72.6 (+0.6)
59.4 (+5.7)
69.1 (+26.3)
GeoThinker- F Qwen3vl-8B
✗
70.00
55.9
55.8
430k
70.4 (+0.4)
58.7 (+2.8)
56.9 (+1.1)
Appendix
Table 9: Impact of general video data co-training. We evaluate performance changes ( Δ ) when co-training with 430k general video instruction samples. While standard 2D MLLM baselines exhibit task interference of degrading 3D spatial accuracy on VSI, GeoThinker- F effectively mitigates cross-task degradation. By internalizing spatial-temporal priors directly within hidden representations, GeoThinker- F maintains robust spatial intelligence while simultaneously enhancing general temporal understanding.
Model
Benchmark
# Frames
16
32
64
128
Cambrian-S-7B
VSI
58.6
63.6
66.4
67.5
VSI-Debiased
49.7
55.6
59.1
59.9
VG-LLM-8B*
VSI
60.5
62.2
63.7
63.1
VSI-Debiased
51.6
52.4
55.2
55.1
GeoThinker
VSI
69.2
72.6
73.4
73.4
Appendix
Table 10: Inference-time frame-count scaling on VSI and VSI-Debiased. Models trained on up to 32 -frame windows are evaluated under scaling inference frames up to 128 frames to assess temporal context extrapolation and resistance to shortcut learning. GeoThinker and GeoThinker- F using configuration of Qwen3VL-8B with 32frame setting.
Cam-Obj Abs. Dist.
Cam. Displace.
Cam. Mov. Dir.
Obj-Obj Rel. Pos.
Cam-Obj Rel. Dist.
Methods
Avg.
Numerical Answer
Multiple-Choice Answer
Baseline
Chance Level (Random)
-
-
-
36.1
50.0
36.1
Chance Level (Frequency)
27.4
5.4
6.2
40.7
52.2
32.4
Human Performance
Human Level
77.0
51.4
46.8
95.1
97.5
94.3
Appendix
Table 11: Performance comparison on VSTI-Bench. GeoThinker-F Qwen3VL-8B achieves a competitive average score of 64.0%, outperforming several proprietary and open-source VLM baselines while eliminating the runtime external geometry encoder. Bold and underlined values denote the best and second-best results, respectively.
Models
RANK
AVG.
VSI
MMSI
MindCube
ViewSpatial
SITE
BLINK
3DSRBench
EmbSpatial
SenseNova-SI-1.3-InternVL3-8B
1
65.2
68.6
42.5
89.9
61.3
47.5
68.0
62.4
81.0
GeoThinker- F Qwen3VL-8B
2
64.8
70.4
30.8
93.2
82.6
53.6
55.8
55.3
76.5
SenseNova-SI-1.2-InternVL3-8B
3
64.5
69.6
42.6
89.0
58.8
49.0
69.4
60.1
77.7
SenseNova-SI-1.5-InternVL3-8B
4
64.4
67.3
38.3
92.1
59.0
47.5
69.5
61.1
80.4
SenseNova-SI-1.4-InternVL3-8B
5
63.9
66.6
40.1
88.8
55.7
47.9
68.1
62.2
81.7
Gemini 3 Pro
6
63.8
52.5
45.2
70.9
50.4
62.3
76.0
68.9
84.3
Appendix
Table 12: EASI-8 leaderboard. VSI denotes VSI-Bench ( Yang et al., 2025a ) . MMSI denotes MMSI-Bench ( Yang et al., 2026c ) . MindCube denotes MindCube-Tiny ( Yin et al., 2025 ) .Viewspatial ( Li et al., 2025a ) ,SITE ( Wang et al., 2025e ) , BLINK ( Fu et al., 2024 ) . 3DSRBench ( Ma et al., 2025 ) .EmbSpatial denotes EmbSpatial-Bench ( Du et al., 2024 ) .
Models
RANK
AVG.
VSI
MMSI
MindCube
ViewSpatial
SITE
BLINK
3DSRBench
EmbSpatial
MMSI-Video
OmniSpatial
SPAR
VSI-Debiased
SenseNova-SI-1.5-InternVL3-8B
1
64.4
67.3
38.3
92.1
59.0
47.5
69.5
61.1
80.4
-/-
-/-
-/-
-/-
SenseNova-SI-1.4-InternVL3-8B
2
63.9
66.6
40.1
88.8
55.7
47.9
68.1
62.2
81.7
-/-
-/-
-/-
-/-
NEO-ov-9B-SFT
3
63.4
64.8
41.3
90.0
55.2
54.0
62.8
61.2
77.8
-/-
-/-
-/-
-/-
Qwen3.5-VL-35B-A3B
4
61.6
61.1
41.9
63.5
50.8
61.4
70.5
59.8
83.6
-/-
-/-
-/-
-/-
Gemini 3 Pro
5
60.2
52.5
45.2
70.9
50.4
62.3
76.0
68.9
84.3
40.4
69.1
48.7
53.8
GeoThinker- F Qwen3VL-8B
6
60.1
70.4
30.8
93.2
82.6
53.6
55.8
55.3
76.5
28.4
42.5
66.9
65.8
Appendix
Table 13: EASI-ALL leaderboard (In-Domain). VSI denotes VSI-Bench ( Yang et al., 2025a ) . MMSI denotes MMSI-Bench ( Yang et al., 2026c ) . MindCube denotes MindCube-Tiny ( Yin et al., 2025 ) .Viewspatial ( Li et al., 2025a ) ,SITE ( Wang et al., 2025e ) , BLINK ( Fu et al., 2024 ) . 3DSRBench ( Ma et al., 2025 ) . EmbSpatial denotes EmbSpatial-Bench ( Du et al., 2024 ) . SPAR denotes SPAR-Bench ( Zhang et al., 2025 ) . MMSI-Video denotes MMSI-Video-Bench ( Lin et al., 2025 ) . OmniSpatial ( Jia et al., 2026 ) . VSI-Debiased ( Brown et al., 2025 ) .
Methods
Frames
Avg.
Numerical Answer
Multiple-Choice Answer
Obj.Count
Abs. Dist.
Obj. Size
Room Size
Rel. Dist.
Rel. Dir.
Route Plan
Proprietary Models
GPT-5.2
64
50.9
56.2
41.5
73.9
63.0
48.4
34.9
38.2
Gemini 3 Flash
1 FPS
57.6
65.7
53.1
77.6
52.8
64.6
47.9
41.8
Gemini 3 Pro
1 FPS
60.9
60.1
54.7
79.3
51.9
68.1
56.0
56.4
Open-source Models
Appendix
Table 14: Performance comparisons on ReVSI-Bench. ReVSI ( Zhang et al., 2026 ) introduces a dynamic evaluation setup where ground-truth answers adapt according to the evaluated frame count. GeoThinker- F maintains competitive accuracy on ReVSI-Bench without executing a runtime external geometry encoder, while trailing its external-encoder GeoThinker counterpart.
Model
Benchmark
# Frames
16
32
64
128
GeoThinker Qwen3vl-8B-32frame
VSI
69.2
72.6
73.4
73.4
VSI-Debiased
64.3
66.3
67.7
68.1
ReVSI
52.6
53.9
54.8
55.9
GeoThinker- F Qwen3vl-8B-32frame
VSI
67.0
70.4
72.1
72.4
VSI-Debiased
61.7
65.8
67.1
66.7
Appendix
Table 15: Zero-shot frame scaling on ReVSI. Evaluates temporal extrapolation from 16 to 128 frames, where target ground-truth annotations dynamically adjust to the changing visual context.
Models
Geometry
AVG.
VSI-Bench
MMSI-Bench
MindCube
ViewSpatial
SITE
CV-Bench
Encoder
Qwen2.5VL-7B
GeoThinker
✓
60.43
68.5
31.7
83.6
41.9
54.8
82.1
GeoThinker- F
✗
60.56
67.7
31.0
88.3
42.1
53.6
80.5
GeoThinker- F + OmniSpatial-Train
✗
66.84
67.3
31.2
86.8
83.2 (+41.1)
53.2
79.1
Qwen3VL-8B
Appendix
Table 16: Ablation study on the influence of the OmniSpatial-Train data mixture. Controlled comparison between GeoThinker and GeoThinker- F with and without the OmniSpatial training dataset, isolating the core architectural contributions of endogenous feature prediction under strictly aligned training data distributions.
Training
General
LFP
VSI ↑
MMSI ↑
Cross-Bench.
Latency ↓
Frame
Video
FineTuning
Avg. ↑
GeoThinker
8
✗
–
65.8
29.1
57.67
1.69
32
✓
–
68.5
31.7
60.43
1.45
GeoThinker- F
8
✗
✗
63.6 (-2.2)
28.1 (-1.0)
65.42
1.04
Appendix
Table 17: Ablation study on Stage-2 training strategies for GeoThinker- F . Evaluated on a Qwen2.5-VL-7B backbone across frame budgets (8 vs. 32 frames), General Video mixture integration, and LFP head fine-tuning status. Orange numbers at ( ⋅ ) denote performance gaps relative to GeoThinker. Jointly fine-tuning LFP heads alongside general video mixtures yields the optimal balance across specialized spatial and multi-modal benchmarks.
Figure 5: Qualitative structural correspondence in standard scenes. For each sequence, we show input RGB frames (top), LFP-predicted geometry representations (middle), and the corresponding VGGT feature targets (bottom). In these selected scenes with clear foreground subjects and structured motion, the endogenous predictions preserve coherent object-centered activation patterns and broad spatial correspondence with the VGGT targets across consecutive frames. This qualitative agreement illustrates that GeoThinker- F can recover useful geometry-related structure from intermediate MLLM hidden states.
Figure 6: Qualitative semantic selectivity under visually ambiguous conditions. For each sequence, we show input RGB frames (top), LFP-predicted geometry representations (middle), and the corresponding VGGT feature targets (bottom). In these selected snow and driving examples, the VGGT targets exhibit comparatively diffuse or background-dominated activations, whereas the endogenous predictions remain concentrated around the primary human or vehicle. This case-level contrast suggests that prediction from MLLM hidden states can preserve semantically relevant spatial focus when low-level visual cues are ambiguous.
Figure 7: Qualitative limitations of endogenous geometry prediction. For each sequence, we show the input RGB frames (top), LFP-predicted geometry representations (middle), and the corresponding VGGT feature targets (bottom). Top: In the presence of rapidly changing non-rigid structures, the predicted representations exhibit weaker correspondence with the VGGT targets. Bottom: The predicted maps display coarser spatial variation than the corresponding VGGT feature targets. These examples illustrate conditions in which endogenous geometry prediction does not fully recover the geometry-related structure captured by the external expert.
Multimodal Large Language Models (MLLMs) excel at 2D semantic understanding but lack intrinsic 3D awareness, resulting in representations that fail to maintain geometric and spatial consistency across video frames. Given the scarcity of large-scale 3D data, we present GeoVR, a novel framework that learns geometric representations using purely 2D video sequences. This approach effectively restructures the semantic latent space within MLLMs to unlock spatial intelligence. Rather than employing superficial feature mixing, GeoVR reshapes the internal representations of the MLLM by distilling geometry knowledge from pre-trained 3D foundation models. This is accomplished through a multi-objective learning strategy driven by four complementary geometric targets: (1) estimating inter-frame camera poses to embed varying viewpoint dynamics, (2) regressing dense depth maps to anchor physical distances, (3) predicting a metric scale factor for real-world calibration, and (4) distilling multi-scale 3D features to align the intermediate feature space. Guided by these explicit physical and geometric constraints, the model's internal representations naturally develop strong 3D awareness. Extensive experiments on spatial reasoning benchmarks demonstrate that GeoVR achieves state-of-the-art performance, establishing a new paradigm for endowing foundation models with spatial intelligence.
Multimodal large language models (MLLMs) are increasingly being applied to spatial cognition tasks, where they are expected to understand and interact with complex environments. Most existing works improve spatial reasoning by introducing 3D priors or geometric supervision, which enhances performance but incurs substantial data preparation and alignment costs. In contrast, purely 2D approaches often struggle with multi-frame spatial reasoning due to their limited ability to capture cross-frame spatial relationships. To address these limitations, we propose EgoMind, a Chain-of-Thought framework that enables geometry-free spatial reasoning through Role-Play Caption, which jointly constructs a coherent linguistic scene graph across frames, and Progressive Spatial Analysis, which progressively reasons toward task-specific questions. With only 5K auto-generated SFT samples and 20K RL samples, EgoMind achieves competitive results on VSI-Bench, SPAR-Bench, SITE-Bench, and SPBench, demonstrating its effectiveness in strengthening the spatial reasoning capabilities of MLLMs and highlighting the potential of linguistic reasoning for spatial cognition. Code and data are released at https://github.com/Hyggge/EgoMind.
Zhenghao Chen, Huiqun Wang, Di Huang
State Key Laboratory of Complex and Critical Software Environment, Beihang University · School of Computer Science and Engineering, Beihang University
Despite progress in vision-language models, 3D spatial reasoning from 2D images remains challenging. Text-based methods describe intermediate geometry with discrete tokens, limiting fidelity for continuous spatial relations. Continuous latents offer richer representations, but a single latent type does not explicitly separate the cues needed across spatial tasks. Decomposed spatial latents address this by representing position, direction, and global geometry separately under geometric supervision. Yet the geometry representation can still collapse toward one dominant direction, and unrestricted attention can leave the latents underused during answer learning. We introduce GeoLatent, combining Common--Residual Geometry Alignment (CR-GEO) with routed optimization to structure the geometry states while promoting latent-mediated answer learning. CR-GEO separates shared from residual teacher geometry; routed optimization jointly trains geometry and language, temporarily directs visual answer learning through the latents, and restores full attention with geometry supervision. In controlled comparisons, CR-GEO raises geometry effective rank from 1.00 to 3.87, while blocking latent readout at the bottleneck lowers direction accuracy from 89.1% to 25.8% on 128 fixed questions. After recovery, the differentiated geometry representation and latent-mediated visual route remain available alongside direct image access. GeoLatent achieves 73.0% on SPAR-Bench and 72.1% on SPBench, outperforming previously reported methods on both.
Yakun Zhu, Yi Bin, Yujuan Ding +5
Tongji University · The Hong Kong Polytechnic University