Humans perceive and understand real-world spaces through a stream of visual observations. Therefore, the ability to streamingly maintain and update spatial evidence from potentially unbounded video streams is essential for spatial intelligence. The core challenge is not simply longer context windows but how spatial information is selected, organized, and retained over time. In this paper, we propose Spatial-TTT towards streaming visual-based spatial intelligence with test-time training (TTT), which adapts a subset of parameters (fast weights) to capture and organize spatial evidence over long-horizon scene videos. Specifically, we design a hybrid architecture and adopt large-chunk updates parallel with sliding-window attention for efficient spatial video processing. To further promote spatial awareness, we introduce a spatial-predictive mechanism applied to TTT layers with 3D spatiotemporal convolution, which encourages the model to capture geometric correspondence and temporal continuity across frames. Beyond architecture design, we construct a dataset with dense 3D spatial descriptions, which guides the model to update its fast weights to memorize and organize global 3D spatial signals in a structured manner. Extensive experiments demonstrate that Spatial-TTT improves long-horizon spatial understanding and achieves state-of-the-art performance on video spatial benchmarks. Project page: https://liuff19.github.io/Spatial-TTT.
Figures & tables
Figure 2: Overview of Spatial-TTT . The model employs a hybrid architecture that interleaves TTT layers with self-attention anchor layers at a 3:1 ratio to preserve pretrained knowledge while enabling efficient long spatial-context compression. Within each TTT layer, sliding window attention (SWA) and TTT branch operate in parallel with shared Q/K/V projections, where the TTT branch applies spatial predictive mechanism with depthwise spatiotemporal convolution to capture geometric structure and temporal continuity.
Models
Numerical Question
Multiple-Choice Question
Avg.
Obj. Count
Abs. Dist
Obj. Size
Room Size
Rel. Dis
Rel. Dir
Route Plan
Appr. Order
Human
94.3
47.0
60.4
45.9
94.7
95.8
95.8
100.0
79.2
Random Choice
62.1
32.0
29.9
33.1
25.1
47.9
28.4
25.2
34.0
Proprietary Models
Seed-2.0 ( Bytedance Seed, 2025 )
49.4
25.3
69.5
25.8
61.8
44.9
44.3
71.0
50.7
Grok-4 ( xAI, 2025 )
37.1
32.9
60.8
45.4
53.1
39.6
47.4
66.8
47.9
Table 1: Evaluation Results on VSI-Bench ( Yang et al., 2025a ) . For Numerical Questions, we report MRA score. For Multiple-Choice Questions, we report ACC score, Avg. is the macro average across all tasks, following the original paper. For human, we directly use the reported results in VSI-Bench, which is on a subset of VSI-Bench with 400 samples. We use and to denote the best and second-best results within proprietary models and open-source models, respectively.
Models
Avg.
Rotation
Among
Around
Human
94.5
-
-
-
Random Choice
33.0
33.3
31.8
35.7
Proprietary Models
Seed-2.0 ( Bytedance Seed, 2025 )
54.8
89.0
45.2
52.0
Grok-4 ( xAI, 2025 )
63.5
93.0
54.4
61.6
Gemini-2.5-pro ( Comanici et al., 2025 )
57.6
88.0
44.9
63.2
Table 2: Evaluation results on MindCube-Tiny ( Yin et al., 2025 ) . Avg. is micro average across all tasks, following the original paper. We use and to denote the best and second-best results within proprietary models and open-source models, respectively.
Models
VSI-SUPER-Recall
VSI-SUPER-Count
10min
30min
60min
120min
10min
30min
60min
120min
General Models and Spatial Intelligence Models
Qwen3-VL-2B ( Bai et al., 2025a )
35.0
30.0
28.3
0.0
0.8
0.0
0.0
0.0
Cambrian-S-7B ( Yang et al., 2025c )
38.3
35.0
6.0
0.0
0.6
0.0
0.0
0.0
Long-video Understanding Models
MovieChat ( Song et al., 2024 )
18.3
21.7
16.7
26.7
0.0
0.0
0.0
0.0
Table 3: Results on VSI-SUPER-Recall (VSR) and VSI-SUPER-Count (VSC) ( Yang et al., 2025c ) . We report the average score on the 10-120 minute subset of VSR and the 10–120 minute subset of VSC. For all subsets, videos are sampled at 1 fps. Qwen3-VL-2B and Cambrian-S-7B run out of memory (OOM) on an 80GB-GPU for videos longer than 120 minutes; therefore, their scores are reported as 0.
Setting
Numerical
Multiple-Choice
Avg.
Spatial-TTT
64.0
64.8
64.4
w/o SP-Mechanism
60.7
63.4
62.1
w/o Dense Data
61.0
61.5
61.3
w/o Hybrid Arch
55.4
52.4
53.9
Table 4: Ablations of Spatial-TTT on VSI-Bench ( Yang et al., 2025a ) . “w/o SP-Mechanism ” denotes replacing spatial-predictive conv with identity projections. “w/o Dense Data” denotes training without dense scene-description data. “w/o Hybrid Arch” denotes pure TTT architecture.
Models
128 frames
256 frames
512 frames
1024 frames
MEM
TFLOPs
MEM
TFLOPs
MEM
TFLOPs
MEM
TFLOPs
Qwen3-VL-2B ( Bai et al., 2025a )
6.2
75.9
8.3
179.9
12.6
473.9
21.2
1403.1
Spatial-MLLM-4B ( Wu et al., 2025a )
25.9
1698.8
41.8
6002.1
OOM
N/A
OOM
N/A
Spatial-TTT-2B (Ours)
6.2
74.3
7.0
156.2
8.4
341.9
11.9
799.4
Table 5: Peak memory usage (GB) and TFLOPs per forward pass for varying input lengths. The results are measured at 352×480. For fair comparison, we select representative models with comparable base model sizes: Qwen3-VL-2B ( Bai et al., 2025a ) as the general-purpose MLLM baseline, and Spatial-MLLM-4B ( Wu et al., 2025a ) as the geometry-augmented spatial VLM baseline. Spatial-MLLM-4B ran out of GPU memory at 512 and 1024 frames on our test hardware (reported as OOM), TFLOPs are therefore not reported.
Streaming video understanding studies how AI systems interpret continuous video in real time while retaining useful history and responding at the right moment.