Organizations: 1*East China Normal University, China. · 3Shanghai Innovation Institute, China. · 2Shanghai Artificial Intelligence Laboratory, China. · University of Agder, Norway.
Spatial intelligence is central to embodied cognition, yet contemporary AI systems still struggle to reason about physical interactions in open-world human environments. Despite strong performance on controlled benchmarks, vision-language models often fail to jointly model physical dynamics, reference frames, and the latent human intentions that drive spatial change. We introduce Teleo-Spatial Intelligence (TSI), which captures an important aspect of spatial reasoning: integrating physical dynamics and action goals to reason about actions and their spatial consequences. To evaluate TSI, we present EscherVerse, a large-scale open-world resource built from 11,328 real-world videos, including an 8,000-example benchmark and a 35,963-example instruction-tuning set. Across 27 state-of-the-art vision-language models and an independent analysis of first-pass human responses from 11 annotators, we identify a persistent teleo-spatial reasoning gap: the strongest proprietary model achieves 57.26% overall accuracy, far below first-pass human performance, which ranges from 84.81% to 95.14% with a mean of 90.62%. Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it. EscherVerse provides a diagnostic testbed for purpose-aware spatial reasoning and highlights a critical gap between pattern recognition and human-level understanding in embodied AI.
Figures & tables
Figure 1: Teleo-Spatial Intelligence (TSI) contrasted with the current spatial understanding paradigm. Left, conventional spatial understanding is primarily object-centred, geometry-focused, and grounded in current-state description. Right, TSI extends spatial reasoning toward human-centred, goal-directed, and intent-aware understanding of future spatial change. Example questions illustrate the difference between static geometric queries and intent-conditioned reasoning about dynamic outcomes.
Figure 2: Conceptual design and evaluation of EscherVerse. a, Comparison of geometry-centred perception with goal-directed spatial reasoning, with the six benchmark categories shown below. b, Schematic motivation for evaluating models on real-world video tasks. c, Category-level accuracy profiles for the eight models named in the key; each coloured line represents one model. TSI, Teleo-Spatial Intelligence. Source data are provided as a Source Data file.
Figure 3: EscherVerse examples and construction. a, Example questions from the six categories, presented as text case cards with model and human responses. b, Stages of video selection and question screening. c, Benchmark composition by category (left) and question format (right); coloured sectors identify the categories and formats listed in the keys. d, Question generation and verification pipeline, including model screening and human review; video symbols are schematic. Original video links and clip time ranges for panel a are listed in Supplementary Table 17. QA, question–answer; LLM, large language model; VLM, vision-language model. Source data are provided as a Source Data file.
Figure 4: Independent first-pass human performance exceeds current multimodal models on EscherVerse. Human performance is shown as the distribution of overall accuracies across 11 annotators, each of whom answered all 8,000 benchmark questions under the first-pass protocol. Current frontier proprietary and open-weight models remain substantially below the human range, with the best proprietary model trailing the human mean by 33.36 percentage points. Source data are provided as a Source Data file.
Figure 5: The impact of real-world spatial instruction tuning. a, Accuracy profiles across six spatial-reasoning categories for Escher-8B, the simulation-trained ISI-8B baseline, Qwen3-VL-8B and Gemini-2.5-pro; the differences vary by category. b, Overall accuracy plotted against model parameter count for open-weight models. Closed-source models are shown in a separate display category labelled Closed (Est.); their horizontal positions are illustrative and do not represent verified parameter counts. c, Overall accuracy on EscherVerse for ViCA2, the Qwen3 base model, simulation-trained ISI-8B and Escher-8B after real-world instruction tuning. Source data are provided as a Source Data file.
Figure 6: Benchmarking spatial intelligence across diverse modalities and reasoning types. a, Heatmap visualization of accuracy across six spatial-reasoning categories. Escher-8B (orange text) outperforms the simulation-trained ISI-8B (blue text) and reduces the observed gap to the closed-source models in the top rows. b, Category-wise comparison between ISI-8B and Escher-8B. The largest annotated gains are in Action & Intent and Dynamic Spatial Relationships, followed by Object Deformation, consistent with a benefit from the EscherVerse-35k training data on these benchmark categories. c, Performance by question format. Patterned bars (hatched for Escher and dotted for ISI) show the observed accuracies, including multiple-select questions. Source data are provided as a Source Data file.
Figure 7: Analysis of model behaviours and failure cases. a, Hierarchical clustering of Table 1 accuracies across six spatial-reasoning categories and two question formats (multiple-select and single-choice). Each column is standardized across the ten displayed models by subtracting its mean and dividing by its population standard deviation. Ward linkage with Euclidean distance groups the standardized profiles; positive Z-scores indicate above-average accuracy within this model subset. The two Escher-8B variants cluster together, then with Qwen3-VL-32B-Thinking, and subsequently with GPT-5 and Gemini-2.5-pro. These similarities do not establish a shared internal reasoning mechanism. b, The robustness-complexity landscape. The x-axis represents overall accuracy, while the y-axis denotes the Robustness Ratio (accuracy on multiple-select questions divided by accuracy on single-choice questions). Bubble area is proportional to accuracy on Object Deformation. Both panels use the reported aggregate accuracies; the asterisk denotes Escher-8B-Instruct, and Think denotes its Thinking variant. The background zones indicate higher and lower observed robustness ratios. The arrow shows the performance difference between ISI-8B and Escher-8B under these reported metrics. c, Tracking failure: the model does not correctly integrate simultaneous egocentric and allocentric motion. d, Perspective failure: baseline models misidentify the golf ball’s final position relative to the person specified in the question. e, Reference-frame failure: baseline models identify the wave or water rather than the shore when asked about the surfer’s orientation. Original video links and clip time ranges for panels c–e are listed in Supplementary Table 17. Source data are provided as a Source Data file.
Model
Spatial Reasoning Abilities
Overall
Centricity
Question Types
Obj. Perm. & Occl.
Dynamic Spatial
Action & Intent
Predictive & Count.
Obj. Def. & State
Ego. vs. Allo.
Overall Acc.
Human- Centric
Object- Centric
Fill-in Blank
Multi- Select
Single Choice
True/ False
Open-Source Models
Qwen3-VL-2B-Instruct
39.87
38.24
43.96
38.55
39.34
37.90
38.90
39.88
35.74
30.72
7.80
34.03
50.35
Qwen2.5-VL-3B-Instruct
39.13
38.98
47.13
41.89
44.55
40.15
40.61
41.12
38.96
36.41
11.36
35.46
50.99
Qwen2.5-VL-7B-Instruct
43.37
37.31
46.83
40.03
39.34
34.90
38.66
38.49
39.22
27.96
13.36
33.07
50.74
Qwen3-VL-4B-Instruct
49.17
44.47
51.66
44.98
54.03
42.96
45.60
46.06
44.12
47.51
34.08
38.44
53.93
Table 1: Comprehensive evaluation of Vision-Language Models on the spatial reasoning benchmark. Accuracy (%) is reported for all models. Independent human performance is reported as overall first-pass accuracy only, because category-level human breakdowns are not part of this table. Source data are provided as a Source Data file.
Condition
Intent
Δ Base
Dynamic
Δ Base
Base
50.21%
–
37.65%
–
Intent-focused
57.51% (3.35)
+7.30
39.09% (0.00)
+1.44
Dynamic-only
54.53% (1.16)
+4.32
43.62% (1.16)
+5.97
Random mixture
52.16% (1.02)
+1.95
41.77% (1.45)
+4.12
Table 2: Equal-budget SFT ablation. Parentheses report the standard deviation across two training seeds rather than confidence intervals. Source data are provided as a Source Data file.
Primary label
n
Weighted
95% interval
Spatiotemporal grounding
86
28.8%
24.0–34.2
Perspective/reference frame
72
23.7%
19.3–28.9
Physical prediction
43
14.2%
10.7–18.6
Perception/recognition
36
12.1%
8.9–16.3
Benchmark ambiguity
29
9.6%
6.8–13.5
Action–goal binding
22
7.4%
5.0–11.0
Table 3: Adjudicated primary error distribution. Weighted intervals use a Kish effective sample size and are descriptive. Source data are provided as a Source Data file.