Vision-language models (VLMs) are increasingly evaluated for egocentric and cross-view video reasoning, yet existing benchmarks largely focus on semantic event understanding, temporal relations, or correspondence between already observed views, leaving their ability to reason directly about future visual states underexplored. We introduce EgoExo-Next, a visual-option benchmark for dynamic visual-state reasoning, where models must identify how an observed action trajectory subsequently appears rather than predict only an action label or textual description. EgoExo-Next contains 2,503 human-curated four-choice questions from six public egocentric and ego--exo video sources and comprises four interconnected subtasks that evaluate egocentric next-state prediction, bidirectional ego--exo state correspondence, exocentric next-state prediction, and their composition in Ego-to-Exo Next-State. Extensive evaluation of proprietary, open-source, and spatial reasoning VLMs reveals a substantial human--model gap, with the best model achieving 43.81% average accuracy compared with 98.55% for humans, and the largest degradation occurring on the composed Ego-to-Exo task. These results suggest that current VLMs remain substantially limited in dynamic visual-state reasoning, particularly when temporal progression and cross-view reasoning must be composed. The benchmark is publicly available at https://huggingface.co/datasets/yutongli2024/EgoExo-Next.
Figures & tables
Figure 1: The EgoExo-Next benchmark. Main panels: Examples of four subtasks for next-state prediction and ego-exo view matching. Upper right: Comparison of model and human performance, showing a gap between current VLMs and humans.
Figure 2: The upper part is the EgoExo-Next Pipeline. The lower part contains examples on EgoExo-Next. Correct answers are highlighted in red.
Figure 3: Benchmark composition by subtask, scene type, and data source.
Models
Subtask 1
Subtask 2
Subtask 3
Subtask 4
Avg.
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Human Level
98.36
99.14
99.34
99.67
98.85
99.05
97.09
96.86
98.55
Proprietary Models
GPT-6
N/A
N/A
N/A
N/A
N/A
N/A
62.23
58.14
N/A
GPT-5.2
51.48
58.57
46.53
29.31
48.14
47.47
32.89
30.91
43.16
Claude-Opus-4.6
34.75
35.71
42.39
32.12
29.23
41.14
26.33
27.04
33.59
Table 1: Performance on EgoExo-Next. Results are accuracy (%). Avg. denotes the mean over all eight settings. Due to the substantially higher evaluation cost of GPT-6, we evaluate it only on Subtask 4 and therefore do not report results for Subtasks 1–3 or an overall average. Bold and underlined values indicate the best and second-best model results, respectively, excluding human performance.
Evaluation Setting
GPT-5.2
Gemini-3.1-Pro
Qwen3-VL-235B
Qwen3-VL-32B
RoboBrain
Avg.
Overall Shortcut Test: Subtasks 1, 2, 3, 4
Full Input
43.16
43.81
40.93
37.03
38.56
40.70
Candidate Only
27.25
26.75
26.00
25.80
25.75
26.31
Temporal Dependency: Subtasks 1, 3, 4
Full Sequence
44.91
41.45
41.77
37.09
37.95
40.63
Last Frame Only
38.20
36.80
35.90
33.40
33.50
35.56
Table 2: Benchmark validity and shortcut analysis. Results are accuracy (%) averaged over subtasks and scene splits.
The Type of Subtask
The Number of Future Candidates
Method
Subtask 1
Subtask 3
Subtask 4
Avg.
1
2
3
4
Avg.
Random
25.00
25.00
25.00
25.00
25.00
25.00
25.00
25.00
25.00
CLIP
27.35
26.62
25.68
26.54
29.05
27.16
25.68
24.26
26.54
DINO
27.02
27.22
25.20
26.49
28.42
27.79
26.11
23.63
26.49
Table 3: Training-free visual-similarity baselines on the three next-state subtasks. The left block reports accuracy by subtask, while the right block groups questions by the number of future candidates among the four options.
Figure 4: Representative failure cases on EgoExo-Next.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Unified cross-view matching prompt template for the two directions of Subtask 2. The placeholders are instantiated according to the query and candidate viewpoints.
Benchmark
Perspective
Input Type
Primary Evaluation Target
Answer Type
Cross-view
Visual State
EgoSchema
Ego
Video
Long-form temporal understanding
Text
N/A
N/A
EgoPlan-Bench
Ego
Video& Image
Goal-conditioned next-action planning
Text
N/A
N/A
EgoThink
Ego
Image
First-person perception&reasoning
Text
N/A
N/A
VidEgoThink
Ego
Video
VQA& planning& grounding& reward
Text
N/A
N/A
EgoTempo
Ego
Video
Fine-grained temporal understanding
Text
N/A
N/A
EgoProx
Ego
Video& Frames
3D proximity&spatial reasoning
Text
N/A
N/A
Appendix
Table 4: Comparison with representative egocentric, ego–exo, and spatial reasoning benchmarks. “Visual State” indicates whether the prediction target itself is an unobserved subsequent visual state rather than an action, event, relation, or textual description.
Setting
GPT-5.2
Gemini-3.1-Pro
Qwen3-VL-235B
Qwen3-VL-32B
RoboBrain
Avg.
Original Subtask 4
32.40
28.10
27.80
30.20
29.70
29.64
+ Current Exo
48.70
44.00
45.10
38.30
40.10
43.24
+ Next-State Ego
40.20
50.10
39.40
37.70
41.30
41.74
+ Both
59.80
63.00
58.50
54.90
56.70
58.58
S4∣S1,S2E→X,S3
60.50
59.20
57.40
54.10
55.80
57.40
Appendix
Table 5: Diagnostic decomposition of ego-to-exo next-state reasoning on the matched subset. Results are accuracy (%).
Figure 6: Effect of caption augmentation on EgoExo-Next. Left: accuracy gains across eight evaluation settings. Right: average accuracy under image-only and image-plus-caption inputs.
Figure 7: Prompt template used by Qwen2.5-VL-72B to generate captions for candidate images.
Participant
Subtask 1
Subtask 2
Subtask 3
Subtask 4
Avg.
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Participant 1
97.05
99.66
99.67
99.34
99.71
98.42
96.12
97.48
98.43
Participant 2
99.67
98.34
98.68
100.00
97.99
99.68
98.06
96.23
98.58
Participant 3
98.36
99.41
99.67
99.67
98.85
99.05
97.09
96.86
98.62
Mean
98.36
99.14
99.34
99.67
98.85
99.05
97.09
96.86
98.55
Appendix
Table 6: Participant-level human performance on EgoExo-Next. Each entry is computed from the participant’s number of correct answers in the corresponding evaluation split.
Predicting future spatial states supports collision avoidance and timely decision-making in dynamic environments. However, existing vision-language models (VLMs) and benchmarks for spatial reasoning primarily focus on observed scenes, leaving predictive spatial reasoning beyond the observed interval underexplored. To this end, we introduce SpatialMind, a metric-scale VLM for spatial reasoning and future prediction. Its metric depth adapter anchors spatial reasoning to real-world scale, while its progressive state chain establishes current spatial states and observed dynamics as the foundation for future prediction. Given a video prefix, SpatialMind predicts distances, motion directions, and spatial relations in both observed and unseen future frames. For training and evaluation, we build a scalable data engine that grounds entity descriptions in metric geometry to generate question-answer pairs and state supervision. Using this engine, we construct the SpatialMind-30K dataset and the SpatialMind-2K benchmark, both covering driving and everyday egocentric scenes. The benchmark spans eight tasks across three levels: current-state understanding, observed-dynamics understanding, and future prediction. Experiments show that SpatialMind substantially outperforms both general and spatially specialized models on our benchmark while achieving competitive zero-shot performance on VSI-Bench, OSI-Bench, and VLM4D.
Feiran Wang, Xiaoqi Wang, Ziwei Li +3
University of Illinois at Chicago · Bosch Research North America · Bosch Center for Artificial Intelligence (BCAI)
Egocentric video understanding requires procedural reasoning under partial observability and continuously shifting viewpoints. Current multimodal large language models (MLLMs) struggle with this setting, often generating plausible but visually inconsistent or weakly grounded responses. We introduce EgoVITA, a framework that decomposes egocentric video reasoning into a structured plan-then-verify process. The model first generates an egocentric plan: a causal sequence of anticipated actions from a first-person perspective. This plan is then evaluated by an exocentric verification stage that uses third-person reasoning over the same video to verify its spatiotemporal and logical consistency, without exocentric video input. This decomposition enables cross-perspective feedback without requiring paired ego-exo supervision. To train this reasoning process, we adopt Group Relative Policy Optimization (GRPO) with two dense reward signals: one that grounds anticipated actions in subsequent visual observations and another that reinforces consistent third-person verification. EgoVITA achieves state-of-the-art performance on egocentric reasoning benchmarks, outperforming Qwen2.5-VL-7B by +7.7 on EgoBlind and +4.4 on EgoOrient, while maintaining strong generalization on exocentric video tasks with only 52k training samples.
Video event prediction (VEP) requires models to infer unobserved future states from partial video evidence. Existing video MLLMs usually verbalize intermediate future reasoning in text space: once visual evidence is verbalized, fine-grained motion, geometry, and interaction cues can be lost, leading to plausible but visually ungrounded hallucinations. We introduce Future-L1, an interleaved latent visual reasoning framework that lets an MLLM alternate between language tokens and continuous latent visual spans during autoregressive decoding. To train this capability, we construct Future-L1-50K by selecting examples where future visual hints help prediction and align latent states to future-frame embeddings, then further optimize sampled latent trajectories with LA-DAPO, a latent-aware RL objective with outcome-contrastive and temporal-diversity rewards. Future-L1 achieves new state-of-the-art results on both benchmarks: on FutureBench, it improves Qwen3-VL-8B from 61.0 to 85.4 and exceeds the previous best Video-CoE by 10.4 points; on TwiFF-Bench, it improves the average score from 2.44 to 3.04. These results suggest that future-oriented video reasoning benefits from preserving intermediate visual semantics in latent space rather than translating every reasoning step into text.
Tianxiang Jiang, Linquan Wu, Sheng Xia +5
University of Science and Technology of China · Shanghai AI Laboratory · City University of Hong Kong +4