Vision-language models (VLMs) are increasingly evaluated for egocentric and cross-view video reasoning, yet existing benchmarks largely focus on semantic event understanding, temporal relations, or correspondence between already observed views, leaving their ability to reason directly about future visual states underexplored. We introduce EgoExo-Next, a visual-option benchmark for dynamic visual-state reasoning, where models must identify how an observed action trajectory subsequently appears rather than predict only an action label or textual description. EgoExo-Next contains 2,503 human-curated four-choice questions from six public egocentric and ego--exo video sources and comprises four interconnected subtasks that evaluate egocentric next-state prediction, bidirectional ego--exo state correspondence, exocentric next-state prediction, and their composition in Ego-to-Exo Next-State. Extensive evaluation of proprietary, open-source, and spatial reasoning VLMs reveals a substantial human--model gap, with the best model achieving 43.81% average accuracy compared with 98.55% for humans, and the largest degradation occurring on the composed Ego-to-Exo task. These results suggest that current VLMs remain substantially limited in dynamic visual-state reasoning, particularly when temporal progression and cross-view reasoning must be composed. The benchmark is publicly available at https://huggingface.co/datasets/yutongli2024/EgoExo-Next.
Figures & tables
Figure 1: The EgoExo-Next benchmark. Main panels: Examples of four subtasks for next-state prediction and ego-exo view matching. Upper right: Comparison of model and human performance, showing a gap between current VLMs and humans.
Figure 2: The upper part is the EgoExo-Next Pipeline. The lower part contains examples on EgoExo-Next. Correct answers are highlighted in red.
Figure 3: Benchmark composition by subtask, scene type, and data source.
Models
Subtask 1
Subtask 2
Subtask 3
Subtask 4
Avg.
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Human Level
98.36
99.14
99.34
99.67
98.85
99.05
97.09
96.86
98.55
Proprietary Models
GPT-6
N/A
N/A
N/A
N/A
N/A
N/A
62.23
58.14
N/A
GPT-5.2
51.48
58.57
46.53
29.31
48.14
47.47
32.89
30.91
43.16
Claude-Opus-4.6
34.75
35.71
42.39
32.12
29.23
41.14
26.33
27.04
33.59
Table 1: Performance on EgoExo-Next. Results are accuracy (%). Avg. denotes the mean over all eight settings. Due to the substantially higher evaluation cost of GPT-6, we evaluate it only on Subtask 4 and therefore do not report results for Subtasks 1–3 or an overall average. Bold and underlined values indicate the best and second-best model results, respectively, excluding human performance.
Evaluation Setting
GPT-5.2
Gemini-3.1-Pro
Qwen3-VL-235B
Qwen3-VL-32B
RoboBrain
Avg.
Overall Shortcut Test: Subtasks 1, 2, 3, 4
Full Input
43.16
43.81
40.93
37.03
38.56
40.70
Candidate Only
27.25
26.75
26.00
25.80
25.75
26.31
Temporal Dependency: Subtasks 1, 3, 4
Full Sequence
44.91
41.45
41.77
37.09
37.95
40.63
Last Frame Only
38.20
36.80
35.90
33.40
33.50
35.56
Table 2: Benchmark validity and shortcut analysis. Results are accuracy (%) averaged over subtasks and scene splits.
The Type of Subtask
The Number of Future Candidates
Method
Subtask 1
Subtask 3
Subtask 4
Avg.
1
2
3
4
Avg.
Random
25.00
25.00
25.00
25.00
25.00
25.00
25.00
25.00
25.00
CLIP
27.35
26.62
25.68
26.54
29.05
27.16
25.68
24.26
26.54
DINO
27.02
27.22
25.20
26.49
28.42
27.79
26.11
23.63
26.49
Table 3: Training-free visual-similarity baselines on the three next-state subtasks. The left block reports accuracy by subtask, while the right block groups questions by the number of future candidates among the four options.
Figure 4: Representative failure cases on EgoExo-Next.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Unified cross-view matching prompt template for the two directions of Subtask 2. The placeholders are instantiated according to the query and candidate viewpoints.
Benchmark
Perspective
Input Type
Primary Evaluation Target
Answer Type
Cross-view
Visual State
EgoSchema
Ego
Video
Long-form temporal understanding
Text
N/A
N/A
EgoPlan-Bench
Ego
Video& Image
Goal-conditioned next-action planning
Text
N/A
N/A
EgoThink
Ego
Image
First-person perception&reasoning
Text
N/A
N/A
VidEgoThink
Ego
Video
VQA& planning& grounding& reward
Text
N/A
N/A
EgoTempo
Ego
Video
Fine-grained temporal understanding
Text
N/A
N/A
EgoProx
Ego
Video& Frames
3D proximity&spatial reasoning
Text
N/A
N/A
Appendix
Table 4: Comparison with representative egocentric, ego–exo, and spatial reasoning benchmarks. “Visual State” indicates whether the prediction target itself is an unobserved subsequent visual state rather than an action, event, relation, or textual description.
Setting
GPT-5.2
Gemini-3.1-Pro
Qwen3-VL-235B
Qwen3-VL-32B
RoboBrain
Avg.
Original Subtask 4
32.40
28.10
27.80
30.20
29.70
29.64
+ Current Exo
48.70
44.00
45.10
38.30
40.10
43.24
+ Next-State Ego
40.20
50.10
39.40
37.70
41.30
41.74
+ Both
59.80
63.00
58.50
54.90
56.70
58.58
S4∣S1,S2E→X,S3
60.50
59.20
57.40
54.10
55.80
57.40
Appendix
Table 5: Diagnostic decomposition of ego-to-exo next-state reasoning on the matched subset. Results are accuracy (%).
Figure 6: Effect of caption augmentation on EgoExo-Next. Left: accuracy gains across eight evaluation settings. Right: average accuracy under image-only and image-plus-caption inputs.
Figure 7: Prompt template used by Qwen2.5-VL-72B to generate captions for candidate images.
Participant
Subtask 1
Subtask 2
Subtask 3
Subtask 4
Avg.
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Indoor
Outdoor
Participant 1
97.05
99.66
99.67
99.34
99.71
98.42
96.12
97.48
98.43
Participant 2
99.67
98.34
98.68
100.00
97.99
99.68
98.06
96.23
98.58
Participant 3
98.36
99.41
99.67
99.67
98.85
99.05
97.09
96.86
98.62
Mean
98.36
99.14
99.34
99.67
98.85
99.05
97.09
96.86
98.55
Appendix
Table 6: Participant-level human performance on EgoExo-Next. Each entry is computed from the participant’s number of correct answers in the corresponding evaluation split.