As multimodal large language models (MLLMs) advance toward understanding and interacting with the physical world, spatial reasoning becomes essential for understanding scene structure and anticipating the effects of physical changes. Existing spatial benchmarks have made substantial progress in evaluating spatial relations, memory, and viewpoint reasoning, but less systematically assess how models connect physical scene conditions with visual observations. We introduce SpatialCast, a benchmark that uses cast shadows to evaluate this ability. Cast shadows provide observable evidence of a 3D scene. Their image appearance depends on object geometry and pose, illumination, and viewpoint. We examine this relationship through three complementary tasks that assess scene-projection consistency, predict intervention-induced changes, and infer the physical changes behind visual transitions. The benchmark contains 21,000 synthetic questions with separate scene and geometry test partitions, complemented by 200 SpatialCast-Real questions from 20 photographed scenes. Evaluations show that closed-source MLLMs outperform general-purpose open-weight MLLMs and MLLMs designed for spatial reasoning, which remain near chance on the synthetic benchmark, while even strong closed-source models struggle with shadow-length and light-height judgments. We also use SpatialCast training data for GRPO post-training, improving open-weight MLLM performance on the benchmark. Further evaluations show improvements on several external spatial benchmarks, suggesting transfer of the learned capabilities. These experiments support SpatialCast as a controlled resource for diagnosing spatial reasoning capabilities and studying their learning and generalization.
Figures & tables
Figure 1: Motivation for SpatialCast. (a) Human intuition for relating 3D form and lighting to visual appearance. (b) We ask whether MLLMs can match scenes with shadow projections, predict intervention outcomes, and infer change parameters from paired observations given the changed factor. (c) Cast shadows provide physically constrained evidence connecting scene geometry, pose, illumination, and viewpoint to observable 2D projections. The shadow footprints are schematic; benchmark observations contain both objects and their cast shadows.
Figure 2: SpatialCast task and dataset overview. Panels (a,b) are schematic illustrations. (a) The tasks share 16 shape families and three scene structures. (b) T1 combines forward prediction and inverse inference of lighting. T2 predicts the outcome of a specified intervention. T3 infers change parameters from before/after observations and a known changed factor. The synthetic question pool contains 21,000 questions, including 4,500 test questions. (c) SpatialCast-Real contains 200 questions across 20 photographed scenes, illustrated by a T2 reference with four candidate outcomes and a T3 before/after pair. Counts refer to the complete pool before training-data screening. Experiments use the full T1/T2/T3 test scope shown here.
Task
Given
Select
Variants
T1 Compatibility
Geometry and light
Compatible projection
Forward
Observed shadow
Compatible light setting
Inverse
T2 Propagation
Initial view I and intervention a
Resulting view I′
L, R, C LR, LC, RC
T3 Attribution
Views I,I′ and the changed factor
Parameters of the change
L, R, C
Table 1: Overview of the three SpatialCast reasoning tasks. L, R, and C denote light, object rotation, and camera viewpoint, respectively. Paired letters denote joint interventions.
Figure 3: Representative SpatialCast questions. (a,b) T1 asks for a projected scene or a light configuration. (c–e) T2 specifies a light, object-rotation, or camera change. Panel (c) shows the initial observation and all A–D candidates. Panels (d,e) show the initial observation and the correct candidate. (f) T3 gives the changed factor and asks for its change parameters. Green highlights mark ground-truth answers. In the top-view diagrams, yellow circles denote the light, red circles the object, and green triangles the camera. H denotes object height. CW and CCW refer to clockwise and counterclockwise turns viewed from above. Scene views share a fixed display crop within each question.
Synthetic (4,500)
Real (200)
Model
T1
T2
T3
Overall
Overall
GPT-6.1 Sol
67.50
88.39
85.33
79.42
79.50
Gemini 4 Argon
43.89
66.17
68.44
57.71
63.00
Qwen3-VL-8B Base [ 2 ]
22.89
25.44
25.33
24.40
34.50
SpatialCast-GRPO (8B)
30.94
48.78
34.22
38.73
36.50
Qwen3-VL-4B Base [ 2 ]
23.17
27.28
23.67
24.91
33.50
Table 2: SpatialCast and SpatialCast-Real performance. Accuracy (%) on 4,500 synthetic questions (1,800 T1, 1,800 T2, 900 T3) and 200 real-image questions. Synthetic Overall is micro-averaged accuracy across the three tasks. Lavender marks SpatialCast-GRPO. Bold marks each local block’s maxima.
Table 3: API-model accuracy across projection reasoning subtypes. Accuracy (%) on the same 4,500 synthetic test questions, with column groups and order matching Fig. 4 . Dir./Len./Cont. denote shadow direction, length, and contour. Azim./Ht. denote light azimuth and height. L/R/C denote light, object-rotation, and camera viewpoint changes, and paired letters denote joint interventions.
Model
T2
T3
Overall
Random
25.00
50.00
34.38
GPT-6 Astra
77.60
88.00
81.50
GPT-6.1 Sol
76.00
85.33
79.50
Claude Sonnet 5.5
66.40
82.67
72.50
Gemini 4 Argon
52.80
80.00
63.00
Qwen3.8-Max (0902) †
48.00
76.00
58.50
Table 4: API-model performance on SpatialCast-Real. Accuracy (%) on 125 T2 and 75 T3 questions (200 overall). Random denotes uniform guessing.
Model
Full Input
Shadow- only
No Shadow
No Initial
GPT-6.1 Sol
81.3
81.7
25.0
41.7
Qwen3-VL-4B Base
27.1
27.1
22.9
19.6
SpatialCast-GRPO (4B)
49.2
42.9
30.0
28.8
Qwen2.5-VL-3B Base
26.3
25.4
22.9
20.0
SpatialCast-GRPO (3B)
34.6
35.0
25.4
24.6
InternVL3-2B Base
24.2
27.1
24.6
24.2
Table 5: Shadow evidence ablation on 240 T2-L questions. Original-label agreement (%); shadow-erased scores are diagnostic. See Supplementary Sec. D.3 .
32-item pilot
T1
T2
T3
Real
Overall
Human mean
60.00
63.33
66.67
80.00
65.00
GPT-6.1 Sol
50.00
75.00
83.33
50.00
65.63
Gemini 4 Argon
50.00
66.67
50.00
75.00
59.38
Uniform random
25.00
25.00
25.00
37.50
26.56
Table 6: Human and API references on the 32-question pilot. Accuracy (%). Human mean averages five participants. Real denotes SpatialCast-Real. Uniform random accounts for two binary real-image T3 items. All other items have four choices.
Model
MC
MMSI
SI
MV
MMMU
Avg.
Qwen3-VL-8B Base
35.33
25.70
40.05
32.79
53.67
37.51
SpatialCast-GRPO (8B)
42.29
27.80
41.06
32.79
51.67
39.12
Qwen3-VL-4B Base
33.81
25.00
38.54
31.15
52.00
36.10
SpatialCast-GRPO (4B)
40.38
27.00
42.07
39.34
52.89
40.34
Qwen2.5-VL-3B Base
35.33
28.60
35.52
37.70
47.22
36.87
SpatialCast-GRPO (3B)
41.24
26.50
39.55
36.07
45.11
37.69
Table 7: External transfer before and after SpatialCast training. Accuracy is reported as a percentage. MC denotes MindCube, and SI/MV denote SPBench single-/multi-view. Avg. averages the five scores before rounding. Bold marks the higher value in each Base/trained pair, including ties. Lavender marks SpatialCast-GRPO. Results use greedy decoding. Full comparisons and numerical results appear in Supplementary Sec. D.6 .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Subtype
Train
Val
Scene
Geometry
Total
T1 Contour
1,002
98
201
99
1,400
T1 Direction
998
100
201
100
1,399
T1 Length
1,000
102
198
101
1,401
T1 Light azimuth
1,998
198
402
198
2,796
T1 Light height
1,002
102
198
102
1,404
T2 L
999
100
199
101
1,399
Appendix
Table 8: Synthetic split counts before training selection. Each subtype covers single objects and object pairs with separate or overlapping shadows. Scene and Geometry denote test splits.
Figure 8: Complete SpatialCast-Real questions from ReNe. Panels (a,b) show T2 candidate photographs in their original order. Panels (c,d) show T3 before/after pairs. Green marks the correct answer. CW denotes clockwise as viewed from above along −Z . The top views show the world axes and calibrated camera bearing.
Task / split
GPT-6.1 Sol
Gemini 4 Argon
T1 (1,800)
1,215
790
T2 (1,800)
1,591
1,191
T3 (900)
768
616
Total (4,500)
3,574
2,597
T1 forward (900)
573
365
T1 inverse (900)
642
425
Appendix
Table 9: API synthetic correct counts by task and partition. Split rows also report accuracy. Subtype sizes in the main comparison are 301/299/300/600/300 for T1 direction/length/contour/ azimuth/height, 300/301/299/300/301/299 for T2 L/R/C/LR/LC/RC, and 300/299/301 for T3 L/R/C.
Model
T2 (125)
T3 (75)
Overall (200)
Random
25.00
50.00
34.38
Qwen3-VL-8B
Qwen3-VL-8B Base [ 2 ]
25.60
49.33
34.50
SpatialCast-GRPO (8B)
28.00
50.67
36.50
Qwen3-VL-4B
Qwen3-VL-4B Base [ 2 ]
24.00
49.33
33.50
Appendix
Table 10: Local-model performance on SpatialCast-Real. Accuracy (%) on 125 T2 and 75 T3 questions. Overall covers all 200. Random is the uniform-choice expectation. Blue marks SpatialCast-GRPO. Each trained model is compared with its own Base.
Model pair
T1
T2
T3
Total
Items
1,800
1,800
900
4,500
8B Base
412
458
228
1,098
8B trained
557
878
308
1,743
4B Base
417
491
213
1,121
4B trained
434
843
202
1,479
3B Base
416
470
211
1,097
Appendix
Table 11: Exact synthetic correct counts. Trained denotes each backbone’s final SpatialCast-GRPO checkpoint.
4B configuration
Questions
Steps
Qwen3-VL-4B Base
–
–
Curriculum
15,000
1,875
Full shuffle
15,000
1,875
SpatialCast-GRPO (4B)
11,186
1,399
T2-only
6,000
750
Appendix
Table 12: Post-training configurations on the same 4B base. All configurations use the same test sets and direct-answer protocol. The question count gives the number of distinct training items. Bold marks column maxima. Both panels list the same 4B configurations; Real denotes SpatialCast-Real.
Figure 9: Shadow-only and its matched shadow-erased control. A T2-L example is shown in initial/A–D order. Shadow-only (upper row) preserves cast-shadow differences on fixed object and floor colors. No Shadow (lower row) removes these differences, making all five images identical.
T1
T2
T3
Real
Overall
Items
10
12
6
4
32
P1
50.00
83.33
33.33
75.00
62.50
P2
60.00
58.33
50.00
100.00
62.50
P3
70.00
58.33
83.33
75.00
68.75
P4
70.00
66.67
100.00
75.00
75.00
P5
50.00
50.00
66.67
75.00
56.25
Appendix
Table 13: Five participants on the 32-question pilot. Accuracy (%). Item counts apply to each participant. Human mean aggregates 104 correct responses across five participants answering the same 32 items. Random is the uniform-choice expectation, accounting for the two binary real-image T3 questions.
Model
Scene-test
Geometry-test
Qwen3-VL-8B
738→1,139
360→604
Qwen3-VL-4B
751→1,004
370→475
Qwen2.5-VL-3B
726→951
371→482
InternVL3-2B
779→1,040
381→516
Appendix
Table 14: Correct answers before and after SpatialCast training. Each entry gives base → trained counts on the same scene-test (3,000 questions) or geometry-test (1,500 questions) partition.
SPBench
Model
MC
MMSI
SI
MV
MMMU
Avg.
Qwen3-VL-8B
Base [ 2 ]
35.33
25.70
40.05
32.79
53.67
37.51
SpatialCast-GRPO
42.29
27.80
41.06
32.79
51.67
39.12
Qwen3-VL-4B
Base [ 2 ]
33.81
25.00
38.54
31.15
52.00
36.10
Appendix
Table 15: Complete external evaluation (%). MC denotes MindCube. SPBench SI/MV report single-/multi-view multiple-choice accuracy. Avg. averages the five scores before rounding. Bold marks each block’s best values, including ties. All reported GRPO-trained models use SpatialCast-GRPO. Numerical-question results appear in Table 16 .
Model
VSI macro
SI MRA
MV MRA
Qwen3-VL-8B
Base [ 2 ]
43.12
56.11
65.97
SpatialCast-GRPO
43.33
56.50
65.43
Qwen3-VL-4B
Base [ 2 ]
39.93
54.26
64.81
Spatial-SSRL [ 16 ]
39.55
55.56
67.56
Appendix
Table 16: Additional external metrics (%). VSI macro averages over question types. SI and MV MRA score single-image and multi-view numerical answers on SPBench. Higher values are better in each column.
Nepal Applied Mathematics and Informatics Institute for research (NAAMII), Nepal · INSAIT, Sofia University, Bulgaria · State University of New York at Stony Brook, Korea