Understanding the physical world requires more than object recognition, scene description, and short-term visual prediction, as real-world physical systems involve multiple continuous fields, latent causal mechanisms, partial observations, and intervention-sensitive dynamics. We propose FireWorldBench, a benchmark for evaluating complex physical world intelligence in multimodal large language models and agents through coupled-field fire dynamics. Fire provides a canonical stress-test environment, where multiple interacting physical fields jointly shape observable states and temporal dynamics. FireWorldBench is organized along two complementary axes, a physical capability axis and a fire scenario task axis, jointly covering physical-state understanding, temporal dynamics, causal mechanisms, and intervention reasoning. The benchmark comprises 520 fire-world entries, including 494 controlled simulation worlds and 26 real-world-aligned event groups, spanning 47 scene archetypes across 7 environment families. These entries combine structured textual observations, multiple 2D physical-field visualizations, and 3D event-level scene modeling, yielding 9,074 text-image interleaved question-answer pairs across choice-based and open-ended report-generation formats. FireWorldBench evaluates whether models can infer latent physical states, explain underlying mechanisms, forecast coupled-field evolution, and assess intervention consequences from multimodal partial observations, providing a challenging testbed for complex physical world intelligence.
Figures & tables
Figure 2: Overview of Dataset. The left sunburst summarizes the hierarchical composition of 520 fire incidents, while the right panels characterize the coverage of 8,360 simulated and 714 real-world-aligned QA pairs along the five physical capability axis (P1–P5) and fire scenario task axis (T1–T3).
Figure 3: Physical field perception and grounding. The same fire-world state is queried through choice-format and open-report interfaces using co-registered 3D scene, temperature-field, and soot-density observations. Both interfaces share the same evidence and physical target, while the open report additionally requires an explicit conclusion, supporting evidence, and causal mechanism.
Figure 4: Construction pipeline of FireWorldBench. Designed scenarios and multimodal records of observed experimental fires are encoded as event constraints and instantiated with FDS. Smokeview produces time-indexed 3D renderings, while the solver state yields aligned physical-field observations. These co-registered outputs are subsequently converted into multimodal question–answer pairs with held-out annotations derived from the underlying physical state.
Figure 5: Real-world-aligned fire-world reconstruction. For a heptane pool-fire experiment, observed video frames anchor the event chronology, while the aligned 3D reconstruction and corresponding FDS realization provide simulation-completed temperature, soot-density, velocity, visibility, and streamwise-velocity fields at matched time points. The sequence preserves the distinction between observed evidence and reconstructed latent physical state.
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
31.93
42.55
0.44
31.35
39.79
0.49
28.38
39.19
0.44
37.16
40.09
0.42
29.38
38.81
0.42
31.64
40.09
0.44
Qwen3-8B [ 35 ]
46.55
56.68
0.31
35.34
42.53
0.48
33.88
45.05
0.38
42.98
45.58
0.37
31.58
39.89
0.44
38.07
45.95
0.40
Gemma-3-12B-IT [ 36 ]
34.67
44.06
0.45
38.57
48.96
0.42
33.30
44.88
0.42
34.03
36.60
0.47
27.89
36.01
0.51
33.70
42.10
0.46
Table 1: Overall evaluation along the physical capability axis on the controlled simulation subset. Models are evaluated by averaging the corresponding choice-format and open-report scores under the same observation track, capability dimension, and metric. Each capability dimension reports accuracy (Acc), macro-F1 (F1), and Brier score, with the Average column summarizing performance across P1–P5.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
32.55
40.39
0.46
29.81
39.07
0.45
32.57
41.68
0.43
31.64
40.38
0.45
Qwen3-8B [ 35 ]
37.53
43.84
0.44
35.61
45.19
0.39
44.50
52.78
0.33
39.21
47.27
0.39
Gemma-3-12B-IT [ 36 ]
37.82
46.69
0.43
29.27
38.84
0.47
34.58
42.19
0.46
33.89
42.57
0.46
Table 2: Overall evaluation along the fire scenario task axis on the controlled simulation subset. Models are evaluated by averaging matched choice-format and open-report scores under the same simulated fire-world entry, observation track, task dimension, and metric. Each task reports accuracy (Acc), macro-F1 (F1), and Brier score, with Average summarizing performance across T1–T3.
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
35.16
43.98
0.46
30.85
39.77
0.48
32.95
41.36
0.43
29.25
37.84
0.46
34.49
44.73
0.36
32.54
41.53
0.44
Qwen3-8B [ 35 ]
37.95
45.41
0.49
35.06
43.30
0.49
48.86
57.95
0.31
39.06
48.25
0.42
34.49
45.00
0.41
39.09
47.98
0.42
Gemma-3-12B-IT [ 36 ]
43.73
52.13
0.35
44.34
55.99
0.34
39.43
47.23
0.38
40.54
49.12
0.41
38.20
47.50
0.40
41.25
50.40
0.38
Table 3: Overall evaluation along the physical capability axis on the real-world-aligned subset. Models are evaluated by aggregating the corresponding choice-format and open-report results under the same real-world-aligned event groups, observation track, capability dimension, and metric. Each capability dimension reports accuracy (Acc), macro-F1 (F1), and Brier score, with the Average column summarizing performance across P1–P5.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
33.84
34.05
0.44
25.69
30.04
0.50
34.96
37.76
0.43
31.50
33.95
0.46
Qwen3-8B [ 35 ]
39.12
40.15
0.45
39.94
41.73
0.39
36.93
41.02
0.47
38.66
40.97
0.44
Gemma-3-12B-IT [ 36 ]
45.82
46.33
0.34
35.67
39.71
0.43
42.10
43.52
0.37
41.20
43.18
0.38
Table 4: Overall evaluation along the fire scenario task axis on the real-world-aligned subset. Models are evaluated by averaging matched choice-format and open-report scores under the same real-world-aligned event group, observation track, task dimension, and metric. Each task reports accuracy (Acc), macro-F1 (F1), and Brier score, with Average summarizing performance across T1–T3.
Method
Llama-3.1-8B-Instruct
Qwen3-8B
InternVL3-8B
Qwen3-VL-8B
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Vanilla
31.1
32.9
0.77
33.4
35.7
0.80
25.1
26.4
0.59
29.2
29.5
0.79
+Prompt
33.4 ↑ 2.3
34.5 ↑ 1.6
0.80 ↑ 0.03
35.5 ↑ 2.1
37.0 ↑ 1.3
0.82 ↑ 0.02
29.4 ↑ 4.3
29.4 ↑ 3.0
0.56 ↓ 0.03
34.5 ↑ 5.3
33.7 ↑ 4.2
0.78 ↓ 0.01
+CoT
32.2 ↑ 1.1
33.6 ↑ 0.7
0.70 ↓ 0.07
35.7 ↑ 2.3
37.1 ↑ 1.4
0.79 ↓ 0.01
28.2 ↑ 3.1
30.0 ↑ 3.6
0.45 ↓ 0.14
31.8 ↑ 2.6
33.9 ↑ 4.4
0.82 ↑ 0.03
+SFT
43.1 ↑ 12.0
44.6 ↑ 11.7
0.89 ↑ 0.12
40.8 ↑ 7.4
40.8 ↑ 5.1
0.84 ↑ 0.04
34.1 ↑ 9.0
37.3 ↑ 10.9
0.87 ↑ 0.28
41.0 ↑ 11.8
39.3 ↑ 9.8
0.82 ↑ 0.03
Table 5: FireWorldGPT results on controlled simulations. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.
Figure 6: Physical capability profiles and perception–mechanism gaps. Results on the controlled-simulation (top) and real-world-aligned (bottom) subsets, each comprising 25 equally weighted model–track results. (a,d) Performance across P1–P5;(b,e) Cross-field coupling and (c,f) mechanism attribution versus field perception. Mean gaps measure field perception minus coupling or mechanism accuracy in percentage points, circles and triangles denote S-track and I-track results, respectively.
Figure 7: Paired performance gaps between structured and multimodal observations. (a) Accuracy and (b) Macro-F1 differences across the physical capability and fire-task axes on the controlled and real-world-aligned subsets. Gaps are computed as S-track minus I-track in percentage points using format-averaged overall scores; positive values favor structured observations.
Figure 8: Active-parameter scaling yields limited returns in complex physical-world understanding. (a) Format-averaged I-track accuracy across Qwen3-VL and InternVL3 checkpoints, with solid and dashed lines denoting controlled and real-world-aligned settings, respectively. (b) Capability-wise differences show a broad advantage of the 32B model over the 22B-active checkpoint.
Figure 9: Physical correctness and confidence quality. Completion accuracy versus the benchmark’s Brier-style score on the physical capability axis for (a) controlled simulation and (b) real-world-aligned data. Circles and triangles denote S-track and I-track results. Dashed segments connect illustrative similar-accuracy contrasts.
Figure 10: Question quality and expert–Gold agreement. (a) Expert-assessed question-quality and leakage-check rates on 300 controlled-simulation QA instances. (b) Expert acceptance of Gold answers and mechanisms, alongside exact agreement between expert responses and Gold at the answer and output-slot levels. Circles denote rates from the original audit, while diamonds denote expert-reported recheck rates. Horizontal error bars show nominal 95% Wilson confidence intervals.
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
23.33
34.36
0.59
22.21
31.01
0.65
23.52
34.43
0.58
28.83
40.15
0.52
26.59
37.92
0.49
24.90
35.57
0.56
Qwen3-8B [ 35 ]
38.40
49.22
0.33
30.75
38.93
0.55
27.70
38.56
0.47
39.31
50.16
0.39
33.70
44.03
0.44
33.97
44.18
0.44
Gemma-3-12B-IT [ 36 ]
22.14
31.59
0.64
32.70
43.56
0.52
32.36
45.62
0.48
31.25
42.09
0.53
28.65
38.52
0.57
29.42
40.28
0.55
Appendix
Table 6: Choice-format evaluation along the physical capability axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
40.53
50.73
0.30
40.49
48.57
0.33
33.25
43.95
0.30
45.48
40.02
0.32
32.17
39.71
0.36
38.38
44.60
0.32
Qwen3-8B [ 35 ]
54.71
64.14
0.29
39.94
46.13
0.42
40.05
51.53
0.29
46.64
40.99
0.35
29.46
35.76
0.43
42.16
47.71
0.35
Gemma-3-12B-IT [ 36 ]
47.21
56.54
0.26
44.44
54.36
0.32
34.25
44.15
0.36
36.82
31.11
0.41
27.13
33.49
0.45
37.97
43.93
0.36
Appendix
Table 7: Open-report evaluation along the physical capability axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
24.67
–
0.59
25.86
–
0.56
24.09
–
0.57
24.87
–
0.57
Qwen3-8B [ 35 ]
33.76
–
0.50
32.74
–
0.42
37.31
–
0.35
34.60
–
0.43
Gemma-3-12B-IT [ 36 ]
32.84
–
0.52
31.19
–
0.51
23.65
–
0.62
29.23
–
0.55
Appendix
Table 8: Choice-format evaluation along the fire scenario task axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
40.42
46.22
0.33
33.75
41.26
0.34
41.05
48.17
0.30
38.41
45.22
0.32
Qwen3-8B [ 35 ]
41.30
45.11
0.38
38.47
46.31
0.36
51.69
57.54
0.30
43.82
49.65
0.35
Gemma-3-12B-IT [ 36 ]
42.79
49.67
0.35
27.35
34.18
0.44
45.50
51.18
0.29
38.55
45.01
0.36
Appendix
Table 9: Open-report evaluation along the fire scenario task axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
20.89
30.74
0.66
22.16
30.93
0.65
28.03
40.00
0.51
28.50
40.17
0.52
33.80
46.67
0.41
26.68
37.70
0.55
Qwen3-8B [ 35 ]
23.10
33.78
0.63
32.40
40.87
0.54
37.12
48.18
0.41
30.62
43.00
0.52
37.50
51.11
0.43
32.15
43.39
0.50
Gemma-3-12B-IT [ 36 ]
42.08
52.41
0.42
43.92
55.69
0.39
37.95
49.00
0.43
42.75
55.25
0.39
43.06
53.89
0.39
41.95
53.25
0.40
Appendix
Table 10: Choice-format evaluation along the physical capability axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.
Model
P1
P2
P3
P4
P5
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
49.42
57.22
0.26
39.55
48.61
0.32
37.88
42.73
0.36
30.00
35.50
0.40
35.19
42.78
0.32
38.41
45.37
0.33
Qwen3-8B [ 35 ]
52.79
57.04
0.35
37.73
45.72
0.43
60.61
67.73
0.20
47.50
53.50
0.33
31.48
38.89
0.39
46.02
52.58
0.34
Gemma-3-12B-IT [ 36 ]
45.39
51.86
0.29
44.77
56.30
0.29
40.91
45.45
0.34
38.33
43.00
0.44
33.33
41.11
0.40
40.55
47.54
0.35
Appendix
Table 11: Open-report evaluation along the physical capability axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
24.49
34.01
0.60
27.80
39.92
0.53
24.70
35.44
0.58
25.66
36.46
0.57
Qwen3-8B [ 35 ]
32.70
42.44
0.53
32.72
44.39
0.47
27.35
38.89
0.57
30.92
41.91
0.52
Gemma-3-12B-IT [ 36 ]
43.37
54.98
0.39
40.45
52.80
0.40
42.37
52.84
0.41
42.06
53.54
0.40
Appendix
Table 12: Choice-format evaluation along the fire scenario task axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.
Model
T1
T2
T3
Average
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
43.18
34.09
0.28
23.58
20.16
0.47
45.22
40.07
0.28
37.33
31.44
0.34
Qwen3-8B [ 35 ]
45.54
37.86
0.36
47.15
39.07
0.32
46.50
43.15
0.36
46.40
40.03
0.35
Gemma-3-12B-IT [ 36 ]
48.26
37.67
0.29
30.89
26.61
0.46
41.83
34.20
0.32
40.33
32.83
0.36
Appendix
Table 13: Open-report evaluation along the fire scenario task axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.
Model
P1
P2
P3
P4
P5
Average
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
57.16
32.34
16.73
52.59
13.55
27.53
64.15
31.07
24.54
31.04
11.76
30.00
12.03
15.59
20.35
43.39
20.86
23.83
Qwen3-8B [ 35 ]
68.70
39.75
20.06
49.63
19.74
28.70
65.96
29.61
35.09
35.95
10.34
41.88
10.50
16.08
23.64
46.15
23.10
29.87
Gemma-3-12B-IT [ 36 ]
64.09
33.68
18.38
51.43
16.68
36.79
69.85
32.71
26.86
43.84
13.66
26.00
14.40
15.88
19.57
48.72
22.52
25.52
Appendix
Table 14: Open-report explanation metrics along the physical capability axis. Models are evaluated on the controlled simulation subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across P1–P5. Higher is better for all three metrics.
Model
T1
T2
T3
Average
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
46.56
12.70
19.10
47.66
27.58
25.80
40.98
17.85
14.30
45.07
19.38
19.73
Qwen3-8B [ 35 ]
46.53
17.17
22.60
51.38
23.35
35.00
50.69
24.86
19.50
49.53
21.79
25.70
Gemma-3-12B-IT [ 36 ]
52.96
14.91
23.80
54.63
26.63
22.00
49.54
23.67
18.20
52.38
21.74
21.33
Appendix
Table 15: Open-report explanation metrics along the fire scenario task axis. Models are evaluated on the controlled simulation subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across T1–T3. Higher is better for all three metrics.
Model
P1
P2
P3
P4
P5
Average
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
50.70
34.30
23.26
54.36
13.86
28.64
85.66
30.91
27.27
41.83
4.78
16.67
18.19
15.37
16.67
50.15
19.84
22.50
Qwen3-8B [ 35 ]
65.34
43.53
10.85
56.10
21.61
28.64
86.51
36.36
60.61
45.97
8.75
38.33
25.94
14.86
22.22
55.97
25.02
32.13
Gemma-3-12B-IT [ 36 ]
63.52
38.90
28.68
57.30
18.58
32.05
84.87
33.03
36.36
48.31
13.49
23.33
26.63
16.77
20.37
56.13
24.15
28.16
Appendix
Table 16: Open-report explanation metrics along the physical capability axis. Models are evaluated on the real-world-aligned subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across P1–P5. Higher is better for all three metrics.
Model
T1
T2
T3
Average
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
Evi-F1
Mech
GLS
S-track: structured sensor and physical records
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]
49.24
9.93
27.60
67.04
20.38
17.90
41.11
28.72
21.30
52.46
19.68
22.27
Qwen3-8B [ 35 ]
53.64
17.45
33.70
66.44
23.58
47.20
53.71
35.07
14.20
57.93
25.37
31.70
Gemma-3-12B-IT [ 36 ]
56.03
16.09
31.90
65.34
25.31
26.00
52.64
32.37
26.20
58.00
24.59
28.03
Appendix
Table 17: Open-report explanation metrics along the fire scenario task axis. Models are evaluated on the real-world-aligned subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across T1–T3. Higher is better for all three metrics.
Environment Family
Acc.
F1
Brier
Evi-F1
Mech
GLS
Transportation and Infrastructure
41.6
51.0
0.42
36.4
45.9
24.6
Office and Digital Facilities
41.0
45.8
0.37
41.6
23.9
23.7
Healthcare and Education
46.8
49.5
0.33
46.9
24.9
28.3
Commercial and Public Spaces
44.4
49.1
0.35
42.0
24.5
27.1
Residential and Care Settings
45.5
48.0
0.32
39.3
24.0
28.8
Industry, Energy, and Logistics
40.5
48.1
0.37
44.1
24.2
23.6
Appendix
Table 18: Open-report performance across environment families on the controlled simulation subset. Results are aggregated separately over the seven environment families defined in FireWorldBench. Acc., F1, Evi-F1, Mech, and GLS are reported as percentages and are higher-is-better, whereas Brier is reported on the [0,1] scale and lower is better. The best result in each metric is shown in bold.
Method
Llama-3.1-8B-Instruct
Qwen3-8B
InternVL3-8B
Qwen3-VL-8B
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Vanilla
24.4
34.9
0.90
30.5
40.9
0.86
22.9
31.8
0.52
28.2
37.5
0.84
+Prompt
28.2 ↑ 3.8
39.0 ↑ 4.1
0.97 ↑ 0.07
31.3 ↑ 0.8
41.3 ↑ 0.4
0.89 ↑ 0.03
23.8 ↑ 0.9
34.1 ↑ 2.3
0.48 ↓ 0.04
33.3 ↑ 5.1
43.1 ↑ 5.6
0.89 ↑ 0.05
+CoT
24.0 ↓ 0.4
34.2 ↓ 0.7
0.67 ↓ 0.23
32.7 ↑ 2.2
43.2 ↑ 2.3
0.82 ↓ 0.04
26.4 ↑ 3.5
35.2 ↑ 3.4
0.21 ↓ 0.31
30.3 ↑ 2.1
40.5 ↑ 3.0
0.87 ↑ 0.03
+SFT
42.6 ↑ 18.2
55.9 ↑ 21.0
0.95 ↑ 0.05
42.6 ↑ 12.1
52.2 ↑ 11.3
0.81 ↓ 0.05
40.2 ↑ 17.3
50.4 ↑ 18.6
0.87 ↑ 0.35
41.3 ↑ 13.1
51.2 ↑ 13.7
0.83 ↓ 0.01
Appendix
Table 19: FireWorldGPT results under multiple-choice evaluation. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.
Method
Llama-3.1-8B-Instruct
Qwen3-8B
InternVL3-8B
Qwen3-VL-8B
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Acc
F1
Brier
Vanilla
37.8
30.8
0.63
36.2
30.5
0.74
27.2
21.0
0.65
30.2
21.5
0.74
+Prompt
38.5 ↑ 0.7
29.9 ↓ 0.9
0.62 ↓ 0.01
39.7 ↑ 3.5
32.7 ↑ 2.2
0.75 ↑ 0.01
35.0 ↑ 7.8
24.7 ↑ 3.7
0.64 ↓ 0.01
35.7 ↑ 5.5
24.2 ↑ 2.7
0.66 ↓ 0.08
+CoT
40.3 ↑ 2.5
32.9 ↑ 2.1
0.72 ↑ 0.09
38.6 ↑ 2.4
30.9 ↑ 0.4
0.75 ↑ 0.01
29.9 ↑ 2.7
24.8 ↑ 3.8
0.69 ↑ 0.04
33.2 ↑ 3.0
27.2 ↑ 5.7
0.76 ↑ 0.02
+SFT
43.5 ↑ 5.7
33.2 ↑ 2.4
0.83 ↑ 0.20
38.9 ↑ 2.7
29.4 ↓ 1.1
0.87 ↑ 0.13
28.0 ↑ 0.8
24.2 ↑ 3.2
0.86 ↑ 0.21
40.6 ↑ 10.4
27.4 ↑ 5.9
0.81 ↑ 0.07
Appendix
Table 20: FireWorldGPT results under open-report evaluation. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.
Figure 11: P1: Temporal Evolution Forecasting. A matched choice-format and open-report case asks how the smoke state will evolve after the soot plume reaches the ceiling and the low-visibility region expands laterally from t=8 s to t=16 s. The Gold response predicts continued upper-layer spreading with further visibility degradation in adjacent ceiling regions. The representative GPT-5.6-Luna response reaches the same forecast and grounds it in the observed ceiling interaction, lateral soot expansion, and progressive visibility loss.
Figure 12: P2: Physical Field Perception and Grounding. A localized anomaly in a care-ward corridor is presented through a 3D scene together with temperature and soot-density fields. The task requires distinguishing a nearby linen-cart or combustible-material source from spatially broader alternatives such as ventilation or system-level failure. The representative GPT-5.6-Luna response localizes the source to the west corridor and supports this attribution using the spatial co-occurrence and rapid decay of the thermal and soot anomalies.
Figure 13: P3: Cross-Field Coupling Understanding. A kiosk fire in a public atrium is queried through its 3D structure, soot-density distribution, and velocity field. The task requires combining plume morphology with the flow state to forecast smoke transport under an unchanged source and an open upper exhaust route. The representative GPT-5.6-Luna response predicts continued plume rise and flow-guided lateral transport, linking buoyant ascent, ambient advection, and geometric confinement in a single coupled-field explanation.
Figure 14: P4: Causal Mechanism Attribution. A charging-bay fire produces a buoyant smoke plume that subsequently bends leftward along a service corridor under a persistent leftward airflow. The Gold response attributes the lateral deflection primarily to corridor crossflow, while distinguishing this mechanism from the initial buoyant rise. The representative GPT-5.6-Luna response reaches the same causal attribution, linking the upward plume development to buoyancy and the sustained lateral transport to corridor crossflow.
Figure 15: P5: Counterfactual Intervention Reasoning. A refinery pipe-rack fire is evaluated under the counterfactual intervention of earlier pump isolation. The task requires decomposing the intervention effect into changes in source strength, plume spread, and equipment exposure. Using the 3D scene, temperature field, and soot-density field, the representative GPT-5.6-Luna response predicts that earlier isolation would reduce continued fuel delivery, suppress subsequent plume development, and lower the thermal and contamination risks to adjacent equipment.
Figure 16: Transportation and Infrastructure: Electric Bus Terminal Charging-bay Fire. This controlled fire world shows the time-resolved development of a charging-bay fire through co-registered 3D scenes and temperature, soot-density, velocity, visibility, and streamwise-velocity fields. The terminal geometry makes the interaction between buoyant plume rise, bay-scale confinement, and longitudinal smoke transport directly observable.
Figure 17: Office and Digital Facilities: Data-centre Cold-aisle Cable-tray Fire. This controlled sequence depicts the growth and transport of heat and smoke from a cable-tray fire within a confined cold-aisle layout. The co-registered fields reveal how overhead structures and aisle geometry constrain the thermal plume, redistribute soot, modify local flow, and progressively degrade visibility.
Figure 18: Healthcare and Education: Teaching Medical Laboratory Solvent-bench Fire. A localized solvent-bench source develops into a vertically extended thermal and soot plume inside a laboratory compartment. The synchronized physical fields expose the resulting changes in plume velocity, streamwise transport, and visibility, while the 3D views preserve the spatial relationship between the source, laboratory fixtures, and surrounding enclosure.
Figure 19: Commercial and Public Spaces: Public Atrium Kiosk Fire. This controlled fire world captures plume growth from a kiosk source within a tall, structurally open atrium. The temporal sequence illustrates how buoyancy, upper-level geometry, and the surrounding flow field jointly determine plume rise, lateral smoke spreading, and the spatial progression of visibility loss.
Figure 20: Residential and Care Settings: Night-shift Care Ward Linen-cart Fire. A corridor-adjacent linen-cart fire is represented through its evolving 3D scene and five co-registered physical fields. The sequence highlights the localized origin of the thermal and soot anomalies, their subsequent extension through the connected ward geometry, and the associated deterioration of visibility along the transport path.
Figure 21: Industry, Energy, and Logistics: Integrated Refinery Pipe-rack Energy Fire. This controlled outdoor fire world places the source among pipe racks, processing units, and adjacent equipment. The multimodal sequence exposes the growth of the buoyant plume, the transport of heat and soot through the equipment layout, and the spatially varying velocity and visibility conditions that determine downstream exposure.
Figure 22: Wildland and Wildland–Urban Interface: Hillside WUI Evacuation-lane Fire. This controlled scene represents fire development near vegetation, structures, and a hillside evacuation route. The co-registered field sequence illustrates how terrain, built obstacles, and ambient flow shape the propagation of heat and smoke, producing spatially heterogeneous visibility and transport conditions along the wildland–urban interface.
Figure 23: Real-world-aligned event: Heptane Pool-fire Experiment. Unlike the preceding controlled worlds, this example begins with an observed experimental sequence. Recorded video frames anchor the event chronology and visible fire development, while the aligned 3D reconstruction and FDS realization provide simulation-completed temperature, soot-density, velocity, visibility, and streamwise-velocity fields at matched time points. The separate rows preserve the distinction between observed evidence and reconstructed latent physical state.
Multimodal large language models (MLLMs) are increasingly envisioned as core components of scientific and engineering agents, yet their ability to interpret physical fields remains poorly understood. Existing physics benchmarks largely emphasize textbook problem solving or intuitive physical reasoning, leaving open whether MLLMs can infer physically meaningful information from continuous field observations. We introduce PhysFieldBench, a benchmark comprising 24 tasks and 1,160 evaluation examples across controlled equation fields, simulated physical fields, and observed physical fields. The tasks assess three forms of inference: identifying physical mechanisms, comparing latent control variables, and predicting outcome properties. Across representative open-source and proprietary MLLMs, zero-shot performance is low: the best model achieves a chance-normalized score of 29.3, while several open-source models remain near chance. In contrast, a task-specific supervised vision transformer performs substantially better, demonstrating that the inputs contain learnable physical information. To diagnose these failures, a structured self-explanation analysis attributes most errors to missed visual patterns and incorrect visual-to-physical mappings. Further, to explore whether post-training can improve physical inference and generalize to unseen tasks, we compare supervised fine-tuning with final answers or chain-of-thought supervision and reinforcement learning. Final-answer supervision performs best overall but transfers less effectively, whereas reinforcement learning after chain-of-thought supervision achieves the best generalization. Together, these findings highlight the need to improve visual-to-physical grounding and cross-task generalization for MLLMs to reliably interpret physical fields in scientific and engineering workflows.
Yuezhou Ma, Huikun Weng, Jialong Wu +5
School of Software, BNRist, Tsinghua University, China
While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in image and video understanding, their ability to comprehend the physical world has become an increasingly important research focus. Despite their improvements, current MLLMs struggle significantly with high-level physics reasoning. In this work, we investigate the first step of physical reasoning, i.e., intuitive physics understanding, revealing substantial limitations in understanding the dynamics of continuum objects. To isolate and evaluate this specific capability, we introduce two fundamental benchmark tasks: Next Frame Selection (NFS) and Temporal Coherence Verification (TCV). Our experiments demonstrate that even state-of-the-art MLLMs perform poorly on these foundational tasks. To address this limitation, we propose Scene Dynamic Field (SDF), a concise approach that leverages physics simulators within a multi-task fine-tuning framework. SDF substantially improves performance, achieving up to 20.7% gains on fluid tasks while showing strong generalization to unseen physical domains. This work not only highlights a critical gap in current MLLMs but also presents a promising cost-efficient approach for developing more physically grounded MLLMs. Our code and data are available at https://github.com/andylinx/Scene-Dynamic-Field.
Nanxi Li, Xiang Wang, Yuanjie Chen +3
Shanghai Jiao Tong University · Tianjin University · Shanghai Innovation Institute
Physical understanding and reasoning depend on forming compact and generalizable representations of the world. While modern vision-language models can recognize and explain diverse physical events, they often lack explicit representations of the underlying mechanisms-such as object states, physical parameters, and governing dynamics-needed for reliably reasoning how the world evolves and responds to interventions. In this work, we introduce Code-as-World, a paradigm that represents physical worlds through executable world representations. By expressing physical composition, dynamic evolution, and visual appearance as executable code, Code-as-World provides a compact, quantitatively grounded, and controllable abstraction of the physical world. To construct such representations from multimodal observations, such as natural-language descriptions or real-world videos, we develop an agentic discovery loop inspired by abductive reasoning, where an agent proposes, executes, renders, verifies, and iteratively refines executable world hypotheses. As a concrete application, we use verified executable worlds to provide scalable physical supervision for training vision-language models on quantitative physical reasoning. Experiments show that Code-as-World-VL achieves state-of-the-art performance on QuantiPhy and surpasses leading proprietary models, highlighting the potential of executable world representations as a scalable foundation for physical intelligence.