FireWorldBench: Benchmarking Complex Physical World Intelligence through Coupled-Field Fire Dynamics

Qiang Chen; Hao Guo; Huatai Zhu; Tairan Huang; Yichao Cao; Hongyan Xu; Keke Huang; Haifeng Li

cs.AISep 19, 2026

FireWorldBench: Benchmarking Complex Physical World Intelligence through Coupled-Field Fire Dynamics

Authors: Qiang Chen, Hao Guo, Huatai Zhu, Tairan Huang, Yichao Cao, Hongyan Xu, Keke Huang, Haifeng Li,

Organizations: Hong Kong University of Science and Technology · Central South University

Abstract

Understanding the physical world requires more than object recognition, scene description, and short-term visual prediction, as real-world physical systems involve multiple continuous fields, latent causal mechanisms, partial observations, and intervention-sensitive dynamics. We propose FireWorldBench, a benchmark for evaluating complex physical world intelligence in multimodal large language models and agents through coupled-field fire dynamics. Fire provides a canonical stress-test environment, where multiple interacting physical fields jointly shape observable states and temporal dynamics. FireWorldBench is organized along two complementary axes, a physical capability axis and a fire scenario task axis, jointly covering physical-state understanding, temporal dynamics, causal mechanisms, and intervention reasoning. The benchmark comprises 520 fire-world entries, including 494 controlled simulation worlds and 26 real-world-aligned event groups, spanning 47 scene archetypes across 7 environment families. These entries combine structured textual observations, multiple 2D physical-field visualizations, and 3D event-level scene modeling, yielding 9,074 text-image interleaved question-answer pairs across choice-based and open-ended report-generation formats. FireWorldBench evaluates whether models can infer latent physical states, explain underlying mechanisms, forecast coupled-field evolution, and assess intervention consequences from multimodal partial observations, providing a challenging testbed for complex physical world intelligence.

Figures & tables

Figure 2: Overview of Dataset. The left sunburst summarizes the hierarchical composition of 520 fire incidents, while the right panels characterize the coverage of 8,360 simulated and 714 real-world-aligned QA pairs along the five physical capability axis (P1–P5) and fire scenario task axis (T1–T3).

Figure 3: Physical field perception and grounding. The same fire-world state is queried through choice-format and open-report interfaces using co-registered 3D scene, temperature-field, and soot-density observations. Both interfaces share the same evidence and physical target, while the open report additionally requires an explicit conclusion, supporting evidence, and causal mechanism.

Figure 4: Construction pipeline of FireWorldBench. Designed scenarios and multimodal records of observed experimental fires are encoded as event constraints and instantiated with FDS. Smokeview produces time-indexed 3D renderings, while the solver state yields aligned physical-field observations. These co-registered outputs are subsequently converted into multimodal question–answer pairs with held-out annotations derived from the underlying physical state.

Figure 5: Real-world-aligned fire-world reconstruction. For a heptane pool-fire experiment, observed video frames anchor the event chronology, while the aligned 3D reconstruction and corresponding FDS realization provide simulation-completed temperature, soot-density, velocity, visibility, and streamwise-velocity fields at matched time points. The sequence preserves the distinction between observed evidence and reconstructed latent physical state.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	31.93	42.55	0.44	31.35	39.79	0.49	28.38	39.19	0.44	37.16	40.09	0.42	29.38	38.81	0.42	31.64	40.09	0.44
Qwen3-8B [ 35 ]	46.55	56.68	0.31	35.34	42.53	0.48	33.88	45.05	0.38	42.98	45.58	0.37	31.58	39.89	0.44	38.07	45.95	0.40
Gemma-3-12B-IT [ 36 ]	34.67	44.06	0.45	38.57	48.96	0.42	33.30	44.88	0.42	34.03	36.60	0.47	27.89	36.01	0.51	33.70	42.10	0.46

Table 1: Overall evaluation along the physical capability axis on the controlled simulation subset. Models are evaluated by averaging the corresponding choice-format and open-report scores under the same observation track, capability dimension, and metric. Each capability dimension reports accuracy (Acc), macro-F1 (F1), and Brier score, with the Average column summarizing performance across P1–P5.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	32.55	40.39	0.46	29.81	39.07	0.45	32.57	41.68	0.43	31.64	40.38	0.45
Qwen3-8B [ 35 ]	37.53	43.84	0.44	35.61	45.19	0.39	44.50	52.78	0.33	39.21	47.27	0.39
Gemma-3-12B-IT [ 36 ]	37.82	46.69	0.43	29.27	38.84	0.47	34.58	42.19	0.46	33.89	42.57	0.46

Table 2: Overall evaluation along the fire scenario task axis on the controlled simulation subset. Models are evaluated by averaging matched choice-format and open-report scores under the same simulated fire-world entry, observation track, task dimension, and metric. Each task reports accuracy (Acc), macro-F1 (F1), and Brier score, with Average summarizing performance across T1–T3.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	35.16	43.98	0.46	30.85	39.77	0.48	32.95	41.36	0.43	29.25	37.84	0.46	34.49	44.73	0.36	32.54	41.53	0.44
Qwen3-8B [ 35 ]	37.95	45.41	0.49	35.06	43.30	0.49	48.86	57.95	0.31	39.06	48.25	0.42	34.49	45.00	0.41	39.09	47.98	0.42
Gemma-3-12B-IT [ 36 ]	43.73	52.13	0.35	44.34	55.99	0.34	39.43	47.23	0.38	40.54	49.12	0.41	38.20	47.50	0.40	41.25	50.40	0.38

Table 3: Overall evaluation along the physical capability axis on the real-world-aligned subset. Models are evaluated by aggregating the corresponding choice-format and open-report results under the same real-world-aligned event groups, observation track, capability dimension, and metric. Each capability dimension reports accuracy (Acc), macro-F1 (F1), and Brier score, with the Average column summarizing performance across P1–P5.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	33.84	34.05	0.44	25.69	30.04	0.50	34.96	37.76	0.43	31.50	33.95	0.46
Qwen3-8B [ 35 ]	39.12	40.15	0.45	39.94	41.73	0.39	36.93	41.02	0.47	38.66	40.97	0.44
Gemma-3-12B-IT [ 36 ]	45.82	46.33	0.34	35.67	39.71	0.43	42.10	43.52	0.37	41.20	43.18	0.38

Table 4: Overall evaluation along the fire scenario task axis on the real-world-aligned subset. Models are evaluated by averaging matched choice-format and open-report scores under the same real-world-aligned event group, observation track, task dimension, and metric. Each task reports accuracy (Acc), macro-F1 (F1), and Brier score, with Average summarizing performance across T1–T3.

Method	Llama-3.1-8B-Instruct			Qwen3-8B			InternVL3-8B			Qwen3-VL-8B
Method	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Vanilla	31.1	32.9	0.77	33.4	35.7	0.80	25.1	26.4	0.59	29.2	29.5	0.79
+Prompt	33.4 $\uparrow$ 2.3	34.5 $\uparrow$ 1.6	0.80 $\uparrow$ 0.03	35.5 $\uparrow$ 2.1	37.0 $\uparrow$ 1.3	0.82 $\uparrow$ 0.02	29.4 $\uparrow$ 4.3	29.4 $\uparrow$ 3.0	0.56 $\downarrow$ 0.03	34.5 $\uparrow$ 5.3	33.7 $\uparrow$ 4.2	0.78 $\downarrow$ 0.01
+CoT	32.2 $\uparrow$ 1.1	33.6 $\uparrow$ 0.7	0.70 $\downarrow$ 0.07	35.7 $\uparrow$ 2.3	37.1 $\uparrow$ 1.4	0.79 $\downarrow$ 0.01	28.2 $\uparrow$ 3.1	30.0 $\uparrow$ 3.6	0.45 $\downarrow$ 0.14	31.8 $\uparrow$ 2.6	33.9 $\uparrow$ 4.4	0.82 $\uparrow$ 0.03
+SFT	43.1 $\uparrow$ 12.0	44.6 $\uparrow$ 11.7	0.89 $\uparrow$ 0.12	40.8 $\uparrow$ 7.4	40.8 $\uparrow$ 5.1	0.84 $\uparrow$ 0.04	34.1 $\uparrow$ 9.0	37.3 $\uparrow$ 10.9	0.87 $\uparrow$ 0.28	41.0 $\uparrow$ 11.8	39.3 $\uparrow$ 9.8	0.82 $\uparrow$ 0.03

Table 5: FireWorldGPT results on controlled simulations. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.

Figure 6: Physical capability profiles and perception–mechanism gaps. Results on the controlled-simulation (top) and real-world-aligned (bottom) subsets, each comprising 25 equally weighted model–track results. (a,d) Performance across P1–P5;(b,e) Cross-field coupling and (c,f) mechanism attribution versus field perception. Mean gaps measure field perception minus coupling or mechanism accuracy in percentage points, circles and triangles denote S-track and I-track results, respectively.

Figure 7: Paired performance gaps between structured and multimodal observations. (a) Accuracy and (b) Macro-F1 differences across the physical capability and fire-task axes on the controlled and real-world-aligned subsets. Gaps are computed as S-track minus I-track in percentage points using format-averaged overall scores; positive values favor structured observations.

Figure 8: Active-parameter scaling yields limited returns in complex physical-world understanding. (a) Format-averaged I-track accuracy across Qwen3-VL and InternVL3 checkpoints, with solid and dashed lines denoting controlled and real-world-aligned settings, respectively. (b) Capability-wise differences show a broad advantage of the 32B model over the 22B-active checkpoint.

Figure 9: Physical correctness and confidence quality. Completion accuracy versus the benchmark’s Brier-style score on the physical capability axis for (a) controlled simulation and (b) real-world-aligned data. Circles and triangles denote S-track and I-track results. Dashed segments connect illustrative similar-accuracy contrasts.

Figure 10: Question quality and expert–Gold agreement. (a) Expert-assessed question-quality and leakage-check rates on 300 controlled-simulation QA instances. (b) Expert acceptance of Gold answers and mechanisms, alongside exact agreement between expert responses and Gold at the answer and output-slot levels. Circles denote rates from the original audit, while diamonds denote expert-reported recheck rates. Horizontal error bars show nominal 95% Wilson confidence intervals.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	23.33	34.36	0.59	22.21	31.01	0.65	23.52	34.43	0.58	28.83	40.15	0.52	26.59	37.92	0.49	24.90	35.57	0.56
Qwen3-8B [ 35 ]	38.40	49.22	0.33	30.75	38.93	0.55	27.70	38.56	0.47	39.31	50.16	0.39	33.70	44.03	0.44	33.97	44.18	0.44
Gemma-3-12B-IT [ 36 ]	22.14	31.59	0.64	32.70	43.56	0.52	32.36	45.62	0.48	31.25	42.09	0.53	28.65	38.52	0.57	29.42	40.28	0.55

Appendix

Table 6: Choice-format evaluation along the physical capability axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	40.53	50.73	0.30	40.49	48.57	0.33	33.25	43.95	0.30	45.48	40.02	0.32	32.17	39.71	0.36	38.38	44.60	0.32
Qwen3-8B [ 35 ]	54.71	64.14	0.29	39.94	46.13	0.42	40.05	51.53	0.29	46.64	40.99	0.35	29.46	35.76	0.43	42.16	47.71	0.35
Gemma-3-12B-IT [ 36 ]	47.21	56.54	0.26	44.44	54.36	0.32	34.25	44.15	0.36	36.82	31.11	0.41	27.13	33.49	0.45	37.97	43.93	0.36

Appendix

Table 7: Open-report evaluation along the physical capability axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	24.67	–	0.59	25.86	–	0.56	24.09	–	0.57	24.87	–	0.57
Qwen3-8B [ 35 ]	33.76	–	0.50	32.74	–	0.42	37.31	–	0.35	34.60	–	0.43
Gemma-3-12B-IT [ 36 ]	32.84	–	0.52	31.19	–	0.51	23.65	–	0.62	29.23	–	0.55

Appendix

Table 8: Choice-format evaluation along the fire scenario task axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	40.42	46.22	0.33	33.75	41.26	0.34	41.05	48.17	0.30	38.41	45.22	0.32
Qwen3-8B [ 35 ]	41.30	45.11	0.38	38.47	46.31	0.36	51.69	57.54	0.30	43.82	49.65	0.35
Gemma-3-12B-IT [ 36 ]	42.79	49.67	0.35	27.35	34.18	0.44	45.50	51.18	0.29	38.55	45.01	0.36

Appendix

Table 9: Open-report evaluation along the fire scenario task axis. Models are evaluated on the controlled simulation subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	20.89	30.74	0.66	22.16	30.93	0.65	28.03	40.00	0.51	28.50	40.17	0.52	33.80	46.67	0.41	26.68	37.70	0.55
Qwen3-8B [ 35 ]	23.10	33.78	0.63	32.40	40.87	0.54	37.12	48.18	0.41	30.62	43.00	0.52	37.50	51.11	0.43	32.15	43.39	0.50
Gemma-3-12B-IT [ 36 ]	42.08	52.41	0.42	43.92	55.69	0.39	37.95	49.00	0.43	42.75	55.25	0.39	43.06	53.89	0.39	41.95	53.25	0.40

Appendix

Table 10: Choice-format evaluation along the physical capability axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	49.42	57.22	0.26	39.55	48.61	0.32	37.88	42.73	0.36	30.00	35.50	0.40	35.19	42.78	0.32	38.41	45.37	0.33
Qwen3-8B [ 35 ]	52.79	57.04	0.35	37.73	45.72	0.43	60.61	67.73	0.20	47.50	53.50	0.33	31.48	38.89	0.39	46.02	52.58	0.34
Gemma-3-12B-IT [ 36 ]	45.39	51.86	0.29	44.77	56.30	0.29	40.91	45.45	0.34	38.33	43.00	0.44	33.33	41.11	0.40	40.55	47.54	0.35

Appendix

Table 11: Open-report evaluation along the physical capability axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across P1–P5.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	24.49	34.01	0.60	27.80	39.92	0.53	24.70	35.44	0.58	25.66	36.46	0.57
Qwen3-8B [ 35 ]	32.70	42.44	0.53	32.72	44.39	0.47	27.35	38.89	0.57	30.92	41.91	0.52
Gemma-3-12B-IT [ 36 ]	43.37	54.98	0.39	40.45	52.80	0.40	42.37	52.84	0.41	42.06	53.54	0.40

Appendix

Table 12: Choice-format evaluation along the fire scenario task axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	43.18	34.09	0.28	23.58	20.16	0.47	45.22	40.07	0.28	37.33	31.44	0.34
Qwen3-8B [ 35 ]	45.54	37.86	0.36	47.15	39.07	0.32	46.50	43.15	0.36	46.40	40.03	0.35
Gemma-3-12B-IT [ 36 ]	48.26	37.67	0.29	30.89	26.61	0.46	41.83	34.20	0.32	40.33	32.83	0.36

Appendix

Table 13: Open-report evaluation along the fire scenario task axis. Models are evaluated on the real-world-aligned subset under the structured-observation track (S-track) and multimodal image-observation track (I-track). Each dimension reports accuracy (Acc), macro-F1 (F1), and Brier score; Average is the equal-weight mean across T1–T3.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	57.16	32.34	16.73	52.59	13.55	27.53	64.15	31.07	24.54	31.04	11.76	30.00	12.03	15.59	20.35	43.39	20.86	23.83
Qwen3-8B [ 35 ]	68.70	39.75	20.06	49.63	19.74	28.70	65.96	29.61	35.09	35.95	10.34	41.88	10.50	16.08	23.64	46.15	23.10	29.87
Gemma-3-12B-IT [ 36 ]	64.09	33.68	18.38	51.43	16.68	36.79	69.85	32.71	26.86	43.84	13.66	26.00	14.40	15.88	19.57	48.72	22.52	25.52

Appendix

Table 14: Open-report explanation metrics along the physical capability axis. Models are evaluated on the controlled simulation subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across P1–P5. Higher is better for all three metrics.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	46.56	12.70	19.10	47.66	27.58	25.80	40.98	17.85	14.30	45.07	19.38	19.73
Qwen3-8B [ 35 ]	46.53	17.17	22.60	51.38	23.35	35.00	50.69	24.86	19.50	49.53	21.79	25.70
Gemma-3-12B-IT [ 36 ]	52.96	14.91	23.80	54.63	26.63	22.00	49.54	23.67	18.20	52.38	21.74	21.33

Appendix

Table 15: Open-report explanation metrics along the fire scenario task axis. Models are evaluated on the controlled simulation subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across T1–T3. Higher is better for all three metrics.

S-track: structured sensor and physical records
Model	P1			P2			P3			P4			P5			Average
Model	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	50.70	34.30	23.26	54.36	13.86	28.64	85.66	30.91	27.27	41.83	4.78	16.67	18.19	15.37	16.67	50.15	19.84	22.50
Qwen3-8B [ 35 ]	65.34	43.53	10.85	56.10	21.61	28.64	86.51	36.36	60.61	45.97	8.75	38.33	25.94	14.86	22.22	55.97	25.02	32.13
Gemma-3-12B-IT [ 36 ]	63.52	38.90	28.68	57.30	18.58	32.05	84.87	33.03	36.36	48.31	13.49	23.33	26.63	16.77	20.37	56.13	24.15	28.16

Appendix

Table 16: Open-report explanation metrics along the physical capability axis. Models are evaluated on the real-world-aligned subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across P1–P5. Higher is better for all three metrics.

S-track: structured sensor and physical records
Model	T1			T2			T3			Average
Model	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS	Evi-F1	Mech	GLS
Open-weight models: fewer than 20B total parameters
Llama-3.1-8B-Instruct [ 34 ]	49.24	9.93	27.60	67.04	20.38	17.90	41.11	28.72	21.30	52.46	19.68	22.27
Qwen3-8B [ 35 ]	53.64	17.45	33.70	66.44	23.58	47.20	53.71	35.07	14.20	57.93	25.37	31.70
Gemma-3-12B-IT [ 36 ]	56.03	16.09	31.90	65.34	25.31	26.00	52.64	32.37	26.20	58.00	24.59	28.03

Appendix

Table 17: Open-report explanation metrics along the fire scenario task axis. Models are evaluated on the real-world-aligned subset. Each dimension reports Evidence-F1 (Evi-F1), Mechanism Alignment (Mech), and Gold-Linked Support (GLS); Average is the equal-weight mean across T1–T3. Higher is better for all three metrics.

Environment Family	Acc.	F1	Brier	Evi-F1	Mech	GLS
Transportation and Infrastructure	41.6	51.0	0.42	36.4	45.9	24.6
Office and Digital Facilities	41.0	45.8	0.37	41.6	23.9	23.7
Healthcare and Education	46.8	49.5	0.33	46.9	24.9	28.3
Commercial and Public Spaces	44.4	49.1	0.35	42.0	24.5	27.1
Residential and Care Settings	45.5	48.0	0.32	39.3	24.0	28.8
Industry, Energy, and Logistics	40.5	48.1	0.37	44.1	24.2	23.6

Appendix

Table 18: Open-report performance across environment families on the controlled simulation subset. Results are aggregated separately over the seven environment families defined in FireWorldBench. Acc., F1, Evi-F1, Mech, and GLS are reported as percentages and are higher-is-better, whereas Brier is reported on the

[0,1]

scale and lower is better. The best result in each metric is shown in bold.

Method	Llama-3.1-8B-Instruct			Qwen3-8B			InternVL3-8B			Qwen3-VL-8B
Method	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Vanilla	24.4	34.9	0.90	30.5	40.9	0.86	22.9	31.8	0.52	28.2	37.5	0.84
+Prompt	28.2 $\uparrow$ 3.8	39.0 $\uparrow$ 4.1	0.97 $\uparrow$ 0.07	31.3 $\uparrow$ 0.8	41.3 $\uparrow$ 0.4	0.89 $\uparrow$ 0.03	23.8 $\uparrow$ 0.9	34.1 $\uparrow$ 2.3	0.48 $\downarrow$ 0.04	33.3 $\uparrow$ 5.1	43.1 $\uparrow$ 5.6	0.89 $\uparrow$ 0.05
+CoT	24.0 $\downarrow$ 0.4	34.2 $\downarrow$ 0.7	0.67 $\downarrow$ 0.23	32.7 $\uparrow$ 2.2	43.2 $\uparrow$ 2.3	0.82 $\downarrow$ 0.04	26.4 $\uparrow$ 3.5	35.2 $\uparrow$ 3.4	0.21 $\downarrow$ 0.31	30.3 $\uparrow$ 2.1	40.5 $\uparrow$ 3.0	0.87 $\uparrow$ 0.03
+SFT	42.6 $\uparrow$ 18.2	55.9 $\uparrow$ 21.0	0.95 $\uparrow$ 0.05	42.6 $\uparrow$ 12.1	52.2 $\uparrow$ 11.3	0.81 $\downarrow$ 0.05	40.2 $\uparrow$ 17.3	50.4 $\uparrow$ 18.6	0.87 $\uparrow$ 0.35	41.3 $\uparrow$ 13.1	51.2 $\uparrow$ 13.7	0.83 $\downarrow$ 0.01

Appendix

Table 19: FireWorldGPT results under multiple-choice evaluation. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.

Method	Llama-3.1-8B-Instruct			Qwen3-8B			InternVL3-8B			Qwen3-VL-8B
Method	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier	Acc	F1	Brier
Vanilla	37.8	30.8	0.63	36.2	30.5	0.74	27.2	21.0	0.65	30.2	21.5	0.74
+Prompt	38.5 $\uparrow$ 0.7	29.9 $\downarrow$ 0.9	0.62 $\downarrow$ 0.01	39.7 $\uparrow$ 3.5	32.7 $\uparrow$ 2.2	0.75 $\uparrow$ 0.01	35.0 $\uparrow$ 7.8	24.7 $\uparrow$ 3.7	0.64 $\downarrow$ 0.01	35.7 $\uparrow$ 5.5	24.2 $\uparrow$ 2.7	0.66 $\downarrow$ 0.08
+CoT	40.3 $\uparrow$ 2.5	32.9 $\uparrow$ 2.1	0.72 $\uparrow$ 0.09	38.6 $\uparrow$ 2.4	30.9 $\uparrow$ 0.4	0.75 $\uparrow$ 0.01	29.9 $\uparrow$ 2.7	24.8 $\uparrow$ 3.8	0.69 $\uparrow$ 0.04	33.2 $\uparrow$ 3.0	27.2 $\uparrow$ 5.7	0.76 $\uparrow$ 0.02
+SFT	43.5 $\uparrow$ 5.7	33.2 $\uparrow$ 2.4	0.83 $\uparrow$ 0.20	38.9 $\uparrow$ 2.7	29.4 $\downarrow$ 1.1	0.87 $\uparrow$ 0.13	28.0 $\uparrow$ 0.8	24.2 $\uparrow$ 3.2	0.86 $\uparrow$ 0.21	40.6 $\uparrow$ 10.4	27.4 $\uparrow$ 5.9	0.81 $\uparrow$ 0.07

Appendix

Table 20: FireWorldGPT results under open-report evaluation. Rows compare Vanilla, physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning across four foundation models. Colored annotations report changes relative to Vanilla: green denotes performance improvement and red denotes degradation, while arrows indicate the numerical direction of change. Acc. and F1 are higher-is-better, whereas lower Brier is better.

Figure 11: P1: Temporal Evolution Forecasting. A matched choice-format and open-report case asks how the smoke state will evolve after the soot plume reaches the ceiling and the low-visibility region expands laterally from $t=8$ s to $t=16$ s. The Gold response predicts continued upper-layer spreading with further visibility degradation in adjacent ceiling regions. The representative GPT-5.6-Luna response reaches the same forecast and grounds it in the observed ceiling interaction, lateral soot expansion, and progressive visibility loss.

Figure 12: P2: Physical Field Perception and Grounding. A localized anomaly in a care-ward corridor is presented through a 3D scene together with temperature and soot-density fields. The task requires distinguishing a nearby linen-cart or combustible-material source from spatially broader alternatives such as ventilation or system-level failure. The representative GPT-5.6-Luna response localizes the source to the west corridor and supports this attribution using the spatial co-occurrence and rapid decay of the thermal and soot anomalies.

Figure 13: P3: Cross-Field Coupling Understanding. A kiosk fire in a public atrium is queried through its 3D structure, soot-density distribution, and velocity field. The task requires combining plume morphology with the flow state to forecast smoke transport under an unchanged source and an open upper exhaust route. The representative GPT-5.6-Luna response predicts continued plume rise and flow-guided lateral transport, linking buoyant ascent, ambient advection, and geometric confinement in a single coupled-field explanation.

Figure 14: P4: Causal Mechanism Attribution. A charging-bay fire produces a buoyant smoke plume that subsequently bends leftward along a service corridor under a persistent leftward airflow. The Gold response attributes the lateral deflection primarily to corridor crossflow, while distinguishing this mechanism from the initial buoyant rise. The representative GPT-5.6-Luna response reaches the same causal attribution, linking the upward plume development to buoyancy and the sustained lateral transport to corridor crossflow.

Figure 15: P5: Counterfactual Intervention Reasoning. A refinery pipe-rack fire is evaluated under the counterfactual intervention of earlier pump isolation. The task requires decomposing the intervention effect into changes in source strength, plume spread, and equipment exposure. Using the 3D scene, temperature field, and soot-density field, the representative GPT-5.6-Luna response predicts that earlier isolation would reduce continued fuel delivery, suppress subsequent plume development, and lower the thermal and contamination risks to adjacent equipment.

Figure 16: Transportation and Infrastructure: Electric Bus Terminal Charging-bay Fire. This controlled fire world shows the time-resolved development of a charging-bay fire through co-registered 3D scenes and temperature, soot-density, velocity, visibility, and streamwise-velocity fields. The terminal geometry makes the interaction between buoyant plume rise, bay-scale confinement, and longitudinal smoke transport directly observable.

Figure 17: Office and Digital Facilities: Data-centre Cold-aisle Cable-tray Fire. This controlled sequence depicts the growth and transport of heat and smoke from a cable-tray fire within a confined cold-aisle layout. The co-registered fields reveal how overhead structures and aisle geometry constrain the thermal plume, redistribute soot, modify local flow, and progressively degrade visibility.

Figure 18: Healthcare and Education: Teaching Medical Laboratory Solvent-bench Fire. A localized solvent-bench source develops into a vertically extended thermal and soot plume inside a laboratory compartment. The synchronized physical fields expose the resulting changes in plume velocity, streamwise transport, and visibility, while the 3D views preserve the spatial relationship between the source, laboratory fixtures, and surrounding enclosure.

Figure 19: Commercial and Public Spaces: Public Atrium Kiosk Fire. This controlled fire world captures plume growth from a kiosk source within a tall, structurally open atrium. The temporal sequence illustrates how buoyancy, upper-level geometry, and the surrounding flow field jointly determine plume rise, lateral smoke spreading, and the spatial progression of visibility loss.

Figure 20: Residential and Care Settings: Night-shift Care Ward Linen-cart Fire. A corridor-adjacent linen-cart fire is represented through its evolving 3D scene and five co-registered physical fields. The sequence highlights the localized origin of the thermal and soot anomalies, their subsequent extension through the connected ward geometry, and the associated deterioration of visibility along the transport path.

Figure 21: Industry, Energy, and Logistics: Integrated Refinery Pipe-rack Energy Fire. This controlled outdoor fire world places the source among pipe racks, processing units, and adjacent equipment. The multimodal sequence exposes the growth of the buoyant plume, the transport of heat and soot through the equipment layout, and the spatially varying velocity and visibility conditions that determine downstream exposure.

Figure 22: Wildland and Wildland–Urban Interface: Hillside WUI Evacuation-lane Fire. This controlled scene represents fire development near vegetation, structures, and a hillside evacuation route. The co-registered field sequence illustrates how terrain, built obstacles, and ambient flow shape the propagation of heat and smoke, producing spatially heterogeneous visibility and transport conditions along the wildland–urban interface.

Figure 23: Real-world-aligned event: Heptane Pool-fire Experiment. Unlike the preceding controlled worlds, this example begins with an observed experimental sequence. Recorded video frames anchor the event chronology and visible fire development, while the aligned 3D reconstruction and FDS realization provide simulation-completed temperature, soot-density, velocity, visibility, and streamwise-velocity fields at matched time points. The separate rows preserve the distinction between observed evidence and reconstructed latent physical state.

Physical Reasoning

Physical reasoning studies how AI systems infer object properties, forces, and causal dynamics to predict and act in the physical world.

See all 69 papers

PhysFieldBench: Can Multimodal Models Understand Physical Fields?
Sep 28, 2026Yuezhou Ma, Huikun Weng, Jialong Wu +5Physical Reasoning Multimodal Large Language Models
School of Software, BNRist, Tsinghua University, China
Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models
Mar 30, 2026Nanxi Li, Xiang Wang, Yuanjie Chen +3Physical Reasoning Multimodal Large Language Models
Shanghai Jiao Tong University · Tianjin University · Shanghai Innovation Institute
Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning
Aug 27, 2026Hanyang Wang, Yimo Cai, Weiliang Chen +14Code World Models World Models

Sep 28, 2026J/K move · Enter open · S save

PhysFieldBench: Can Multimodal Models Understand Physical Fields?

Yuezhou Ma, Huikun Weng, Jialong Wu, Chenyi Zhao+4

School of Software, BNRist, Tsinghua University, China

Physical Reasoning Multimodal Large Language Models Scientific Reasoning in Language Models Multimodal Model Evaluation

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) are increasingly envisioned as core components of scientific and engineering agents, yet their ability to interpret physical fields remains poorly understood. Existing physics benchmarks largely emphasize textbook problem solving or intuitive physical reasoning, leaving open whether MLLMs can infer physically meaningful information from continuous field observations. We introduce PhysFieldBench, a benchmark comprising 24 tasks and 1,160 evaluation examples across controlled equation fields, simulated physical fields, and observed physical fields. The tasks assess three forms of inference: identifying physical mechanisms, comparing latent control variables, and predicting outcome properties. Across representative open-source and proprietary MLLMs, zero-shot performance is low: the best model achieves a chance-normalized score of 29.3, while several open-source models remain near chance. In contrast, a task-specific supervised vision transformer performs substantially better, demonstrating that the inputs contain learnable physical information. To diagnose these failures, a structured self-explanation analysis attributes most errors to missed visual patterns and incorrect visual-to-physical mappings. Further, to explore whether post-training can improve physical inference and generalize to unseen tasks, we compare supervised fine-tuning with final answers or chain-of-thought supervision and reinforcement learning. Final-answer supervision performs best overall but transfers less effectively, whereas reinforcement learning after chain-of-thought supervision achieves the best generalization. Together, these findings highlight the need to improve visual-to-physical grounding and cross-task generalization for MLLMs to reliably interpret physical fields in scientific and engineering workflows.

Abstract

Physical Reasoning

Explore similar work

PhysFieldBench: Can Multimodal Models Understand Physical Fields?

Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models

Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning