Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous spatiotemporal dynamics governed by Partial Differential Equations (PDEs), often resulting in non-physical hallucinations. Existing approaches typically resort to costly, domain-specific fine-tuning, which severely limits cross-domain generalization and interpretability. To bridge this gap, we propose OMNIFLOW, a neuro-symbolic architecture designed to ground frozen multimodal LLMs in fundamental physical laws without requiring domain-specific parameter updates. OMNIFLOW introduces a novel Semantic-Symbolic Alignment mechanism that projects high-dimensional flow tensors into topological linguistic descriptors, enabling the model to perceive physical structures rather than raw pixel values. Furthermore, we construct a Physics-Guided Chain-of-Thought (PG-CoT) workflow that orchestrates reasoning through dynamic constraint injection (e.g., mass conservation) and iterative reflexive verification. We evaluate OMNIFLOW on a comprehensive benchmark spanning microscopic turbulence, theoretical Navier-Stokes equations, and macroscopic global weather forecasting. Empirical results demonstrate that OMNIFLOW significantly outperforms traditional deep learning baselines in zero-shot generalization and few-shot adaptation tasks. Crucially, it offers transparent, physically consistent reasoning reports, marking a paradigm shift from black-box fitting to interpretable scientific reasoning.
Figures & tables
Figure 1: Comparison of reasoning paradigms. Traditional models (top) are non-interpretable black-boxes. General VLMs (middle) suffer from physical hallucinations. OmniFlow (bottom) integrates a Symbolic Lens and Consistency Loop to deliver grounded forecasts and expert reports, uniting numerical precision with logical reasoning.
Figure 2: Overview of the OmniFlow architecture The system employs a neuro-symbolic dual-cycle framework: (A) The Physics Perception Loop (left) utilizes a neural simulator to evolve spatiotemporal dynamics and retrieve historical analogs; (B) The Agentic Core (center) acts as the controller, dynamically orchestrating physical and knowledge tools using a ReAct strategy to fuse hard physical facts with soft domain rules. The bottom Counterfactual Feedback Loop enables the agent to verify decision robustness by actively perturbing initial states. (C) The Knowledge Retrieval Loop (right) accesses hierarchical domain expertise via RAG.
2D Turbulence (Micro)
ERA5 (Global)
SEVIR (Regional)
Model
RMSE ↓
SSIM ↑
PSNR ↑
RMSE ↓
SSIM ↑
PSNR ↑
RMSE ↓
SSIM ↑
PSNR ↑
Monolithic Foundation Models (Zero-shot Image Generation)
Banana Pro
118.4 ± 5.2
0.142 ± .02
8.12 ± .4
102.5 ± 4.8
0.194 ± .03
8.85 ± .3
108.6 ± 5.5
0.165 ± .02
8.40 ± .5
Seedream 4.5
98.2 ± 4.1
0.265 ± .03
9.85 ± .5
85.2 ± 3.9
0.352 ± .04
10.15 ± .4
90.4 ± 4.2
0.312 ± .03
9.92 ± .4
ChatGPT-Images
112.5 ± 6.3
0.185 ± .02
8.70 ± .6
96.4 ± 5.1
0.228 ± .03
9.42 ± .5
101.3 ± 5.8
0.205 ± .02
9.15 ± .6
OMNIFLOW
64.32 ± 2.5
0.552 ± .04
11.45 ± .3
59.10 ± 1.8
0.685 ± .03
12.70 ± .2
52.45 ± 1.2
0.712 ± .04
13.82 ± .3
Table 2: Zero-shot forecasting comparison: Monolithic Foundation Models vs. OMNIFLOW. Results are reported as mean ± std based on 50 samples evaluated in the PNG pixel space ( 0−255 ). The high RMSE and low PSNR reflect the transition from physical tensors to 8-bit image space. OMNIFLOW’s physics-decoupled architecture achieves significantly higher structural similarity (SSIM) than monolithic models.
Figure 3: Quantitative evaluation of scientific reasoning quality. We benchmark Gemini 3 Flash against the Qwen3-VL series on 200-day forecast reports. Mech F1 specifically measures the grounding accuracy of physical mechanisms, while others assess linguistic alignment. Results show a clear scaling trend in reasoning depth.
Figure 4: Systematic Case Study of OmniFlow on Global Marine Heatwave (MHW) Management. Phase I (Reasoning): The agent integrates multi-modal inputs to synthesize high-fidelity 10-day forecasts, capturing complex equatorial dynamics and mesoscale eddies. Phase II (Intervention): By executing an active counterfactual probe ( do(Forcing=0) ), OmniFlow quantifies the causal sensitivity ( S=0.78 ) of thermal anomalies to atmospheric drivers. Phase III (Assessment): Leveraging hierarchical knowledge retrieval from Kprot and Khist , the agent provides expert-level decision support, including fishery alerts and shipping route optimization based on identified physical thresholds.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Physical Regime
Resolution
Temporal Horizon
Key Variables
Evaluation Scale
2D Turbulence
Microscopic Flow
128×128
100 timesteps
Vorticity
1,280 sequences
SEVIR
Regional Weather
384×384
5-min intervals
VIL, IR, VIS, GLM
Standard Events
ERA5
Global Climate
180×360
200-day forecast
21 variables (T, Z, U/V, Q)
Long-term Validation
Appendix
Table 3: Summary of Dataset Statistics (B6)
Category
Component / Parameter
Specifications / Values
Architecture
Reasoning Core
Gemini 3 Flash (Agentic Brain)
Numerical Simulator (NES)
Diffusion Transformer (DiT)
Inference
2D Turbulence Strategy
1→1 Training; 1→99 Recursive Testing
ERA5 Strategy
200-day Autoregressive Forecast
Ensemble Size ( K )
8 – 32 (Adaptive)
Hyperparameters
Perturbation Factor ( λ )
0.01 – 0.05
Appendix
Table 4: OMNIFLOW Configuration and Hyperparameters (C2)
Physics reasoning requires constructing a consistent model of the underlying physical system rather than relying solely on symbolic or formula-based manipulation. Although large language models have shown strong ability in solving math and coding problems, they still struggle with physics problems, as these problems entangle the physical modeling process with mathematical calculations. Humans approach physics by first building a representation of the system before performing calculations. Inspired by this, we introduce a unified framework that distills intermediate representations that explicitly encode the physical modeling process and adopt a two-stage post-training strategy, where supervised fine-tuning establishes structured modeling, and reinforcement learning with rubric-based feedback improves the quality of the modeling process. Experiments on multiple multimodal physics benchmarks show that our approach generally improves physical reasoning performance across different models and datasets. Across PhysReason, PhyX, and SeePhys, physical modeling outperforms GRPO by ~3% on average. showing that explicit physical modeling is an effective strategy for improving physics reasoning in small VLMs.
Ye Zhang, Xuehang Guo, Rui Pan +4
University of Pennsylvania · William & Mary · University of Illinois Urbana-Champaign +3
While Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in image and video understanding, their ability to comprehend the physical world has become an increasingly important research focus. Despite their improvements, current MLLMs struggle significantly with high-level physics reasoning. In this work, we investigate the first step of physical reasoning, i.e., intuitive physics understanding, revealing substantial limitations in understanding the dynamics of continuum objects. To isolate and evaluate this specific capability, we introduce two fundamental benchmark tasks: Next Frame Selection (NFS) and Temporal Coherence Verification (TCV). Our experiments demonstrate that even state-of-the-art MLLMs perform poorly on these foundational tasks. To address this limitation, we propose Scene Dynamic Field (SDF), a concise approach that leverages physics simulators within a multi-task fine-tuning framework. SDF substantially improves performance, achieving up to 20.7% gains on fluid tasks while showing strong generalization to unseen physical domains. This work not only highlights a critical gap in current MLLMs but also presents a promising cost-efficient approach for developing more physically grounded MLLMs. Our code and data are available at https://github.com/andylinx/Scene-Dynamic-Field.
Nanxi Li, Xiang Wang, Yuanjie Chen +3
Shanghai Jiao Tong University · Tianjin University · Shanghai Innovation Institute
We have witnessed remarkable advances in LLM reasoning capabilities with the advent of DeepSeek-R1. However, much of this progress has been fueled by the abundance of internet question-answer (QA) pairs, a major bottleneck going forward, since such data is limited in scale and concentrated mainly in domains like mathematics. In contrast, other sciences such as physics lack large-scale QA datasets to effectively train reasoning-capable models. In this work, we show that physics simulators can serve as a powerful alternative source of supervision for training LLMs for physical reasoning. We generate random scenes in physics engines, create synthetic question-answer pairs from simulated interactions, and train LLMs using reinforcement learning on this synthetic data. Our models exhibit zero-shot sim-to-real transfer to real-world physics benchmarks: for example, training solely on synthetic simulated data improves performance on IPhO (International Physics Olympiad) problems by 5-10 percentage points across model sizes. These results demonstrate that physics simulators can act as scalable data generators, enabling LLMs to acquire deep physical reasoning skills beyond the limitations of internet-scale QA data. Code available at: https://sim2reason.github.io/.