We have witnessed remarkable advances in LLM reasoning capabilities with the advent of DeepSeek-R1. However, much of this progress has been fueled by the abundance of internet question-answer (QA) pairs, a major bottleneck going forward, since such data is limited in scale and concentrated mainly in domains like mathematics. In contrast, other sciences such as physics lack large-scale QA datasets to effectively train reasoning-capable models. In this work, we show that physics simulators can serve as a powerful alternative source of supervision for training LLMs for physical reasoning. We generate random scenes in physics engines, create synthetic question-answer pairs from simulated interactions, and train LLMs using reinforcement learning on this synthetic data. Our models exhibit zero-shot sim-to-real transfer to real-world physics benchmarks: for example, training solely on synthetic simulated data improves performance on IPhO (International Physics Olympiad) problems by 5-10 percentage points across model sizes. These results demonstrate that physics simulators can act as scalable data generators, enabling LLMs to acquire deep physical reasoning skills beyond the limitations of internet-scale QA data. Code available at: https://sim2reason.github.io/.
Figures & tables
Figure 1: Overview of the Sim2Reason (Sim2Reason) pipeline. From left to right: we procedurally generate diverse physics scenes using a DSL, (a) compile them into MuJoCo simulations, and (b) record physically grounded state/force traces. (c–e) From these traces we automatically instantiate multiple types of question–answer pairs (numeric, reverse, and symbolic), and apply filtering to remove degenerate/shortcut questions and unstable simulation segments. (f) Finally, we post-train an LLM with RLVR on the resulting synthetic data and evaluate zero-shot sim-to-real transfer on real-world benchmarks (e.g., IPhO and other physics/math datasets).
Figure 2: Overview of our synthetic data-generation pipeline. We procedurally generate simulatable scenes by randomly selecting and connecting DSL entities (Section 2.1 ), then simulate each scene in MuJoCo and record time-series data of key physical attributes (Section 2.2 ). From these traces we craft natural-language QA pairs in three formats (Section 2.3 )-numeric, reverse, symbolic-and finally deduplicate and filter degenerate/shortcut-solvable questions before RL post-training (Section 2.4 ).
Figure 3: Example of our scene-generation DSL (top) and the corresponding simulator-rendered scene produced by compiling the DSL into simulator scene (bottom). The DSL composes scenes from reusable entities and bodies with explicit connection modes, enabling scalable procedural generation while restricting randomization to physically meaningful parameters.
Figure 4: Illustration of a shortcut solution . The correct answer depends on the coupled motion of the block m and wedge M , but weaker models may collapse the dotted region into a single body of mass M+m and still match the numeric answer. We filter QA pairs whose answers are invariant to such approximations.
Figure 5: Validation accuracy (green) versus average response length (blue, in tokens) for Qwen3-30B-Instruct over RL post-training steps. Longer responses are strongly associated with higher validation accuracy, suggesting that post-training encourages more extensive intermediate reasoning.
Model
Synthetic Numeric
Synthetic Symbolic
HCV
IPhO Mechanics
Qwen3-30B
14.8%
8.8%
53.9%
35.6%
+ RL (synthetic)
17.4% ( +2.6% )
8.0% ( -0.8% )
59.0% ( +5.1% )
40.0% ( +4.4% )
Qwen2.5-32B
8.9%
5.6%
50.6%
19.8%
+ RL (synthetic)
21.9% ( +13.0% )
10.4% ( +4.8% )
53.9% ( +3.3% )
25.2% ( +5.4% )
Qwen2.5-14B
7.0%
5.6%
49.3%
16.07%
+ RL (synthetic)
17.0% ( +10.0% )
10.4% ( +4.8% )
51.7% ( +2.4% )
20.45% ( +4.4% )
Table 1: Performance of Qwen2.5 family Instruct models before and after RL on synthetic datasets, expressed in percentage. Improvements are shown in parentheses.
Benchmark
Model
Score
JEEBench
Qwen2.5 32B
34.38%
+ RL (synthetic)
52.28%( +17.90% )
PHYSICS
Qwen2.5 32B
39.42%
+ RL (synthetic)
43.09%( +3.67% )
OlympiadBench
Qwen2.5 32B
41.41%
+ RL (synthetic)
44.53%( +3.12% )
Table 2: Mean accuracy of Qwen 2.5 32B Instruct on other real world benchmarks.
Model (Qwen 32B)
Synthetic
IPhO
Baseline
14.0%
19.8%
+ SFT
16.0% ( +2.0% )
15.9% ( -3.9% )
+ RL (Ours)
32.0% ( +18.0% )
25.2% ( +5.4% )
Table 3: Comparison of RL vs. SFT on 32B model performance.
Table 4: Ablations on (a) QA format and (b) Data filtration
Base Model
Post-Trained Model
IPhO Accuracy (%)
Qwen2.5-32B
DAPO-32B
24.7
Qwen2.5-32B
LIMO-32B
25.5
Qwen3-30B
Prime P1 30B
38.6
Qwen3-30B
Sim2Reason (Ours)
40. 0
Table 5: Comparison with open-weight post-trained models trained on real-world QA datasets.
Model (Qwen 3B)
IPhO
Baseline
5.68
+ RL DAPO-17K (Real)
9.98
+ RL Mixed: DAPO-17K (Real) + Synthetic
10.35
+ RL Synthetic (Ours)
13.15
Table 6: Comparison with a real-world post-training dataset.
Category
Qwen 32B
+ RL (synthetic)
High School and Below
65.5%
68.3% ( +2.8% )
High School Olympiad
52.9%
54.0% ( +1.1% )
Undergraduate
47.9%
48.4% ( +0.5% )
Postgraduate
32.2%
37.8% ( +5.6% )
Table 7: Detailed performance across difficulty levels on the PHYSICS benchmark.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Timestep pruning for simulation traces with unmodelled transitions. Left: MuJoCo scene snapshots at the start and at time 3s. Right: recorded time-series signals; when a sliding-window deviation criterion flags an outlier (e.g., due to contact between block and pulley), we keep only the stable prefix (green) and discard the remainder before generating QA pairs.
Figure 9: Correlation between accuracy on Sim2Reason synthetic questions and IPhO mechanics questions.
Body
Symbol(s)
Description
Mass
m
Point mass / block mass.
Sphere
r,m
Sphere radius and mass.
Polygonal prism
n,r,h,m
Number of sides, circumscribed radius, height, and mass.
Cylinder
r,h,m
Cylinder radius, height, and mass.
Disc
r,m
Disc radius and mass.
Bar
w,ℓ,h,m
Bar width, length, height, and mass.
Appendix
Table 8: Bodies used by the DSL and the corresponding randomizable parameters.
Category
Quantity
Description
Mass
displacement
Body displacement / position (in world frame).
Mass
com_offset
Vector from body frame origin to center of mass.
Mass
velocity (6D)
Linear and angular velocity.
Mass
acceleration (6D)
Linear and angular acceleration.
Mass
mass
Body mass.
Mass
momentum (6D)
Linear and angular momentum.
Appendix
Table 9: Physical quantities recorded from MuJoCo for each simulated scene.
Figure 29: Scaling up the DSL using LLMs. In this experiment, we task the LLM to extend our entity vocabulary by inventing new entities to support simulation of this question from F=MA 2024 . We observe that while modern LLMs fail to simulate a target scene by generating raw simulator code (left), they can do so by extending our DSL with novel entities (right).
Figure 30: Scaling up the DSL using LLMs. In this experiment, we task the LLM to extend our entity vocabulary by inventing new entities to support simulation of this question from USA PhO 2019 . In this case, we observe that LLMs are able to simulate a target scene by generating raw simulator code (left), and by extending our DSL with novel entities (right).
Figure 31: Scaling up the DSL using LLMs. In this experiment, we task the LLM to extend our entity vocabulary by inventing new entities to support simulation of this question from JEE Advanced 2019 . We observe that while LLMs fail to simulate a target scene by generating raw simulator code (left), they can do so by extending our DSL with novel entities (right).
Figure 32: We task LLMs to port a subset of our entities from MuJoCo simulator to Omniverse simulator, to evaluate the portability of our DSL across simulators. LLMs could successfully transfer DSL entities across simulators, which could potentially help expand the scope of simulator-based synthetic data generation.
Physics reasoning requires constructing a consistent model of the underlying physical system rather than relying solely on symbolic or formula-based manipulation. Although large language models have shown strong ability in solving math and coding problems, they still struggle with physics problems, as these problems entangle the physical modeling process with mathematical calculations. Humans approach physics by first building a representation of the system before performing calculations. Inspired by this, we introduce a unified framework that distills intermediate representations that explicitly encode the physical modeling process and adopt a two-stage post-training strategy, where supervised fine-tuning establishes structured modeling, and reinforcement learning with rubric-based feedback improves the quality of the modeling process. Experiments on multiple multimodal physics benchmarks show that our approach generally improves physical reasoning performance across different models and datasets. Across PhysReason, PhyX, and SeePhys, physical modeling outperforms GRPO by ~3% on average. showing that explicit physical modeling is an effective strategy for improving physics reasoning in small VLMs.
Ye Zhang, Xuehang Guo, Rui Pan +4
University of Pennsylvania · William & Mary · University of Illinois Urbana-Champaign +3
Understanding how (multimodal) large language models perform on physics problems requires benchmarks that reflect the difficulty and breadth of expert-level physical reasoning. Existing physics benchmarks remain limited in the following two important ways: (1) short of high-difficulty datasets, and (2) lack of comprehensive coverage of visual forms, knowledge points, and step-by-step solution processes. As a result, model performance on current datasets may not be fully representative of their ability to solve complex physics problems. To address these issues, we present PhysElite, a large-scale bilingual multimodal benchmark for Olympiad-level physics reasoning. PhysElite contains 11,586 Olympiad-tier problems. For each problem, we provide corresponding visual diagrams, step-by-step bilingual Chinese-English solution derivations, and the final answer. We benchmark 18 open-source and closed-source MLLMs, and find that even the strongest model reaches only 33.7% answer accuracy. We additionally conduct step-level process evaluation to diagnose where models fail in the reasoning chain. Our datasets are released at https://huggingface.co/datasets/physelite/PhysElite.
Frontier LLMs now perform strongly across a wide range of physics evaluations, but it is hard to disentangle genuine reasoning from recall of established science. We introduce DiscoverPhysics, an interactive benchmark that asks a LLM agent to discover the laws of motion of a simulated world whose physics deliberately deviates from our own. We construct 22 worlds governed by, among others, screened and fractional-power gravity, multi-species couplings, hidden dark-matter-like particles, non-coordinate-free physics, and time-varying interactions. Each world is generated on demand by an N-body simulator, for which the agent proposes several rounds of experiments, observes raw trajectory data, and ultimately submits both a natural-language explanation of the world's physics and a Python implementation of the inferred law. Because solving a world requires the agent to design informative experiments and revise its hypotheses, the benchmark probes long-horizon reasoning over an experimental history. We evaluate submissions along two complementary axes: trajectory MSE on held-out particles and an LLM-judged explanation score following an expert-written rubric assessing conceptual understanding of each world. Across eleven frontier models, we find that the strongest agents pass only half of the worlds and consistently fail on those where latent structure must be uncovered. Open-source models lag substantially behind commercial models, both in their ability to design informative experiments and in extracting conclusions from the data. We further find that good predictive accuracy does not guarantee high explanation quality and that conceptual understanding depends on hypothesis refinement through well-chosen experiments.
Matt L. Wiemann, Lindsay M. Smith, Peter Melchior +4
Princeton University · Work done during internship at NYU/Polymathic AI. · Boston University +3