Axiomatic World Modeling for Physics Reasoning

Xinye Yang; Zhenyang Liu; Yuxuan Wang; Yuanyuan Lei

cs.AIOct 17, 2026

Axiomatic World Modeling for Physics Reasoning

Authors: Xinye Yang, Zhenyang Liu, Yuxuan Wang, Yuanyuan Lei

Organizations: Northwestern University, Evanston, IL · Department of Physics, University of Florida, Gainesville, FL · Computer & Information Science and Engineering, University of Florida, Gainesville, FL

Abstract

Recent research-level physics-reasoning benchmarks for LLMs reveal a failure mode that lies not in knowledge or execution, but in modeling the world being reasoned about. Models often silently alter given quantities, fabricate laws, or inject unstated assumptions to force a problem into familiar patterns, and then reason inside this corrupted world. We define this premise-level divergence as modeling drift: the gap between a model's internal axiomatic world model (AWM) and the true physical world specified in the problem. Although RLVR and PRM have yielded substantial gains in mathematics and code, their reward shapes do not directly supervise world modeling in axiomatic domains such as physics. We propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), which optimizes the policy by penalizing divergence between its inferred axiomatic world model and the true physics world. RLAWM probes physics reasoning at two hierarchical levels (modeling and solving), each scored by a physically grounded reward. The modeling reward targets abstract knowledge formulation, evaluating whether the model captures the true axiomatic world before attempting calculation. The solving reward targets instantiated knowledge reasoning, checking whether the model's concrete derivation remains consistent with the axiomatic world. This two-level structure separates what world the model believes it is reasoning in from how it reasons within that world. The framework operates exclusively during training, with no extra cost during inference. RLAWM improves performance by up to +12.2 points on PhysReason and +5.2 on PHYSICS over the baseline. It also generalizes zero-shot across difficulty, source, and domain, with a 2.45x gain and +10 points over the strongest baseline on olympiad-level physics and math.

Figures & tables

Figure 1: Modeling drift on a representative physics problem. Left: the true axiomatic world model (AWM): objects, geometry, field, and laws. Middle: a representative drift trajectory of the base model: wrong geometry propagates into a malformed energy equation, and upon deriving $v_{O}^{2}<0$ the model silently sets $E\!\leftarrow\!0$ for self-consistency instead of repairing the geometry. Right: an RLAWM-trained model recovers the correct geometry and reasons consistently within the true AWM.

Figure 2: The RLAWM pipeline. RLAWM probes physical reasoning at two hierarchical levels, each carrying a grounded reward. Modeling (top) concerns how the policy builds its world. It externalizes the AWM as a five-field [modeling] block, which $R_{M}$ scores against the problem’s ground-truth AWM. Solving (bottom) concerns how the policy executes reasoning over that AWM: it writes its reasoning as pseudocode, where a frozen checker probes for physical invariants, trace consistency, and response under perturbations to yield $R_{S}$ .

Backbone: Qwen3-VL-8B-Instruct
	PhysReason					PHYSICS-text_only
Method	Easy	Medium	Difficult	Knowledge	Overall	Stat. Mech.	Quantum	Atomic	E&M	Mechanics	Optics	Overall
Vanilla LLM	39.5	15.5	13.3	46.0	30.0	16.3	17.3	9.7	33.8	18.0	16.7	18.5
RLVR	39.1	22.5	17.5	47.3	32.8	16.8	16.3	14.0	25.0	16.0	36.1	19.2
Direct-Likert	43.8	25.7	20.6	52.8	37.1	23.7	23.8	14.4	33.8	19.7	31.6	23.8
Reference-Likert	31.0	20.1	20.3	55.6	33.2	18.8	22.3	14.4	31.6	18.2	34.2	22.2

Table 1: SubPass@1 (%) on PhysReason and PHYSICS-text_only . Best per column within each backbone is in bold . RLAWM’s

R_{M}

with gpt-4o-mini-2024-07-18 as the scorer.

Panel of Figure 3 — Figure 3: SubPass@ $k$ curves across $k\in\{1,5,10,20,50,100\}$ on PhysReason. As $k$ grows, RLVR and Reference-Likert fall below the base model. RLAWM stays on top at every $k$ .

Backbone: Qwen3-VL-8B-Instruct
	PhysReason					PHYSICS-text_only
Reward	Easy	Medium	Difficult	Knowledge	Overall	Stat. Mech.	Quantum	Atomic	E&M	Mechanics	Optics	Overall
$R_{M}{+}R_{A}$	48.3	39.8	17.6	63.0	44.0	28.2	19.7	18.2	33.5	20.0	36.1	25.1
$R_{S}{+}R_{A}$	47.6	36.3	24.2	66.0	45.3	25.4	19.4	19.5	37.3	17.2	44.4	25.7
Joint $R_{M}{+}R_{S}{+}R_{A}$ †	57.6	39.6	23.9	64.0	47.9	26.4	21.9	28.0	42.7	20.5	38.9	29.0
Backbone: Qwen3-VL-4B-Instruct

Table 2: Per-stage reward ablation. SubPass@1 (%); best per column per backbone in bold .

Table 6

		Qwen3-VL-8B-Instruct					Qwen3-VL-4B-Instruct
$R_{M}$	$R_{M}$ ’s Scorer	Easy	Medium	Difficult	Knowledge	Overall	Easy	Medium	Difficult	Knowledge	Overall
PMF	4o	57.6	39.6	23.9	64.0	47.9	51.5	37.0	23.1	61.3	44.8
PMF	qwen	55.1	31.6	20.5	68.0	45.8	44.8	36.6	17.4	65.3	43.0
Judge	4o	57.3	36.5	21.3	66.0	47.1	50.5	34.2	19.7	64.0	44.0
Judge	qwen	51.4	30.9	19.8	66.7	44.2	43.8	31.1	16.1	62.0	40.2

Table 5: Modeling-scorer ablation (PhysReason). SubPass@1 (%); best per column within each backbone in bold . 4o : gpt-4o-mini ; qwen : Qwen3-VL-4B .

Reward	Setting	Easy	Medium	Difficult	Knowledge	Overall
Reference-Likert	Zero-RL	41.7	17.9	10.9	48.7	31.4
Reference-Likert	M/S SFT	43.8	18.6	11.1	51.3	32.9
RLAWM	Zero-RL	49.2	23.9	15.8	54.0	37.3

Table 6: Reward signal vs. SFT data (PhysReason, Qwen3-VL-8B). SubPass@1 (%). Zero-RL : GRPO without SFT. M/S SFT : Reference-Likert trained on RLAWM’s restructured SFT data.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Notation	Symbol	Description
Modeling reward (§ 3.2 )
Policy	$\pi_{\theta}$	LLM-parameterized policy
Externalized AWM	$\hat{m}$	Policy’s structured externalization of its belief (the five-field [modeling] block)
Reference structure	$m^{*}$	An operational reference representation constructed from the problem statement and its ground-truth solution. It serves as a proxy for the problem’s physical axiom set rather than as a separately formalized axiomatic theory.
Per-motif F1	$\mathrm{F1}_{\text{QEL/VCL/AEB}}$	PMF motif scores
Modeling reward	$R_{M}$	$R_{M}^{\text{PMF}}$ (PMF F1 over the externalized AWM)

Appendix

Table 7: Summary of core notation used throughout the paper.

Dataset	License	Split	#	Composition
PhysReason [ 31 ]	MIT	train	1,080	easy 24.5% / medium 25.0% / difficult 25.0% / knowledge 25.5%
		test	120	easy 25.0% / medium 25.0% / difficult 20.8% / knowledge 29.2%
PHYSICS-text_only [ 5 ]	MIT	train	897	thermodynamics & statistical mechanics 21.3% / quantum mechanics 20.3% / atomic physics 18.6% / electromagnetism 17.1% / mechanics 13.4% / optics 9.4%
		test	99	thermodynamics & statistical mechanics 21.2% / quantum mechanics 20.2% / atomic physics 19.2% / electromagnetism 17.2% / mechanics 13.1% / optics 9.1%
OlympiadBench-Phys [ 8 ]	MIT	test	236	text-only, English, physics, with verifiable ground truth ( OE_TO_physics_en_COMP )
PHYBench [ 21 ]	MIT	test	500	full set; original text-only problems written by physics students, symbolic-expression answers, high-school to olympiad difficulty

Appendix

Table 8: Dataset overview. We split PhysReason and PHYSICS-text_only randomly

9{:}1

into training and test partitions; OlympiadBench-Phys, PHYBench, and AIME 2026 are evaluation-only. PhysReason categories combine three reasoning-difficulty tiers ( easy / medium / difficult ) with one knowledge-recall category ( knowledge ); PHYSICS-text_only subdomains are taken directly from the source dataset and listed in test-split frequency order. PhysReason, PHYSICS, OlympiadBench, and PHYBench are released under the MIT License. AIME 2026 is taken from MathArena and released under CC BY-NC-SA 4.0.

Model	Role	License
Qwen3-VL-8B-Instruct [ 1 ]	Policy backbone (main and ablations)	Apache 2.0
Qwen3-VL-4B-Instruct [ 1 ]	Policy backbone; compact PMF scorer (ablation)	Apache 2.0
Gemma-3-12B-IT [ 24 ]	Policy backbone (cross-family verification)	Gemma Terms of Use
Qwen2.5-Coder-1.5B-Instruct [ 9 ]	Frozen translator $f$ (pseudocode $\to$ Python)	Apache 2.0
gpt-4o-mini-2024-07-18 [ 18 ]	PMF scorer for $R_{M}$ ; Judge baseline	Proprietary (OpenAI API ToS)

Appendix

Table 9: Pre-trained model assets used in this paper.

Hyperparameter	Value
Fine-tuning type	Full fine-tuning
Vision tower	Frozen
Multimodal projector	Trainable
Language model	Trainable
Cutoff length	8192
Maximum samples	5000

Appendix

Table 10: Main SFT hyperparameters shared by all methods.

Hyperparameter	Value
Advantage estimator	GRPO
Normalize GRPO advantages by std.	Yes
KL in reward	No
Actor KL loss	Enabled
KL coefficient	0.001
KL loss type	Low-variance KL

Appendix

Table 11: Main GRPO hyperparameters shared by RLAWM and all RL baselines.

Run	Overall
Direct-Likert (strongest baseline)	37.1
RLAWM, seed 10	49.1
RLAWM, seed 42 (reported)	47.9
RLAWM, seed 100	47.3
Mean $\pm$ std	$48.1\pm 0.9$

Appendix

Table 12: Seed robustness of RLAWM on PhysReason. SubPass@1 Overall (%) on Qwen3-VL-8B-Instruct; seed 42 is the run reported in Table 1 .

Error type	QEL	VCL	AEB
Wrong theorem selected	Decreases	May decrease	Unchanged
Correct theorem but missing variable	Decreases	May decrease	Unchanged
Broken interaction between equations	Unchanged	Decreases	Unchanged
Missed premise or assumption	Unchanged	Unchanged	Decreases
Missed object or physical stage	Decreases	Decreases	May decrease

Appendix

Table 13: Diagnostic effect of common modeling errors on the three PMF motif components. “May decrease” means that the component is affected only when the local error changes the corresponding motif type.

Failure type	Share of all rollouts
Sandbox execution error	11.8%
Variable-contract mismatch	7.0%
Pseudocode translation failure	0.4%
Missing solving code	0.2%
Total (scored $R_{S}=0$ )	19.4%

Appendix

Table 14: Failure types behind

R_{S}=0

. Share of all training rollouts on PhysReason with Qwen3-VL-8B-Instruct.

Physical Reasoning

Physical reasoning studies how AI systems infer object properties, forces, and causal dynamics to predict and act in the physical world.

See all 69 papers

Decoupled Physical Modeling and Execution for Physics Reasoning
Aug 22, 2026Ye Zhang, Xuehang Guo, Rui Pan +4Physical Reasoning VLM Reasoning
University of Pennsylvania · William & Mary · University of Illinois Urbana-Champaign +3
Solving Physics Olympiad via Reinforcement Learning on Physics Simulators
Apr 13, 2026Mihir Prabhudesai, Aryan Satpathy, Yangmin Li +6Scientific Reasoning in Language Models Synthetic Data Generation for Language Models
Carnegie Mellon University
Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models
Jul 6, 2026Raj Jaiswal, Dhruv Jain, Rishabh Dhawan +4LLM Self-Correction Physical Reasoning
IIIT Delhi · IIT BHU · NII Tokyo +2

Aug 22, 2026J/K move · Enter open · S save

Decoupled Physical Modeling and Execution for Physics Reasoning

Ye Zhang, Xuehang Guo, Rui Pan, Pengfei Yu+3

University of Pennsylvania · William & Mary · University of Illinois Urbana-Champaign+3

Physical Reasoning VLM Reasoning

Open paper AskPDF arXiv

Physics reasoning requires constructing a consistent model of the underlying physical system rather than relying solely on symbolic or formula-based manipulation. Although large language models have shown strong ability in solving math and coding problems, they still struggle with physics problems, as these problems entangle the physical modeling process with mathematical calculations. Humans approach physics by first building a representation of the system before performing calculations. Inspired by this, we introduce a unified framework that distills intermediate representations that explicitly encode the physical modeling process and adopt a two-stage post-training strategy, where supervised fine-tuning establishes structured modeling, and reinforcement learning with rubric-based feedback improves the quality of the modeling process. Experiments on multiple multimodal physics benchmarks show that our approach generally improves physical reasoning performance across different models and datasets. Across PhysReason, PhyX, and SeePhys, physical modeling outperforms GRPO by ~3% on average. showing that explicit physical modeling is an effective strategy for improving physics reasoning in small VLMs.

Abstract

Physical Reasoning

Explore similar work

Decoupled Physical Modeling and Execution for Physics Reasoning

Solving Physics Olympiad via Reinforcement Learning on Physics Simulators

Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models