GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training

Haixu Wu; Minghao Guo; Zongyi Li; Zhiyang Dou; Mingsheng Long; Kaiming He; Wojciech Matusik

cs.LGFeb 23, 2026

GeoPT: Scaling Physics Simulation via Lifted Geometric Pre-Training

Authors: Haixu Wu, Minghao Guo, Zongyi Li, Zhiyang Dou, Mingsheng Long, Kaiming He, Wojciech Matusik

Organizations: MIT CSAIL · Tsinghua University

Abstract

Neural simulators promise efficient surrogates for physics simulation, but scaling them is bottlenecked by the prohibitive cost of generating high-fidelity training data. Pre-training on abundant off-the-shelf geometries offers a natural alternative, yet faces a fundamental gap: supervision on static geometry alone ignores dynamics and can lead to negative transfer on physics tasks. We present GeoPT, a unified pre-trained model for general physics simulation based on lifted geometric pre-training. The core idea is to augment geometry with synthetic dynamics, enabling dynamics-aware self-supervision without physics labels. Pre-trained on over one million samples, GeoPT consistently improves industrial-fidelity benchmarks spanning fluid mechanics for cars, aircraft, and ships, and solid mechanics in crash simulation, reducing labeled data requirements by 20-60% and accelerating convergence by 2 $\times$ . These results show that lifting with synthetic dynamics bridges the geometry-physics gap, unlocking a scalable path for neural simulation and potentially beyond. Code is available at https://github.com/Physics-Scaling/GeoPT.

Figures & tables

Figure 1: Neural aerodynamics simulation on DrivAerML ( Ashton et al., 2024 ) based on Transolver ( Wu et al., 2024 ) backbone. Geometry-only pre-training and conditioning refer to pre-training by predicting vector distance ( Faugeras and Gomes, 2000 ) of given positions and utilizing geometry representation extracted by Hunyuan3D ( Tencent, 2025 ) as auxiliary feature, respectively.

Figure 2: GeoPT offers a way to scale up neural simulators with off-the-shelf geometries and enables fast fine-tuning for various physics.

Figure 3: Geometry-physics analysis. (a) Visualization of learned correlations on DrivAerML ( Ashton et al., 2024 ) . We train Transolver ( Wu et al., 2024 ) using different supervisions and visualize the spatial distribution of learned aggregation weights in four tokens . Brighter colors indicate higher token assignment likelihood, revealing correlations captured by the model. See Appendix F full results. (b) We lift the geometry space by augmenting it with synthetic velocity fields, which further derive a dynamics-aware supervision.

Figure 4: Overall design of GeoPT. (a) To ensure the pre-training diversity, we pre-train the model with geometry randomly sampled from the public repository ( Chang et al., 2015 ) and generate the supervision for random tracking points under random dynamics. (b) Through a dynamics-lifted framework, we can configure the dynamics condition to “prompt” the corresponding pre-training capability of GeoPT.

Type	Benchmarks	#Mesh	#Variable	#Size
Aero-	DrivAerML	$\sim$ 160M	Geometry	$\sim$ 6TB
dynamics	NASA-CRM	$\sim$ 450k	Geo, Speed, AoA	$\sim$ 3GB
dynamics	AirCraft	$\sim$ 330k	Geo, Speed, AoA, Slip	$\sim$ 7GB
Hydro-	DTC Hull	$\sim$ 240k	Geo, Yaw Angle	$\sim$ 2GB
Crash	Car-Crash	$\sim$ 1M	Impact Angle	$\sim$ 8GB

Table 1: Summary of experimental simulations. #Mesh records the size of the discretized meshes for each sample. #Variable records the varied simulation configurations among different samples.

Figure 5: Performance comparison across fine-tuning epochs and physics samples. We show detailed curves at 200 epochs and 100 samples for clarity. Here, geometry-only pre-training adopts vector distance, which is better than SDF. See Appendix C for full results.

Figure 6: GeoPT scaling tests. (a) Gradually increase model layers from 8 to 32 and record the performance change of training from scratch and with GeoPT. (b) Reduce the pre-training diversity of both geometries and dynamics. See Appendix F for full results.

Figure 7: Simulation results and learned representations from GeoPT, including (a) visualization of the prediction results with the worst relative L2 performance in DrivAerML, (b) the error map of surface pressure and surrounding velocity, (c) correlations learned by pre-trained GeoPT under varied dynamics information, such as different directions and speeds of $V_{S}$ . See Appendix D for more results.

Figure 8: (a) Analysis for the geometry usage, including the comparison between geometry-only and dynamics-lifted spaces, as well as pre-training v.s. conditioning. (b) Backbone comparison.

Figure 9: Experiments of neural radiosity simulation on the Cornell box ( Goral et al., 1984 ) with an unseen Stanford bunny geometry.

Figure 10: Scaling up GeoPT-Huge pre-training with general 3D shape repositories. Left: performance under different pre-training scales. Right: geometry examples from the pre-training datasets.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 11: Ablations for the pre-training choices in GeoPT, including the comparison among (a) pre-training with a single subset or mixed data from ShapeNet-V1, -V2, (b) discretizing the dynamics into various step numbers

\tau

, and (c) pre-training with different geometry information, such as SDF and vector distance. All the experiments are based on GeoPT-Based under the full-data-full-training setting.

Figure 12: The pre-training process of GeoPT. (a) We plot the test MSE change during 200 pre-training epochs, which is calculated on 300 leave-out geometries. The initial unstable stage is caused by learning rate warm up . (b) Prediction case of the pre-trained GeoPT-Base. For clarity, we only plot 35 points under 3 dynamic steps, where the prediction is in blue and the ground truth is in orange.

Figure 13: GeoPT-Base performance change w.r.t. the direction shifted (by 0 ∘ to 40 ∘ ) from the correct configuration. Zero shift refers to configuring the dynamics field direction along the incoming flow or impact angle, which is our default setting. For clarity, we adopt the same

y

-axis range of 0.75 among different benchmarks in visualization. All the experiments are under the full-data-full-training setting.

Figure 14: GeoPT-Base performance change w.r.t. the configured velocity norm in

V_{S}

, which is sampled from

[0,2]

during pre-training. For real-world low-speed cases, such as car and watercraft simulations, we explore the choices within less than 1.0. As for high-speed cases, such as aircraft, we explore a larger range

[0.2,2.6]

. Since both NASA-CRM and AirCraft involve varying simulation speeds, we normalize the real-world configuration into different intervals. All the experiments are under the full-data-full-training setting.

Figure 15: Test loss (relative L2) change during fine-tuning. Note that these test losses are calculated from the downsampled physics field to ensure training efficiency, which is proportional to the final performance, but may be shifted w.r.t. the full mesh results.

Figure 16: Showcase of NASA-CRM. (a) The pressure coefficient field of the airplane flying under 0.7219 Mach and 3.46 ∘ angle of attack. (b) Error map and the relative L2 of different models for this case. For clarity, we zoom in on high-error zones.

Figure 17: Showcase of AirCraft. We plot the Z-force coefficient field for aircraft flying under 7 Mach, 7 ∘ angle of attack and 2 ∘ sideslip.

Figure 18: Showcase of DTCHull. (a) We plot the hydrostatic pressure–corrected pressure and surrounding velocity streamlines for a ship moving under 11 ∘ yaw angle. (b) We highlight the surrounding velocity error in the streamline and the pressure error in underside view.

Figure 19: Showcase of Car-Crash. We plot the element-wise maximum 2D Von Mises stress during the crash with 26.93 ∘ impact angle.

Figure 20: Examples from DTCHull and Car-Crash benchmarks, which involve diverse geometries and deformations, respectively.

Type	Benchmarks	#Mesh	#Variable	#Size	#Train	#Test	Output Physics
Aero-	DrivAerML	$\sim$ 160M	Geometry	$\sim$ 6TB	100	20	Pressure & Velocity
dynamics	NASA-CRM	$\sim$ 450k	Geo, Speed, AoA	$\sim$ 3GB	105	44	Pressure Coefficient
dynamics	AirCraft	$\sim$ 330k	Geo, Speed, AoA, Sideslip	$\sim$ 7GB	100	50	Aerodynamic Six Components
Hydro-	DTCHull	$\sim$ 240k	Geo, Yaw Angle	$\sim$ 2GB	100	20	Time-Avg Pressure & Velocity
Crash	Car-Crash	$\sim$ 1M	Impact Angle	$\sim$ 8GB	100	30	Time-max 2D Von Mises stress

Appendix

Table 2: Summary of experimental simulations. #Mesh records the size of the discretized meshes for each sample. #Variable records the varied simulation configurations among different samples. #Train and #Test represent the number of training and test samples.

Table 3: Configurations for GeoPT experiments. “B, L, H” represent the base, large and huge models.

Table 4: GeoPT Extension to time-dependent simulation and other backbones. All metrics are tested under full-data and full-training.

Table 5: Comparison between GeoPT and the previous method under aligned training settings. The results directly adopted from the previous paper are marked as

\ast

, where the results in DrivAerML (surface-only) and NASA-CRM are both from GAOT ( 2025 ) and AirCraft is from Transolver++ ( 2025 ) .

{\dagger}

marks our implementation. (a-b) The MSE and Mean AE are calculated on the normalized values.

Table 6: Quantitative results for scaling performance and supplement results in DTCHull and Car-Crash. Here “w/o GeoPT” refers to Transolver trained from scratch. “Unique Geo” and “Dyn” represent the use ratio of geometry data and sampled dynamics, respectively.

Standard deviation ( $\times 10^{-3}$ )	DrivAerML	NASA-CRM	AirCraft	DTCHull	Car-Crash
From Scratch 100% Training data	0.43	0.94	1.03	1.34	0.71
GeoPT 100% Training data	0.68	0.35	1.23	0.88	0.31
GeoPT 80% Training data	0.80	0.66	1.27	1.10	0.56
GeoPT 60% Training data	0.81	0.93	1.17	0.49	0.13
GeoPT 40% Training data	0.31	0.76	1.83	0.91	0.76

Appendix

Table 7: Standard deviation of relative L2 under 5 runs.

Panel of Table 8 — Table 8: Experiments on DrivAerML ( Ashton et al., 2024 ) , where we gradually increase the training data from 20 samples to 100 samples and training epochs from 50 epochs to 200 epochs. The relative L2 of Transolver (a) trained from scratch, (b) pre-trained with vector distance supervision, (c) trained with geometry condition, and (d) pre-trained with GeoPT are recorded.

Panel of Table 9 — Table 9: Experiments on NASA-CRM ( Bekemeyer et al., 2025 ) . The relative L2 error under different training data and epochs is recorded.

Panel of Table 10 — Table 10: Experiments on AirCraft ( Luo et al., 2025 ) . The relative L2 error under different training data and epochs is recorded.

Panel of Table 11 — Table 11: Experiments on DTCHull. The relative L2 error under different training data and epochs is recorded.

Panel of Table 12 — Table 12: Experiments on Car-Crash. The relative L2 error under different training data and epochs is recorded.

Figure 22: Visualization of physical states learned from different supervision signals, including (a) pre-training by learning to predict vector distance, (b) directly training with DrivAerML physics supervision and (c) pre-training with the dynamic process proposed by GeoPT. Here, we visualize the last layer. The visualizations from other layers can differ in absolute value but share a similar distribution.

Figure 23: Visualization of DrivAerML physical states in GeoPT “prompted” from different dynamics configurations, including varied (a-c) direction and (d-e) velocity norm (speed) configurations. Here, we visualize the fourth layer as a supplement to the last layer visualization in Fig. 22 . Notably, the states in the fourth layer are less distinguishable in the middle layers when compared with the last layer. This can be viewed as an architectural feature of Transolver, which can enable better global interaction among different states.

Figure 24: Visualization of NASA-CRM physical states in GeoPT “prompted” from different dynamics configurations, including varied (a-c) direction and (d-e) speed configurations. Here, we visualize the physical states in the last layer. Similar to DrivAerML, changing the angle of attack in the $y$ - $z$ plane will affect the state distribution and increasing speed will lead to a more concentrated state distribution.

Figure 25: Visualization of DTCHull physical states in GeoPT “prompted” from different dynamics configurations, including varied (a-c) direction and (d-e) speed configurations. Here, we visualize the physical states in the last layer. Notably, to ensure a clear presentation of the inside ship surface, we downsample the mesh by 10 times for visualization.

Physics-Based Simulation

Physics-based simulation models physical systems with equations and numerical solvers, increasingly augmented by learned models for speed and control.

See all 228 papers

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers
Oct 2, 2026Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu +4Synthetic Data Pretraining Neural PDE Solvers
Technical University of Munich · University of Stuttgart
PI-JEPA: Label-Free Surrogate Pretraining for Coupled Multiphysics Simulation via Operator-Split Latent Prediction
Apr 1, 2026Brandon Yee, Pairie KohMultiphysics Simulation PDE Surrogate Modeling
Physics Lab, Yee Collins Research Group · Graduate School of Business, Stanford University
Solving Physics Olympiad via Reinforcement Learning on Physics Simulators
Apr 13, 2026Mihir Prabhudesai, Aryan Satpathy, Yangmin Li +6Scientific Reasoning in Language Models Synthetic Data Generation for Language Models
Carnegie Mellon University

Oct 2, 2026J/K move · Enter open · S save

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu, Benjamin Holzschuh+3

Technical University of Munich · University of Stuttgart

Synthetic Data Pretraining Neural PDE Solvers PDE Solving PDE Surrogate Modeling Data-Efficient Learning

Open paper AskPDF arXiv

Neural surrogate models for Partial Differential Equations (PDEs) on unstructured 3D geometries are often limited by poor generalization and the high cost of generating large-scale training datasets. Consequently, pre-training on massive datasets of related PDE dynamics has emerged as a critical alternative to enhance the robustness and scalability of these models. However, this strategy is neither compute- nor data-efficient, as it relies on massive pre-computed data that is very costly to generate. In this work, we introduce a disk-data-free pre-training framework tailored to both steady-state and transient regimes. For steady-state problems, we propose a geometry-driven strategy that leverages intrinsic shape descriptors to learn representations of complex 3D domains. For transient problems, we introduce a physics-driven approach based on online generation of synthetic PDE data, enabling scalable pre-training without reliance on expensive datasets. Across multiple experiments, our approach achieves faster convergence, greater data efficiency, and higher accuracy during fine-tuning, particularly under realistic low-data regimes. This methodology provides a practical pathway toward data-efficient neural emulators for large-scale simulations.

Abstract

Physics-Based Simulation

Explore similar work

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

PI-JEPA: Label-Free Surrogate Pretraining for Coupled Multiphysics Simulation via Operator-Split Latent Prediction

Solving Physics Olympiad via Reinforcement Learning on Physics Simulators