Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

Luis Medrano-Navarro; Giacomo Baldan; Qiang Liu; Benjamin Holzschuh; Jan Hagnberger; Mathias Niepert; Nils Thuerey

cs.AIOct 2, 2026

Geometry Meets Physics: Data-Efficient Pre-Training for Unstructured Neural PDE Solvers

Authors: Luis Medrano-Navarro, Giacomo Baldan, Qiang Liu, Benjamin Holzschuh, Jan Hagnberger, Mathias Niepert, Nils Thuerey

Organizations: Technical University of Munich · University of Stuttgart

Abstract

Neural surrogate models for Partial Differential Equations (PDEs) on unstructured 3D geometries are often limited by poor generalization and the high cost of generating large-scale training datasets. Consequently, pre-training on massive datasets of related PDE dynamics has emerged as a critical alternative to enhance the robustness and scalability of these models. However, this strategy is neither compute- nor data-efficient, as it relies on massive pre-computed data that is very costly to generate. In this work, we introduce a disk-data-free pre-training framework tailored to both steady-state and transient regimes. For steady-state problems, we propose a geometry-driven strategy that leverages intrinsic shape descriptors to learn representations of complex 3D domains. For transient problems, we introduce a physics-driven approach based on online generation of synthetic PDE data, enabling scalable pre-training without reliance on expensive datasets. Across multiple experiments, our approach achieves faster convergence, greater data efficiency, and higher accuracy during fine-tuning, particularly under realistic low-data regimes. This methodology provides a practical pathway toward data-efficient neural emulators for large-scale simulations.

Figures & tables

Panel of Figure 1 — Figure 1 : Point-wise absolute error comparison between trained from scratch and pre-trained models (PT). Our GMP pre-training shows increased accuracy in critical regions.

Property	Traditional FM	Zhang et al. [17]	GeoPT	GMP (Ours)
Physics-label-free	✗	✓	✓	✓
Low pre-training cost	✗	✓	✗	✓
Unique pre-training	✓	✗	✓	✓
No high-detail CAD geometries	✗	✗	✓	✓
Zero disk storage	✗	✗	✗	✓

Table 1 : Comparison of pre-training paradigms across geometric and physical constraints.

Panel of Figure 2 — Figure 2 : Sketches for our two pre-training strategies. For steady predictions, we use geometric pre-training (left). The model learns to predict geometric features of randomized geometries, like the normals and curvature on the surface, and the Vector Distance Field (VDF) in the surrounding volume. For transient problems, we use physics pre-training (right). The model learns to autoencode online generated solutions to canonical, synthetic PDEs at randomized locations.

Figure 3 : Deformed grid.

Figure 4 : Geometric pre-training sketch.

Panel of Figure 5 — Figure 5 : Sketch of GMP-Net combining three encoders: one processing the general geometry of the object if available, and the other two the autoregressive physical fields. A decoder cross-attends the query positions to the output of the encoders. Finally, we compute a weighted sum using three learnable weights. Each encoder contains cross-attention from a set of anchor points to the rest of the point cloud, and an MLP modulated with external or simulation parameters.

Figure 6 : Samples showing surface (pressure in a,c) and volume (velocity streamlines in b,d) fields: DrivAerML (car, left) and SHIFT-Wing (aircraft, right).

Dataset	Time	Dim.	Type	# Samples	# Points (S;V)	# Fields
DrivAerML [ 56 ]	Steady	3D	S; V	500	$\sim$ 9M ; 160M	4+4
SHIFT-Wing [ 57 ]	Steady	3D	S; V	6000	$\sim$ 3M ; 6M	4+4
KS (ours)	Transient	2D	V	630 $\times$ 30	- ; 4096	1
Ellipse [ 59 ]	Transient	2D	V	1200 $\times$ 100	- ; 1024	3
SHIFT-Crash [ 58 ]	Both	3D	S	768 $\times$ 12	$\sim$ 400K ; -	4

Table 2: Overview of the fine-tuning datasets used for downstream tasks evaluation.

	DrivAerML				SHIFT-Wing				SHIFT-Crash
Training samples	16	32	64	128	16	32	64	128	16	32	64	128
Transolver++ (scratch) $\downarrow$	0.685	0.559	0.329	0.180	0.603	0.403	0.235	0.209	0.611	0.601	0.548	0.328
Transolver++ (GeoPT) $\downarrow$	0.520	0.443	0.309	0.218	0.308	0.266	0.233	0.204	0.488	0.428	0.410	0.388
Transolver++ (GMP ours) $\downarrow$	0.503	0.453	0.302	0.178	0.318	0.265	0.232	0.205	0.530	0.563	0.356	0.322
Rel. Improvement $\uparrow$	27%	19%	8%	1%	47%	34%	1%	2%	13%	6%	34%	2%
AB-UPT (scratch) $\downarrow$	0.439	0.313	0.244	0.184	0.569	0.456	0.379	0.298	0.396	0.363	0.340	0.326

Table 3 : Comparison between training from scratch and pre-training (PT) on steady-state tasks: relative

L_{2}

error and pre-training improvement. Tests for different numbers of training samples. A Transolver++ variant pre-trained with GeoPT [ 16 ] is also shown for comparison.

Figure 7 : DrivAerML test loss of GMP-Net trained from scratch and with pre-training. Horizontal lines indicate the minimum error. Pre-trained models achieve lower error and faster convergence.

Panel of Figure 8 — Figure 8 : Summary of model performance in steady problems. We show the final relative $L_{2}$ error for different fine-tuning dataset sizes.

Panel of Figure 9 — Figure 9 : Summary of model performance in steady problems. a) Faster convergence analysis shows how much faster pre-trained models converge to the accuracy observed by scratch versions. b) Data efficiency showing how much data can be saved. c) $R^{2}$ improvement on the computation of the aerodynamic coefficients thanks to pre-training.

	DrivAerML average $\downarrow$
Training samples	16	32	64	128
GMP-Net (scratch)	0.412	0.348	0.187	0.099
GMP (occupancy)	0.424	0.379	0.243	0.138
GMP (VDF)	0.402	0.333	0.189	0.103
GMP (N and C)	0.387	0.311	0.182	0.101
GMP (all)	0.346	0.299	0.178	0.097

Table 4 : Effect of geometric descriptors on GMP pre-training accuracy gains. Mean rel

L_{2}

error.

	2D KS $\downarrow$			2D Ellipse $\downarrow$			3D SHIFT-Crash $\downarrow$
Timesteps	$1$	$5$	$10$	$1$	$5$	$10$	$1$	$5$	$10$
Erwin	0.143	0.713	1.281	0.016	0.083	0.438	0.039	0.131	0.245
Transolver++	0.203	0.779	1.028	0.006	0.021	0.042	0.020	0.072	0.072
GMP-Net (scratch)	0.030	0.196	0.615	0.008	0.025	0.044	0.009	0.030	0.040
GMP-Net (PT)	0.024	0.148	0.479	0.006	0.020	0.033	0.008	0.026	0.034
Rel. Improvement	20%	24%	22%	25%	20%	23%	11%	13%	15%

Table 5 : NRMSE for transient problems. Comparison of from-scratch models and GMP pre-training.

# Trajectories	16	32	64
GMP-Net (scratch) $\times 10^{-3}$ $\downarrow$	8.8	6.9	5.4
GMP-Net (PT) $\times 10^{-3}$ $\downarrow$	6.2	4.8	3.9
Improvement $\uparrow$	29%	30%	28%

Table 6 : 1-step NMSE test loss and GMP pre-training reduction.

Appendix figures & tables86 assets

Supplementary material from the paper’s appendix.

Appendix

Vol. Points	Surf. Points	PT Time (h)	Rel $L_{2}$ Error FT (64 samples)
1024	512	2.5	0.194
2048	1024	4.0	0.178
4096	2048	9.0	0.175

Appendix

Table 7: Performance and fine-tuning relative

L_{2}

error on DrivAerML across different volume and surface point resolutions during pre-training (PT).

Panel of Figure 11 — Figure 11: Geometry pre-training.

Setting	Transolver++	AB-UPT	GMP-Net
Scratch	0.329	0.244	0.187
GMP relative	0.325	0.238	0.185
GMP ConFIG	0.302	0.218	0.178
Max. Imp	8%	17%	6%

Appendix

Table 8: Performance evaluation (Relative

L_{2}

error) on DrivAerML (64 samples).

Figure 12: Spectra of different pre-training solutions. Source: Liu et al. [11] .

Figure 13: Scaling of the size of SMART.

Figure 14: Scaling of the size of Erwin.

Panel of Figure 15 — Figure 15: DrivAerML final test nMSE for different sizes of the point cloud.

	DrivAerML		NASA CRM		Car Crash
Training samples	32	64	32	64	32	64
Transolver (scratch) $\downarrow$	0.166	0.110	0.242	0.136	0.315	0.214
Transolver (GeoPT) $\downarrow$	0.126	0.090	0.203	0.113	0.235	0.192
Rel. Improvement $\uparrow$	24%	18%	16%	17%	25%	10%

Appendix

Table 9: GeoPT comparison of relative

L_{2}

error between Transolver (scratch) and GeoPT at 200 epochs, alongside relative improvement across varying sample sizes. Source: Wu et al. [16] .

Category	Hyperparameter	Value / Description
Data generation	H’, W’, D’	32
	PDEs	Diffusion, Hyper-diffusion, Burgers, Korteweg-de Vries (KdV), Kuramoto-Sivashinsky (KS), Fisher-KPP, Swift-Hohenberg
	Initial conditions	Gaussian noise, Truncated Fourier series, Diffused noise, Decayed Energy, Poisson
	Resolution	64, 128, 256, 384
	# surface points	1024
	# volume points	2048

Appendix

Table 10: Hyperparameters for the physics pre-training.

Category	Hyperparameter	Value / Description
Data	# surface points	1024
	# volume points	2048
Optimization	Optimizer	AdamW
	Learning Rate	1e-4
	LR schedule	cosine
	Batch Size	8

Appendix

Table 11: Hyperparameters for the geometric pre-training.

Category	Hyperparameter	Value / Description
Data	Training/Test split	KS: 580/50 trajectories
		Ellipse: 1000/200 trajectories
		Crash: 600/100 trajectories
Optimization	Optimizer	AdamW
	Max learning Rate	1e-4
	Lr. schedule	Cosine

Appendix

Table 12: Hyperparameters for autoregressive training.

Category	Hyperparameter	Value / Description
Data	Training/Test split	(Varying size from 16 to 128)
	# points	32768 (S) + 32768 (V)
Optimization	Optimizer	LION
	Learning Rate	1e-4
	LR schedule	Cosine
	Batch Size	1

Appendix

Table 13: Hyperparameters for steady DrivAerML.

Category	Hyperparameter	Value / Description
Data	Training/Test split	(Varying size from 16 to 128)
	# points	32768 (S) + 32768 (V)
Optimization	Optimizer	LION
	Learning Rate	1e-4
	LR schedule	Cosine
	Batch Size	1

Appendix

Table 14: Hyperparameters for steady SHIFT-Wing.

Category	Hyperparameter	Value / Description
Data	Training/Test split	(Varying size from 16 to 128)
	# points	32768
Optimization	Optimizer	LION
	Learning Rate	1e-4
	LR schedule	Cosine
	Batch Size	1

Appendix

Table 15: Hyperparameters for steady SHIFT-Crash.

Hyperparameter	Value / Description
spatial dim	3
Surface channels	4
Volume channels	4
Parameter channels	16 (DrivAerML)
	1 (SHIFT-Wing)
	6 (SHIFT-Crash)

Appendix

Table 16: GMP-Net/SMART model configuration for steady problems.

Hyperparameter	Value / Description
spatial dim	2 (KS) / 2 (Ellipse) / 3 (Crash)
Input surface channels	0 (KS) / 0 (Ellipse) / 1 (Crash)
Input volume channels	1 (KS) / 3 (Ellipse) / 0 (Crash)
Output surface channels	0 (KS) / 0 (Ellipse) / 1 (Crash)
Output volume channels	1 (KS) / 3 (Ellipse) / 0 (Crash)
Parameter channels	1

Appendix

Table 17: GMP-Net model configuration for transient problems.

Table 33

Model	DrivAerML		SHIFT-Wing		SHIFT-Crash ∗
Model	Surface	Volume	Surface	Volume	Surface
PTV3	31.33	44.31	20.87	46.76	35.25
Erwin	28.52	26.67	11.52	38.67	42.15
Transolver++	11.32	13.04	7.67	24.04	23.97
GeoTransolver	10.33	10.90	7.50	24.90	23.39
AB-UPT	10.76	8.83	10.07	25.11	39.65

Appendix

Table 20: Test relative

L_{2}

error (%) averaged across surface and volume fields for DrivAerML, SHIFT-Wing, and SHIFT-Crash benchmarks.

Model	1	5	10
Erwin	0.143 $\pm$ 0.011	0.713 $\pm$ 0.043	1.281 $\pm$ 0.073
Transolver++	0.203 $\pm$ 0.014	0.779 $\pm$ 0.052	1.028 $\pm$ 0.035
GMP-Net (scratch)	0.030 $\pm$ 0.003	0.196 $\pm$ 0.039	0.615 $\pm$ 0.060
GMP-Net (PT)	0.024 $\pm$ 0.002	0.148 $\pm$ 0.030	0.479 $\pm$ 0.073

Appendix

Table 21: 2D KS equation. Mean RMSE for timesteps 1, 5, and 10.

Model	1	5	10
Erwin	0.016 $\pm$ 0.002	0.083 $\pm$ 0.034	0.438 $\pm$ 0.244
Transolver++	0.006 $\pm$ 0.001	0.021 $\pm$ 0.003	0.042 $\pm$ 0.005
GMP-Net (scratch)	0.008 $\pm$ 0.001	0.025 $\pm$ 0.043	0.044 $\pm$ 0.009
GMP-Net (PT)	0.006 $\pm$ 0.001	0.020 $\pm$ 0.004	0.033 $\pm$ 0.011

Appendix

Table 22: 2D flow around an ellipse. Mean RMSE for timesteps 1, 5, and 10.

Model	1	5	10
Erwin	0.039 $\pm$ 0.001	0.131 $\pm$ 0.034	0.245 $\pm$ 0.021
Transolver++	0.020 $\pm$ 0.016	0.072 $\pm$ 0.039	0.072 $\pm$ 0.021
GMP-Net (scratch)	0.009 $\pm$ 0.008	0.030 $\pm$ 0.017	0.040 $\pm$ 0.024
GMP-Net (PT)	0.008 $\pm$ 0.007	0.026 $\pm$ 0.018	0.034 $\pm$ 0.013

Appendix

Table 23: Deformation and von Mises stress average of SHIFT-Crash. Mean RMSE for timesteps 1, 5, and 10.

Model	Size (M) $\downarrow$	VRAM (GB) $\downarrow$	Throughput (it/s) $\uparrow$
Erwin [ 43 ]	6.8	2.15	2.45
Transolver++ [ 55 ]	6.9	3.50	3.27
AB-UPT [ 45 ]	11.4	12.79	0.47
SMART [ 46 ]	11.4	9.58	0.70

Appendix

Table 24: Model complexity and computational efficiency comparison (batch size 1).

Panel of Figure 16 — Figure 16: Lift coefficient computation for SHIFT-Wing test samples when using 32 and 256 training samples. While the 32 case seems to be too sparse to accurately predict the aerodynamic forces of test samples, both cases clearly show increased accuracy of our GMP pre-training.

Panel of Figure 17 — Figure 17: Example of random pre-training samples built as assemblies of primitive shapes (cubes, spheres, tetrahedra). The plots show the surface curvature and the normals.

(Entire dataset)	DrivAerML
Transolver++ (scratch) $\downarrow$	0.120
Transolver++ (PT) $\downarrow$	0.104
Rel. Improvement $\uparrow$	13%
AB-UPT (scratch) $\downarrow$	0.098
AB-UPT (PT) $\downarrow$	0.093
Rel. Improvement $\uparrow$	5%

Appendix

Table 25: Relative

L_{2}

error and pre-training improvement across models for the complete DrivAerML dataset.

Benchmark	Region	Improvement (%)
DrivAerML	Front	23.5
	Mid car	10.0
	Underbody	17.4
	Tail / Diffuser	19.8
SHIFT-Wing	Leading edge	21.6
	Upper surface	15.3

Appendix

Table 26: Aerodynamic regional breakdown for automotive and aerospace benchmarks. 64 training samples with GMP-Net.

Figure 18: Test loss of GMP-Net trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on DrivAerML.

Figure 19: Test loss of AB-UPT trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on DrivAerML.

Figure 20: Test loss of Transolver++ trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on DrivAerML.

Figure 21: Test loss of GMP-Net trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Wing.

Figure 22: Test loss of AB-UPT trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Wing.

Figure 23: Test loss of Transolver++ trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Wing.

Figure 24: Test loss of GMP-Net trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Crash.

Figure 25: Test loss of AB-UPT trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Crash.

Figure 26: Test loss of Transolver++ trained from scratch and with pre-trained weights. Horizontal lines indicate the minimum achieved error. Pre-trained models achieve lower error and faster convergence on SHIFT-Crash.

Panel of Figure 27 — Figure 27: Faster convergence improvement of GMP pre-training.

Panel of Figure 28 — Figure 28: Data saving due to GMP pre-training.

Figure 29: 2D KS equation rollout error comparison (sample 1). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. GMP-Net also achieves lower errors than Transolver++.

Figure 30: 2D KS equation rollout error comparison (sample 2). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. GMP-Net also achieves lower errors than Transolver++.

Figure 31: 2D KS equation rollout error comparison (sample 3). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. GMP-Net also achieves lower errors than Transolver++.

Figure 32: 2D flow around an ellipse rollout error comparison (sample 1). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. Compared to Transolver++, GMP-Net achieves lower errors in longer rollouts, but higher in the intermediate timesteps.

Figure 33: 2D flow around an ellipse rollout error comparison (sample 2). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. Compared to Transolver++, GMP-Net achieves lower errors in longer rollouts, but higher in the intermediate timesteps.

Figure 34: 2D flow around an ellipse rollout error comparison (sample 3). GMP successfully improves the accuracy over the from-scratch variant for all timesteps. Compared to Transolver++, GMP-Net achieves slightly lower errors overall.

Figure 35: Crash deformation rollout error comparison (sample 1). GMP successfully improves the accuracy over the from-scratch variant. Compared to Transolver++, GMP-Net achieves lower errors.

Figure 36: Crash deformation rollout error comparison (sample 2). GMP successfully improves the accuracy over the from-scratch variant. Compared to Transolver++, GMP-Net achieves lower errors.

Figure 37: DrivAerML surface wall shear stress absolute error comparison. 32 training samples for AB-UPT.

Figure 38: DrivAerML surface pressure absolute error comparison. 32 training samples for AB-UPT.

Figure 39: DrivAerML surface pressure absolute error comparison. 32 training samples for AB-UPT.

Figure 40: DrivAerML surface pressure absolute error comparison. 64 training samples for AB-UPT.

Figure 41: DrivAerML surface wall shear stress absolute error comparison. 64 training samples for AB-UPT.

Figure 42: DrivAerML surface pressure absolute error comparison. 64 training samples for AB-UPT.

Figure 43: DrivAerML surface wall shear stress absolute error comparison. 16 training samples for GMP-Net.

Figure 44: DrivAerML surface wall shear stress absolute error comparison. 32 training samples for GMP-Net.

Figure 45: DrivAerML surface wall shear stress absolute error comparison. 32 training samples for GMP-Net.

Figure 46: DrivAerML surface pressure absolute error comparison. 16 training samples for Transolver++.

Figure 47: DrivAerML surface pressure absolute error comparison. 16 training samples for Transolver++.

Figure 48: DrivAerML surface wall shear stress absolute error comparison. 32 training samples for Transolver++.

Figure 49: DrivAerML surface pressure absolute error comparison. 32 training samples for Transolver++.

Figure 50: SHIFT-Wing surface pressure absolute error comparison. 128 training samples for AB-UPT.

Figure 51: SHIFT-Wing surface wall shear stress absolute error comparison. 128 training samples for AB-UPT.

Figure 52: SHIFT-Wing surface pressure absolute error comparison. 128 training samples for AB-UPT.

Figure 53: SHIFT-Wing surface pressure absolute error comparison. 64 training samples for AB-UPT.

Figure 54: SHIFT-Wing surface pressure absolute error comparison. 128 training samples for GMP-Net.

Figure 55: SHIFT-Wing surface pressure absolute error comparison. 128 training samples for GMP-Net.

Figure 56: SHIFT-Wing surface wall shear stress absolute error comparison. 16 training samples for GMP-Net.

Figure 57: SHIFT-Wing surface pressure absolute error comparison. 64 training samples for GMP-Net.

Figure 58: SHIFT-Wing surface wall shear stress absolute error comparison. 16 training samples for Transolver++.

Figure 59: SHIFT-Wing surface pressure absolute error comparison. 16 training samples for Transolver++.

Figure 60: SHIFT-Wing surface wall shear stress absolute error comparison. 32 training samples for Transolver++.

Figure 61: SHIFT-Wing surfacepressure absolute error comparison. 32 training samples for Transolver++.

Figure 62: SHIFT-Crash Von Mises stress absolute error comparison. 16 training samples for AB-UPT.

Figure 63: SHIFT-Crash dissplacement absolute error comparison. 16 training samples for AB-UPT.

Figure 64: SHIFT-Crash Von Mises stress absolute error comparison. 16 training samples for AB-UPT.

Figure 65: SHIFT-Crash displacement absolute error comparison. 64 training samples for AB-UPT.

Figure 66: SHIFT-Crash displacement absolute error comparison. 64 training samples for AB-UPT.

Figure 67: SHIFT-Crash Von Mises stress absolute error comparison. 16 training samples for GMP-Net.

Figure 68: SHIFT-Crash Von Mises stress absolute error comparison. 32 training samples for GMP-Net.

Figure 69: SHIFT-Crash displacement absolute error comparison. 32 training samples for GMP-Net.

Figure 70: SHIFT-Crash displacement absolute error comparison. 64 training samples for GMP-Net.

Figure 71: SHIFT-Crash Von Mises stress absolute error comparison. 64 training samples for GMP-Net.

Figure 72: SHIFT-Crash displacement absolute error comparison. 16 training samples for Transolver++.

Figure 73: SHIFT-Crash displacement absolute error comparison. 64 training samples for Transolver++.

Figure 74: SHIFT-Crash Von Mises stress absolute error comparison. 64 training samples for Transolver++.

Figure 75: SHIFT-Crash displacement absolute error comparison. 64 training samples for Transolver++.

Figure 76: SHIFT-Crash displacement absolute error comparison. 64 training samples for Transolver++.

Figure 77: SHIFT-Crash Von Mises stress absolute error comparison. 64 training samples for Transolver++.

Explore similar work

Feb 4, 2026cs.LG

Transolver-3: Scaling Up Transformer Solvers to Industrial-Scale Geometries

Deep learning has emerged as a transformative tool for the neural surrogate modeling of partial differential equations (PDEs), known as neural PDE solvers. However, scaling these solvers to industrial-scale geometries with over

10^8

cells remains a fundamental challenge due to the prohibitive memory complexity of processing high-resolution meshes. We present Transolver-3, a new member of the Transolver family as a highly scalable framework designed for high-fidelity physics simulations. To bridge the gap between limited GPU capacity and the resolution requirements of complex engineering tasks, we introduce two key architectural optimizations: faster slice and deslice by exploiting matrix multiplication associative property and geometry slice tiling to partition the computation of physical states. Combined with an amortized training strategy by learning on random subsets of original high-resolution meshes and a physical state caching technique during inference, Transolver-3 enables high-fidelity field prediction on industrial-scale meshes. Extensive experiments demonstrate that Transolver-3 can handle meshes with over 160 million cells, achieving impressive performance across three challenging simulation benchmarks, including aircraft and automotive design tasks. Code is available at https://github.com/thuml/Transolver-3.

Hang Zhou, Haixu Wu, Haonan Shangguan +4

School of Software, BNRist, Tsinghua University, China.

Neural Partial Differential Equation Solvers Neural Solvers Mesh Generation Solvers Physics Simulation

Ask PDF

May 14, 2026cs.LG

Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning

We introduce Tadpole, a novel foundation model for three-dimensional partial differential equations (PDEs) that addresses key challenges in transferability, scalability to high dimensionality, and multi-functionality. Tadpole is pre-trained as an autoencoder on synthetic 3D PDE data generated by an efficient online data-generation framework. This enables large-scale, diverse training without storage or I/O overhead, demonstrated by scaling to an equivalent of hundreds of terabytes of training data. By autoencoding single-channel spatial crops, Tadpole learns rich and transferable representations across heterogeneous physical systems with varying numbers of state variables and spatial resolutions. Although pre-trained solely as an autoencoder, Tadpole can be efficiently applied for multiple downstream tasks beyond reconstruction, including dynamics learning and generative modeling. For dynamics learning, we propose a novel parameter-efficient fine-tuning strategy that integrates low-rank adaptation, latent-space transformations, and reintroduced skip connections, achieving accurate temporal modeling with a minimal number of trainable parameters. Tadpole demonstrates strong fine-tuning performance across various downstream tasks, highlighting its versatility and effectiveness as a foundation model for 3D PDE learning. Source code and pre-trained weights of Tadpole are available at https://github.com/tum-pbs/tadpole

Qiang Liu, Felix Koehler, Benjamin Holzschuh +1

TUM School of Computation, Information and Technology, Technical University of Munich, Garching, Germany · MCML, Munich Center for Machine Learning, Munich, Germany

Autoencoder Architectures Partial Differential Equations Parameter-Efficient Adaptation Generative Models Foundation Model Cropping

Ask PDF

Jul 9, 2026cs.LG

PGD-NO: A Neural Operator with Precomputed Geometry Decomposition for 3D Million-scale Physics Simulations

While neural PDE solvers have demonstrated significant potential for accelerating engineering simulations, existing architectures remain constrained by high memory consumption and the single node bottleneck, where the maximum processable mesh resolution is strictly limited by the VRAM of a single compute unit. To address these challenges, we propose PGD-NO, a neural operator with Precomputed Geometry Decomposition, that relocates the computational overhead of geometric encoding to a deterministic pre-computation phase. By utilizing an iterative geometry decomposition algorithm to extract geometry tokens, our model decouples feature extraction from solution querying. This architecture enables linear memory scalability, allowing high fidelity learning on meshes exceeding 10 million nodes, a scale where existing architectures typically encounter memory exhaustion. PGD-NO demonstrates competitive predictive accuracy across diverse industrial benchmarks and provides intrinsic interpretability through attention mechanisms. By effectively overcoming traditional mesh-size constraints, PGD-NO offers a robust and efficient solution for the next generation of large-scale, high-fidelity industrial design applications.

Weiheng Zhong, Jing Bi, Victor Oancea +1

Department of Civil and Environmental Engineering, Grainger College of Engineering, University of Illinois Urbana-Champaign, Urbana, IL, US · Dassault Systemes Americas, Johnston, RI, US · Siebel School of Computing and Data Science, Grainger College of Engineering, University of Illinois Urbana-Champaign, Urbana, IL, US

Neural Partial Differential Equation Solvers Neural Operators Physics Simulation Construction

Ask PDF