Large Geometry Models Are Priors For Fast Physics Pretraining
Organizations: Hassana Labs.
Abstract
Language and vision models became data-efficient by pretraining on plentiful unlabelled data; physical field models have such data, over ten million 3D shapes, but geometry pretraining designs its targets rather than deriving them from the physics. Euler-Lagrange geometry pretraining derives them: for attached flow, a shape enters the flow's Euler-Lagrange equation only through coefficients that distance and normal line (the normal up to sign) fix to first order. Their reconstruction error therefore bounds what a representation loses about any field, and fine-tuning on costly simulations need only assemble the global field. We realise it in AREALS, a masked surface autoencoder whose areal tokens carry position, normal and area, pretrained self-supervised and sign-free on 61,822 raw meshes. Because the operating condition enters only through the far field, sensors can supply it: AREALS completes a hidden field in one forward pass without condition labels. It matches the accuracy of the same architecture trained from scratch with half the simulations on DrivAerML and DrivAerNet++ and about a quarter on WindsorML. On DoubleDelta wings, 20 sensors give and lower field and force error than GABI, a Bayesian inversion method with a learned prior, and beyond the training angles pretraining halves the error.
Figures & tables
| Cars | (AREALS) | , Hessian | Patch |
|---|---|---|---|
| 18 | 0.277 / 0.443 | 0.301 / 0.475 | 0.457 / 0.442 |
| 35 | 0.228 / 0.381 | 0.246 / 0.388 | 0.384 / 0.379 |
| 69 | 0.193 / 0.265 | 0.206 / 0.301 | 0.307 / 0.273 |
| 137 | 0.178 / 0.228 | 0.179 / 0.215 | 0.257 / 0.223 |
| 274 | 0.158 / 0.203 | 0.161 / 0.182 | 0.221 / 0.196 |
| Sensor-conditioned | References | ||||
|---|---|---|---|---|---|
| Benchmark | GABI | Scratch | AREALS | Shape | Condition |
| DoubleDelta, nine angles | 0.642 | 0.222 | 0.164 | 0.302 | 0.137 |
| DoubleDelta, – | — | 0.409 | 0.206 | 0.373 | 0.193 |
| BlendedNet++ | — | 0.212 | 0.176 | 0.743 | 0.066 |
Appendix figures & tables31 assets
Supplementary material from the paper’s appendix.
Appendix
| Result | Geometry checkpoint | Field training | Evaluation | Observations | Metric | Seeds |
|---|---|---|---|---|---|---|
| Unreported angle ( Figure D.1 , Tables 2 , D.4 , D.2 and D.5 ) | Production vs. none | 2,160 cases, 40 epochs per model | 144 test cases (16 wings) | 0–500 sensors; patches | Hidden nRMSE; force error | 3; scratch and references 1 |
| Held-out angles ( Table D.6 ) | Production | 1,200 cases (odd angles) | 64 cases (16 wings, even angles) | 5–500 sensors | Hidden nRMSE | 1 |
| Explicit condition inference ( Table D.1 ) | Production | True-condition model, 228 wing families, – | 16 wings at – ; timing on 4 validation wings | 20–500 sensors | Hidden nRMSE; time per case | 3 |
| Extrapolated angles ( Tables 2 , C.5 and C.1 ) | Production vs. none | 228 wing families, – | 16 wings at – | 5–500 sensors | Hidden nRMSE | 3 |
| BlendedNet++ ( Tables 2 , C.5 and C.1 ) | Production vs. none | 8,736 aircraft | 1,872 unseen aircraft, higher Mach | 5–500 sensors | Hidden nRMSE | 3 |
| DrivAerML official split ( Tables C.5 and C.1 ) | Production vs. none | 400 cars | 50 test cars | 5–500 sensors | Hidden nRMSE | 3 |
| Benchmark | Test set | Shape only | Scratch | AREALS | True condition | AREALS scratch | |
|---|---|---|---|---|---|---|---|
| DoubleDelta | 16 wings, – | 5 | 0.373 | 0.409 | 0.310 | 0.193 | |
| 20 | 0.373 | 0.409 | 0.206 | 0.193 | |||
| 100 | 0.375 | 0.411 | 0.178 | 0.194 | |||
| 500 | 0.383 | 0.420 | 0.187 | 0.200 | |||
| BlendedNet++ | 1,872 aircraft, high Mach | 5 | 0.742 | 0.281 | 0.240 | 0.066 | |
| 20 | 0.743 | 0.212 | 0.176 | 0.066 |
| Geometry trunk | Frozen | Fine-tuned |
|---|---|---|
| Random initialisation | 4.7 | 25.4 |
| AREALS pretraining | 30.4 | 82.6 |
| Observations | Random trunk | AREALS pretraining |
|---|---|---|
| 5 sensors | 6.9 | |
| 20 sensors | 0.8 | 18.5 |
| 100 sensors | 2.8 | 35.2 |
| 500 sensors | 9.3 | 54.8 |
| 60% hidden patches | 10.7 | 36.3 |
| Mean | 4.7 | 30.4 |
| Benchmark | Masked | Unmasked | Masked unmasked | |
|---|---|---|---|---|
| DoubleDelta, – | 0 | 0.4306 | 0.4211 | |
| 5 | 0.3119 | 0.3036 | ||
| 20 | 0.2081 | 0.2057 | ||
| 100 | 0.1782 | 0.1801 | ||
| 500 | 0.1867 | 0.1847 | ||
| DrivAerML, official | 0 | 0.2403 | 0.2412 |
| Benchmark | Train | Scratch | AREALS | Reduction |
|---|---|---|---|---|
| DoubleDelta, – | 228 | 0.409 | 0.206 | 49.6% |
| BlendedNet++, high Mach | 8,736 | 0.212 | 0.176 | 16.8% |
| DrivAerML, official | 400 | 0.295 | 0.242 | 18.0% |
| DrivAerML, reserved ∗ | 420 | 0.284 | 0.226 | 20.2% |
| AREALS | Explicit inference | Time per case (s) | ||||
| Benchmark | nRMSE | 95% interval | nRMSE | 95% interval | AREALS / explicit | |
| DoubleDelta, – | 5 | 0.310 | — | — | — | — |
| 20 | 0.2060 0.0035 | [0.181, 0.230] | 0.2999 0.0007 | [0.264, 0.336] | 0.122 / 0.356 | |
| 100 | 0.1775 0.0061 | [0.154, 0.202] | 0.2999 0.0023 | [0.263, 0.337] | 0.123 / 0.355 | |
| 500 | 0.1872 0.0086 | [0.163, 0.211] | 0.3066 0.0002 | [0.268, 0.345] | 0.123 / 0.354 | |
| DoubleDelta, nine angles | 20 | 0.1637 0.0030 | — | — | — | — |
| Sensors | Geometry only | Measurements | True angle |
|---|---|---|---|
| 5 | 0.3021 | 0.2111 | 0.1373 |
| 20 | 0.3023 | 0.1637 | 0.1374 |
| 100 | 0.3036 | 0.1434 | 0.1383 |
| 500 | 0.3101 | 0.1410 | 0.1426 |
| Budget | AREALS, per model | GABI, published schedule |
|---|---|---|
| Epochs | 40 | 455 |
| Batch size | 8 | 100 |
| Optimiser updates | 10,800 | 10,010 |
| Field-example exposures | 86,400 | 982,800 |
| Training seeds | 3 | 3 per graph |
| nRMSE | Force error (%) | |||
|---|---|---|---|---|
| Sensors | AREALS | GABI | AREALS | GABI |
| 0 | 0.300 | 0.858 | 15.5 | 25.3 |
| 5 | 0.211 | 0.745 | 8.6 | 21.4 |
| 20 | 0.164 | 0.642 | 3.7 | 14.7 |
| 100 | 0.143 | 0.537 | 2.2 | 12.2 |
| 500 | 0.141 | 0.515 | 1.5 | 10.6 |
| AREALS | GABI | ||||||
|---|---|---|---|---|---|---|---|
| Observed | Geometry only | Measurements | True angle | Straight line | Along mesh | Along mesh straight line | GABI AREALS |
| Nothing | 0.302 | 0.300 0.001 | 0.137 | 0.914 | 0.858 0.037 | [ , ] | 0.558 [0.422, 0.696] |
| 5 sensors | 0.302 | 0.211 0.004 | 0.137 | 0.834 | 0.745 0.104 | [ , ] | 0.534 [0.447, 0.617] |
| 20 sensors | 0.302 | 0.164 0.003 | 0.137 | 0.736 | 0.642 0.155 | [ , ] | 0.479 [0.402, 0.556] |
| 100 sensors | 0.304 | 0.143 0.001 | 0.138 | 0.648 | 0.537 0.117 | [ , ] | 0.393 [0.342, 0.442] |
| 500 sensors | 0.310 | 0.141 0.002 | 0.143 | 0.623 | 0.515 0.113 | [ , ] | 0.374 [0.322, 0.424] |
| Sensors | Geometry only | Measurements | True angle | [95% interval] |
|---|---|---|---|---|
| 5 | 0.2832 | 0.1947 | 0.1333 | 63.1% [41.3, 83.0%] |
| 20 | 0.2834 | 0.1550 | 0.1334 | 91.7% [86.6, 98.6%] |
| 100 | 0.2846 | 0.1464 | 0.1343 | 96.4% [90.3, 104.4%] |
| 500 | 0.2908 | 0.1433 | 0.1384 | 99.8% [94.5, 107.3%] |
| Transfer pathway | nRMSE |
|---|---|
| Point-MAE encoder (21.8M) + fresh query decoder | 0.2556 |
| AREALS encoder (2.3M) + fresh query decoder | 0.2708 |
| AREALS encoder + pretrained query decoder | 0.2399 |
| Perturbation | 256 tokens | 1,024 tokens |
|---|---|---|
| Point order | 2.27e-08 | 1.58e-08 |
| Coupled coplanar subdivision | 6.60e-09 | 5.20e-09 |
| Independent surface resampling | 0.01316 | 0.00921 |
| Resampling, fixed anchors/frame/surfels | 0.00467 | 0.00217 |
| Full | 60% of the surface hidden | |||
|---|---|---|---|---|
| Prior | input | All | Hidden | Repeated parts |
| Surfels, 8 per token | 5.7 | 12.4 | 17.6 | 10.9 |
| + symmetry maps | 5.7 | 11.4 | 15.7 | 6.9 |
| Analytic shapes | Meshes | |||
|---|---|---|---|---|
| Radius | Exact Betti numbers | Area error (%) | Path error (%) | Area error (%) |
| 0.10 | 3 of 3 | 1.3 | 2.4 | 7.1 |
| 0.14 | 3 of 3 | 0.8 | 0.4 | 5.6 |
| 0.20 | 3 of 3 | 0.3 | 0.0 | 4.9 |
| 0.28 | 3 of 3 | 0.1 | 0.1 | 3.5 |
| 0.40 | 3 of 3 | 0.1 | 0.5 | 2.9 |
| Below 0.05 | Below 0.1 | |||
| Shape | AREALS | Visible disks | AREALS | Visible disks |
| Mesh 1 | 96.1 | 9.1 | 99.5 | 23.9 |
| Mesh 2 | 92.2 | 11.5 | 95.2 | 30.3 |
| Mesh 3 | 78.9 | 9.2 | 83.1 | 23.4 |
| Mesh 4 | 97.6 | 8.9 | 99.0 | 20.3 |
| Mesh 5 | 93.7 | 16.4 | 97.6 | 37.2 |
| Checkpoint | Distance ratio | Within | Beyond | Beyond | |
|---|---|---|---|---|---|
| Decoder | Pretrained | 0.254 [0.251, 0.256] | 51.5 | 17.0 | 3.5 |
| Fine-tuned | 0.406 [0.400, 0.412] | 27.5 | 40.0 | 11.0 | |
| From scratch | 0.786 [0.775, 0.796] | 10.6 | 72.4 | 42.3 | |
| In proportion to area | 1 | 3.2 | 88.5 | 63.6 | |
| Encoder | Pretrained | 0.524 [0.522, 0.527] | 23.4 | 46.7 | 21.2 |
| Fine-tuned | 0.648 [0.642, 0.654] | 18.9 | 59.3 | 31.9 |
| Geometry prior | Tokens | Points | Rel. |
|---|---|---|---|
| AREALS | 256 | 4,096 | 0.3283 |
| AeroJEPA tutorial | 256 | 4,096 | 0.3338 |
| AREALS | 1,024 | 4,096 | 0.3244 |
| Utonia ( Zhang et al., 2026a ) | 1,024 | 4,096 | 0.3294 |
| Point-MAE ( Pang et al., 2022 ) | 1,024 † | 1,024 | 0.3353 |
| AREALS | 1,024 | 65,536 | 0.3198 |
| Arm | Parameters | Targets | Peak rate | Weight decay | Distance or position | Normal line | Other |
|---|---|---|---|---|---|---|---|
| AREALS | 3,215,503 | , | 0.05 | distance nRMSE 0.0089 | — | ||
| Hessian | 3,217,948 | , , | 0.05 | distance nRMSE 0.0183 | error 0.495 (label floor 0.007) | ||
| Patch | 2,268,298 | , position-constrained | 0 | position RMS 0.0013 | relative RMS 0.019 |
| Test | Validation | ||||||
|---|---|---|---|---|---|---|---|
| Arm | Cars | Pretrained | Scratch | Gain | Difference | 95% interval | Pretrained / scratch |
| AREALS ( ) | 18 | 0.2773 | 0.4431 | 37.4% | 0.2649 / 0.4413 | ||
| 35 | 0.2281 | 0.3809 | 40.1% | 0.2114 / 0.3602 | |||
| 69 | 0.1934 | 0.2649 | 27.0% | 0.1769 / 0.2649 | |||
| 137 | 0.1778 | 0.2279 | 22.0% | 0.1506 / 0.2037 | |||
| 274 | 0.1582 | 0.2026 | 21.9% | 0.1410 / 0.1837 | |||
| Three fine-tuning seeds | Fine-tuning seed 0 | |||||
|---|---|---|---|---|---|---|
| Cars | Coefficient prior | Scratch | Difference [95% interval] | Coefficients | alone | Difference [95% interval] |
| 18 | 0.2789 | 0.4359 | 0.2637 | 0.2648 | ||
| 35 | 0.2304 | 0.3729 | 0.2222 | 0.2248 | ||
| 69 | 0.1896 | 0.2649 | 0.1874 | 0.1876 | ||
| 137 | 0.1690 | 0.2279 | 0.1730 | 0.1688 | ||
| 274 | 0.1565 | 0.1915 | 0.1552 | 0.1482 | ||
| GeoPT | AREALS | |
|---|---|---|
| Pretraining task | Regress geometric features, such as signed distance, along straight trajectories of points given random velocities, halted at the wall | Complete raw surfaces with 60% of the area hidden: unsigned distance and normal line at any point |
| Origin of the targets | Designed: synthetic dynamics that obey no conservation law | Derived: in wall coordinates a change of shape is a change of the flow equation’s coefficients, fixed to first order by distance and normal line ( Lemmas 1 and 2 ); reconstruction is sufficient and both targets are necessary ( Proposition 1 ) |
| Network inputs | Geometry and a velocity field, in pretraining and in fine-tuning | Geometry; once fine-tuned, also field values and their presence flags |
| Geometry-only model | None | The pretrained prior, which contains no field and no condition |
| Operating condition | Required: the velocity input carries the freestream direction and speed | Never given, in training or at test |
| Measurements | None | Any number of sensors, in one forward pass |
Physics-Informed ML
Physics-informed machine learning combines data-driven models with physical laws to solve, predict, and infer scientific systems.
See all 895 papers