PhysICL: In-Context Learning of Physical Fields on Unstructured Meshes from Procedural Priors
Organizations: Safran Tech, Digital Sciences & Technologies Department, Rue des Jeunes Bois, Châteaufort, 78114 Magny-Les-Hameaux, France.
Abstract
Engineering design studies produce a few tens to a few hundred simulations of one problem, each on its own mesh, and their surrogates are retrained for every study. Tabular foundation models instead infer a new task from its examples in one forward pass, after pretraining on synthetic tasks. We demonstrate the feasibility of in-context learning for physical field prediction on unstructured meshes. A single model, PhysICL, pretrained solely on procedural tasks, predicts fields, with quantiles, directly on each query mesh from labeled simulations, without updating its weights. The same checkpoint accepts variable numbers of context examples, scalar inputs and input fields, and any number of output fields, while mesh size and connectivity can vary within a task. Its procedural prior draws generic laws and boundary-element physical models on morphed 2D and 3D template meshes, evaluated only at the points the model samples, fast enough that tasks generated on the fly by CPU workers keep the GPUs busy throughout pretraining. On five PLAID datasets never seen in pretraining, its field error decreases as the context grows and, from , is below that of the nearest context simulation on every dataset and context draw. It is on par with per-dataset reduced-order models on two of them, behind them where the fields are low-rank. On AhmedML surfaces with a median of 1.1 million nodes, twenty times the largest pretraining mesh, its fixed spatial budgets keep the peak GPU memory at at 4.27 GiB, against 4.07 GiB on the 6,000-node meshes of 2D_MSHE. Finally, on 2D_MSHE, far from the prior, fine-tuning on procedural tasks that reproduce its geometry and loads, without any of its outputs, at a tenth of the pretraining cost, divides the stress error by 2.7.
Figures & tables
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Parameters |
|---|---|
| Patch encoder | 0.72 M |
| Context labels in patches | 1.05 M |
| Row encoder | 2.94 M |
| (1 block) | 1.10 M |
| (4 blocks) | 4.03 M |
| (6 blocks, width 512) | 12.87 M |
| Choice | Value | Source | Ablation |
| Variable identity | random codes | Appendix A.1 ; cf. ClimaX (learned variable embeddings) ( Nguyen et al., 2023 ) | — |
| Outputs | one field per task, decoded alone | avoids cross-field contamination (Section 4 ) | — |
| Output read (a query carrying the output’s code) | on | Perceiver IO | off: |
| Variable pooling | value-coded attention | ClimaX aggregation, Appendix A.3 | sum pooling: |
| Encoder nodes per patch | 8 | inverse-inclusion estimator | 8 against 16: , 1.47 the steps/s; 32: , slower |
| Decoded nodes per query | 1,024 | loss estimator | 512: ; 2,048: |
| Dataset | Field | Transolver-3 | PhysICL | ratio |
|---|---|---|---|---|
| 2D_MSHE | u1 | 0.0092 | 0.05 | 5.5 |
| u2 | 0.0097 | 0.052 | 5.4 | |
| P11 | 0.027 | 0.057 | 2.1 | |
| P12 | 0.041 | 0.11 | 2.6 | |
| P22 | 0.028 | 0.06 | 2.1 | |
| P21 | 0.041 | 0.11 | 2.5 |
| Group | ratio | front | ratio | front |
|---|---|---|---|---|
| radial, plate | 0.99 | 3/4 | 0.93 | 2/4 |
| radial, heat | 0.25 | 4/4 | 0.92 | 3/4 |
| radial, flow/Darcy | 0.44 | 3/4 | 1.2 | 3/4 |
| shapes, plate | 0.46 | 4/4 | 1.2 | 3/4 |
| shapes, heat | 2.2 | 1/4 | 3.1 | 2/4 |
| Dataset | Dim. | Nodes | Shared | Fields | Scalars |
|---|---|---|---|---|---|
| 2D_profile | 2 | 37k | no | 4 | 0 |
| 2D_MSHE | 2 | 6k | no | 7 | 1 |
| Tensile2d | 2 | 10k | no | 5 | 3 |
| VKI-LS59 | 2 | 36k | yes | 2 | 6 |
| Rotor37 | 3 surf. | 30k | yes | 3 | 3 |
| AhmedML | 3 surf. | 1108k | no | 6 | 4 |
| Dataset | Context | ||||||
|---|---|---|---|---|---|---|---|
| Rotor37 | correct | 0.0482 | 0.0336 | 0.0261 | 0.0228 | 0.0206 | 0.0193 |
| outputs permuted | 0.101 ( 2.1) | 0.0836 ( 2.5) | 0.0723 ( 2.8) | 0.0653 ( 2.9) | 0.0587 ( 2.9) | 0.0584 ( 3.0) | |
| inputs permuted | 0.14 ( 2.9) | 0.089 ( 2.6) | 0.0701 ( 2.7) | 0.0654 ( 2.9) | 0.0613 ( 3.0) | 0.058 ( 3.0) | |
| VKI-LS59 | correct | 0.113 | 0.0794 | 0.067 | 0.0536 | 0.0503 | 0.0452 |
| outputs permuted | 0.221 ( 2.0) | 0.182 ( 2.3) | 0.198 ( 2.9) | 0.198 ( 3.7) | 0.193 ( 3.8) | 0.169 ( 3.7) | |
| inputs permuted | 0.2 ( 1.8) | 0.128 ( 1.6) | 0.105 ( 1.6) | 0.0857 ( 1.6) | 0.0851 ( 1.7) | 0.0798 ( 1.8) |
| Dataset | PhysICL | Nearest | Best | Floor | PhysICL (s) | Best (s) | |
|---|---|---|---|---|---|---|---|
| 2D_profile | 16 | 0.116 0.0088 | 0.228 | 0.129 | 0.101 | 15.5 / 0.95 | 7.88 / 3.78 |
| 2D_MSHE | 16 | 0.502 0.019 | 1.23 | 0.628 | 0.505 | 15.5 / 1.37 | 11.2 / 3.57 |
| Tensile2d | 16 | 0.00249 0.00067 | 0.0308 | 0.000534 | 0.000122 | 16.1 / 1.18 | 4.96 / 0.913 |
| VKI-LS59 | 16 | 0.0794 0.083 | 0.1 | 0.0702 | 0.00917 | 16.3 / 0.999 | 1.6 / 0.478 |
| Rotor37 | 16 | 0.0336 0.0015 | 0.0383 | 0.00761 | 0.00282 | 16.2 / 0.906 | 4.32 / 2.15 |
| 2D_profile | 64 | 0.064 0.0065 | 0.144 | 0.0693 | 0.0416 | 17.5 / 0.895 | 22.3 / 8.87 |
| pretrained | proxy | augmented proxy | best baseline | PCA floor | ||
| median NMSE, all nodes | ||||||
| P11 | 16 | 0.775 0.020 | 0.371 0.022 | 0.314 0.044 | 0.866 | 0.656 |
| P11 | 64 | 0.674 0.023 | 0.355 0.016 | 0.267 0.020 | 0.632 | 0.459 |
| P11 | 250 | 0.585 0.006 | 0.349 0.003 | 0.220 0.004 | 0.544 | 0.249 |
| P12 | 16 | 0.873 0.018 | 0.378 0.015 | 0.337 0.007 | 0.811 | 0.685 |
| P12 | 64 | 0.771 0.027 | 0.365 0.008 | 0.278 0.014 | 0.644 | 0.432 |
| Dataset | Field output | PhysICL | probes/PCA + GP | probes/PCA + Kumo | nearest context | PCA floor |
|---|---|---|---|---|---|---|
| 2D_profile | Mach | 0.069 | 0.252 | 0.0824 | 0.166 | 0.0499 |
| 2D_profile | Pressure | 0.0573 | 0.198 | 0.0603 | 0.133 | 0.0393 |
| 2D_profile | Velocity-x | 0.0705 | 0.236 | 0.074 | 0.162 | 0.0467 |
| 2D_profile | Velocity-y | 0.059 | 0.178 | 0.0569 | 0.116 | 0.0304 |
| 2D_MSHE | P11 | 0.643 | 0.773 | 0.632 | 1.52 | 0.459 |
| 2D_MSHE | P12 | 0.759 | 0.787 | 0.644 | 1.33 | 0.432 |
| Dataset | Scalar output | PhysICL | probes/PCA + GP | probes/PCA + Kumo | nearest context |
|---|---|---|---|---|---|
| 2D_MSHE | effective_energy | 0.433 | 0.854 | 0.922 | -1.26 |
| Tensile2d | max_U2_top | 0.943 | 0.999 | 1 | 0.848 |
| Tensile2d | max_sig22_top | 0.999 | 0.999 | 0.999 | 0.477 |
| Tensile2d | max_von_mises | 0.992 | 0.991 | 0.992 | 0.931 |
| VKI-LS59 | Pr | 0.637 | 0.871 | 0.924 | 0.556 |
| VKI-LS59 | Q | 0.996 | 0.999 | 0.999 | 0.71 |
| axis_nn / PhysICL | interpolation / PhysICL | |||||
|---|---|---|---|---|---|---|
| template | aspect | pockets | ||||
| 165 | 1.92 | 2/2 | 2.32 | 2.08 | 0.24 | 0.24 |
| 213 | 1.51 | 0/2 | 2.07 | 1.55 | 0.15 | 0.18 |
| 233 | 1.55 | 0/2 | 5.12 | 3.92 | 0.18 | 0.24 |
| 315 | 1.99 | 1/2 | 1.85 | 0.64 | 0.16 | 0.15 |
| 329 | 1.71 | 0/2 | 0.67 | 0.90 | 0.20 | 0.29 |
PDE Surrogate Modeling
PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.
See all 361 papers