ISLA: A Coordinate-Free, Measure-Weighted Transformer for Boundary-Driven PDEs
Organizations: NVIDIA, Santa Clara, California, USA
Abstract
A foundational principle of physics is that all coordinate systems are equally valid: the behavior of a PDE-governed system does not depend on the origin or orientation of the frame, nor on the units or mesh used to write it down, and neither should a surrogate model. Yet the neural surrogates with the lowest errors on today's aerodynamics benchmarks lack these invariances and fail under simple transformations of the same problem: for models trained on PDEs of cars in one pose (i.e., translation and rotation) and from one mesher, rotating the full PDE raises the surface-pressure error from 4% to more than 100%, and re-sampling its boundary raises it to about 30%. We introduce ISLA (Invariant Slice Attention), a slice-attention transformer for steady, boundary-driven PDEs in which no learned operation receives a coordinate and every sum over the boundary is a quadrature. Positions enter only as distances and relative angles to learned slice anchors, lengths are measured in a unit computed from the boundary, and interior points query the boundary through a tree-accelerated learned kernel without influencing one another. With every boundary cell a token and an exact boundary read, ISLA is equivariant to rigid motions and reflections, invariant to the units of its declared data, and consistent under mesh refinement, while at every setting each query is independent of all other queries. On AeroZoo, 2,816 flows that vary in shape, size, speed, and boundary conditions, its mean reference-scaled velocity and pressure errors are at least 3.1 and 3.3 times lower than any baseline's. On AirfRANS its errors are second only to those of a boundary-integral model (GLOBE) and at least 1.6 times lower than those of Transolver and GeoTransolver.
Figures & tables
| Mean ref.-scaled | Mean rel. | Share | ||||
|---|---|---|---|---|---|---|
| Model | ref. | |||||
| Free-stream reference | 0.440 | 0.442 | 1.809 | 0.389 | 1.000 | – |
| Zero predictor | 1.203 | 1.193 | 1.809 | 1.000 | 1.000 | – |
| Transolver, SI units | 1.202 | 1.193 | 1.810 | 0.999 | 1.009 | 1% |
| Transolver, scaled | 0.617 | 0.621 | 1.419 | 0.539 | 2.192 | 41% |
| GeoTransolver, SI units | 1.184 | 1.180 | 1.811 | 0.981 | 1.015 | 1% |
| Relative error [%] | ||||
| Field | Transolver | GeoTransolver | GLOBE | ISLA |
| full (720 training cases), test cases | ||||
| 1.17 | 1.20 | 0.34 | 0.55 | |
| 1.76 | 1.46 | 0.55 | 0.57 | |
| 0.53 | 0.47 | 0.15 | 0.30 | |
| 1.14 | 1.09 | 0.28 | 0.57 | |
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Value |
|---|---|
| Reference architecture | |
| Width , blocks, slices | 512, 12, 128 |
| MLP expansion; activation | 4; GELU |
| Routing temperature | 10 |
| Boundary token budget (interior) | 200,000 |
| Boundary read | 8 heads, 16 kernel basis functions, opening angle , leaves of 4 |
| Cells per case | |||||
|---|---|---|---|---|---|
| Dataset | Cases | What varies | Simulation | Surface | Volume |
| AeroZoo ( Sharpe et al., 2026 ) | 3,815 | shape, enclosure, size, speed, | unsteady RANS, | 0.43 M | 5.0 M |
| ( to ), orientation | transition SST | (median) | (median) | ||
| AirfRANS ( Bonnet et al., 2022 ) | 1,000 | airfoil, angle of attack, | RANS (2-D) | about 1,000 | about 179,000 |
| DrivAerML ( Ashton et al., 2024c ) | 484 | car shape (16 parameters) | hybrid RANS-LES | 7.4 to 10.1 M | about 142 M |
| WindsorML ( Ashton et al., 2024a ) | 350 | Windsor-body shape (7 parameters) | wall-modeled LES | 2.0 to 2.6 M | 271 to 330 M |
| Cases (train / validation / test) | ||||
|---|---|---|---|---|
| Split | AirfRANS | DrivAerML | WindsorML | AhmedML |
| full | 720 / 80 / 200 | 400 / 34 / 50 | 280 / 35 / 35 | 400 / 50 / 50 |
| medium | – | 133 / 34 / 50 | 93 / 35 / 35 | 133 / 50 / 50 |
| scarce | 200 / 80 / 200 | 67 / 34 / 50 | 47 / 35 / 35 | 67 / 50 / 50 |
| super_scarce | – | 11 / 34 / 50 | 8 / 35 / 35 | 11 / 50 / 50 |
| out of distribution | 724 / 80 / 196 (one) | 339 / 48 / 97 | 243–244 / 35–36 / 70–71 | 350 / 50 / 100 |
| Setting | Transolver | GeoTransolver | GLOBE | ISLA |
|---|---|---|---|---|
| AirfRANS ( full , aoa_extrapolation , scarce ) | ||||
| Networks | 3, 20, 15 | 9, 9, 9 | 5, 4, 7 | 17, 13, 6 |
| Parameters | 9 M | 9 M and larger | 1.1 M (two of 0.3 M) | 67 M |
| DrivAerML | ||||
| Networks ( ) | ( full : ) | ( full : ) | 5 ( full : 9), each a joint surface and volume network | |
| Layers width | or ( ), ( ) | – | ||
| Task | Model | Step time [s] | Peak memory [GiB] | GPU-hours / run |
|---|---|---|---|---|
| AirfRANS | Transolver | 0.30 | 62 | 36 |
| GeoTransolver | 0.21 | 39 | 27 | |
| ISLA | 0.32 | 25 | 38 | |
| DrivAerML surface | Transolver | 0.27 | 62 | 22 |
| GeoTransolver | 0.39 | 78 | 28 | |
| ISLA | 0.16 | 56 | 11 |
| Ref.-scaled | Ref.-scaled | Med. | Share | |||||
| Model | Units | Parameters | median | mean | median | mean | ref. | ref. |
| Free-stream reference | – | 0.110 | 0.440 | 0.059 | 1.809 | – | – | |
| Zero predictor | – | 1.005 | 1.203 | 0.059 | 1.809 | – | – | |
| Transolver | SI units | 9.0 M | 1.007 | 1.202 | 0.059 | 1.810 | 9.19 | 1% |
| Transolver | per-case scales | 9.0 M | 0.345 | 0.617 | 0.056 | 1.419 | 1.15 | 41% |
| GeoTransolver | SI units | 27.6 M | 1.004 | 1.184 | 0.062 | 1.811 | 8.88 | 1% |
| Median reference-scaled velocity error | ||||||||
| Model | Sphere | Open box | Ground effect | Confined box | Hemisphere | Duct | Crossflow | Dual inlet |
| ( test cases ) | (63) | (95) | (107) | (59) | (29) | (41) | (34) | (87) |
| Free-stream reference | 0.034 | 0.043 | 0.074 | 0.159 | 0.194 | 0.510 | 0.969 | 1.513 |
| Transolver, scaled | 1.236 | 0.083 | 0.091 | 0.162 | 1.578 | 0.443 | 0.972 | 1.118 |
| GeoTransolver, scaled | 1.076 | 0.429 | 0.390 | 0.427 | 1.280 | 0.531 | 1.200 | 1.607 |
| GeoTransolver + BC, SI | 0.078 | 0.091 | 0.113 | 0.202 | 0.204 | 0.542 | 0.947 | 1.516 |
| Median ref.-scaled error | |||
|---|---|---|---|
| Model | |||
| Free-stream reference | 0.0057 | 35.6 | 0.0013 |
| GeoTransolver + BC, per-case scales | 0.0043 | 19.6 | 0.00070 |
| ISLA | 0.0030 | 5.81 | 0.00036 |
| Cases | ISLA | Free-stream reference | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ref. | Ref. | Med. | Share | Ref. | Ref. | ||||||||
| Split | Held out of training | train | test | median | mean | median | mean | ref. | ref. | median | mean | median | mean |
| full | nothing | 2,011 | 515 | 0.053 | 0.082 | 0.023 | 0.298 | 0.50 | 98% | 0.110 | 0.440 | 0.059 | 1.809 |
| bank_blobs | blob bodies | 1,662 | 872 | 0.059 | 0.085 | 0.024 | 0.294 | 0.61 | 99% | 0.099 | 0.409 | 0.060 | 1.655 |
| bank_zero_to_cad | CAD-part bodies | 2,250 | 224 | 0.079 | 0.117 | 0.041 | 0.362 | 0.59 | 99% | 0.138 | 0.466 | 0.083 | 2.004 |
| reynolds | 1,577 | 1,001 | 0.067 | 0.125 | 0.030 | 0.576 | 0.75 | 82% | 0.076 | 0.379 | 0.046 | 1.640 | |
| Ref.-scaled | Ref.-scaled | Med. | Share | ||||
|---|---|---|---|---|---|---|---|
| ISLA, step schedule | Parameters | median | mean | median | mean | ref. | ref. |
| 12 blocks, 128 slices | 67.5 M | 0.054 | 0.083 | 0.022 | 0.321 | 0.49 | 98% |
| 8 blocks, 64 slices | 28.2 M | 0.054 | 0.085 | 0.022 | 0.316 | 0.49 | 98% |
| Relative error [%] | Force | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Split (train) | Cases | Model | |||||||
| full (720) | validation | 80 | Transolver | 1.22 | 1.84 | 0.56 | 1.26 | 0.974 | 1.000 |
| GeoTransolver | 1.34 | 1.52 | 0.50 | 1.27 | 0.970 | 1.000 | |||
| GLOBE | 0.37 | 0.57 | 0.16 | 0.34 | 0.997 | 1.000 | |||
| ISLA | 0.57 | 0.61 | 0.27 | 0.57 | 0.993 | 1.000 | |||
| full (720) | test | 200 | Transolver | 1.17 | 1.76 | 0.53 | 1.14 | 0.899 | 1.000 |
| Rel. error: ISLA lowest other | ||||||
|---|---|---|---|---|---|---|
| Dataset | Split (train) | Cases | ||||
| DrivAerML | full (400) | val. | 1.02 | 1.05 | 1.09 | 1.05 |
| scarce (67) | val. | 1.03 | 1.04 | 1.13 | 1.12 | |
| super_scarce (11) | val. | 0.87 | 0.92 | 0.91 | 0.91 | |
| WindsorML | full (280) | val. | 0.94 | 0.99 | 0.95 | 0.93 |
| super_scarce (8) | val. | 1.19 | 1.24 | 0.97 | 1.06 | |
| Relative error [%] | Force | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Split (train) | Cases | Model | |||||||
| full (400) | validation | 34 | Transolver | 3.72 | 5.92 | 4.56 | 3.21 | 0.990 | 0.992 |
| GeoTransolver | 3.60 | 5.70 | 4.43 | 3.11 | 0.992 | 0.992 | |||
| GLOBE | 7.67 | 8.67 | 7.36 | 5.89 | 0.959 | 0.980 | |||
| ISLA | 3.67 | 5.98 | 4.83 | 3.28 | 0.993 | 0.993 | |||
| full (400) | test | 50 | Transolver | – | – | – | – | – | – |
| Relative error [%] | Force | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Split (train) | Cases | Model | |||||||
| full (280) | validation | 35 | Transolver | 7.12 | 6.51 | 5.65 | 0.06 | 0.582 | 0.939 |
| GeoTransolver | 7.00 | 6.74 | 5.65 | 0.06 | 0.583 | 0.941 | |||
| GLOBE | 7.36 | 6.52 | 6.18 | 0.06 | 0.571 | 0.944 | |||
| ISLA | 6.61 | 6.46 | 5.38 | 0.06 | 0.657 | 0.953 | |||
| medium (93) | validation | 35 | Transolver | 8.09 | 7.12 | 6.70 | 0.07 | 0.436 | 0.921 |
| Relative error [%] | Force | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Split (train) | Cases | Model | |||||||
| full (400) | validation | 50 | Transolver | 3.59 | 5.57 | 1.80 | 3.25 | 0.961 | 0.951 |
| GeoTransolver | 3.34 | 5.36 | 1.79 | 3.22 | 0.963 | 0.975 | |||
| GLOBE | 3.75 | 5.27 | 2.52 | 4.04 | 0.842 | 0.892 | |||
| ISLA | 3.07 | 5.15 | 2.32 | 3.55 | 0.968 | 0.960 | |||
| full (400) | test | 50 | Transolver | 3.41 | 5.35 | 1.71 | 3.23 | 0.921 | 0.919 |
| Relative error [%] | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| One network | Entry (ensemble) | |||||||||
| Dataset | Split (train) | Field | Transolver | GeoTransolver | GLOBE | ISLA | Transolver | GeoTransolver | GLOBE | ISLA |
| AhmedML | full (400) | 3.72 | 3.43 | 3.97 | 3.27 | 3.59 | 3.34 | 3.75 | 3.07 | |
| 5.76 | 5.56 | 5.39 | 5.35 | 5.57 | 5.36 | 5.27 | 5.15 | |||
| 1.78 | 1.88 | 2.78 | 2.50 | 1.80 | 1.79 | 2.52 | 2.32 | |||
| 3.20 | 3.42 | 4.59 | 3.87 | 3.25 | 3.22 | 4.04 | 3.55 | |||
| Relative error as stored [%] | Error (rotated, translated) stored | ||||||
|---|---|---|---|---|---|---|---|
| Task | Field | Transolver | GeoTransolver | ISLA | Transolver | GeoTransolver | ISLA |
| AirfRANS | pressure | 0.888 | 0.979 | 0.831 | 134 | 158 | 1.004 |
| velocity | 2.82 | 3.55 | 0.411 | 44 | 32 | 1.004 | |
| eddy viscosity | 1.70 | 1.68 | 1.58 | 87 | 86 | 1.000 | |
| DrivAerML surface | pressure | 4.13 | 4.00 | 4.02 | 29 | 31 | 1.000 |
| wall shear | 6.67 | 6.55 | 6.48 | 21 | 20 | 1.000 | |
| Rel. error: re-sampled stored | ||||
|---|---|---|---|---|
| Task | Field | Transolver | GeoTransolver | ISLA |
| DrivAerML surface | pressure | 7.2 | 3.6 | 1.005 |
| wall shear | 5.6 | 3.3 | 1.010 | |
| HiLiftAeroML surface | pressure | 2.8 | 3.4 | 1.000 |
| wall shear | 2.8 | 3.1 | 1.000 | |
| SHIFT-SUV surface | pressure | 1.51 | 1.37 | 0.999 |
| Relative error [%] | ||||||||
| Transolver | GeoTransolver | GLOBE | ISLA | |||||
| Presentation | ||||||||
| As stored | 4.05 | 6.20 | 3.92 | 5.93 | 8.87 | 9.67 | 3.91 | 6.40 |
| Rotated about the vertical axis | 105 | 112 | 104 | 125 | 11.0 | 11.5 | 3.91 | 6.41 |
| Rotated on | 110 | 115 | 122 | 130 | 11.1 | 11.8 | 3.91 | 6.42 |
| in principal-axis frame | 23.3 | 31.5 | 21.2 | 28.8 | – | – | – | – |
| Surface pressure (rel. ) | Wall shear stress (rel. ) | ||||||
| Model | Seeds | Stored [%] | Rotated [%] | Rot. st. | Stored [%] | Rotated [%] | Rot. st. |
| Transolver | 4 | 4.13 | 118 | 29 | 6.67 | 141 | 21 |
| Transolver, trained on rotations | 2 | 4.32 | 4.32 | 1.001 | 6.99 | 6.99 | 1.001 |
| GeoTransolver | 4 | 4.00 | 123 | 31 | 6.55 | 133 | 20 |
| GeoTransolver, trained on rotations | 2 | 4.13 | 4.14 | 1.002 | 6.75 | 6.74 | 0.999 |
| ISLA | 4 | 4.02 | 4.02 | 1.000 | 6.48 | 6.48 | 1.000 |
| Relative change of prediction | ||||||
| Surface task (DrivAerML, float32) | ||||||
| Transolver | GeoTransolver | ISLA | ||||
| Change of presentation | ||||||
| Rotation and translation | 1.2 | 1.6 | 1.2 | 1.4 | ||
| rotation alone | 1.1 | 1.6 | 1.1 | 1.4 | ||
| translation alone | ||||||
| Inference, float32 | Inference, bfloat16 | ||||
|---|---|---|---|---|---|
| Model | Parameters | time [s] | peak mem. [GiB] | time [s] | peak mem. [GiB] |
| Transolver | 47.2 M | 0.561 | 6.3 | 0.136 | 5.8 |
| GeoTransolver | 9.0 M | 0.213 | 5.4 | 0.092 | 5.1 |
| ISLA | 66.3 M | 0.368 | 6.4 | 0.084 | 4.0 |
| ISLA, blocks compiled | 66.3 M | 0.321 | 3.4 | 0.050 | 2.7 |
| Inference, float32 | Inference, bfloat16 | |||
| Query points | time [s] | peak mem. [GiB] | time [s] | peak mem. [GiB] |
| 0.63 | 16 | 0.35 | 16 | |
| 0.96 | 16 | 0.47 | 16 | |
| 3.98 | 48 | 1.57 | 31 | |
| 33.75 | 50 | 11.99 | 32 | |
| Rel. error ISLA (stored) | |||
| Variant | As stored | Rotated | Re-sampled |
| ISLA | 1.00 | 1.00 | 1.00 |
| Counts in place of measures | 0.99 | 0.99 | 11 |
| Coordinates as inputs | 0.96 | 21 | 0.97 |
| No relations to anchors | 18 | 18 | 18 |
| Body alone as the boundary | 0.98 | 0.98 | 0.98 |
PDE Surrogate Modeling
PDE surrogate modeling builds fast approximations of partial differential equation solutions for simulation, prediction, and design.
See all 361 papers