PANO: Parallel Latent Space Neural Operator for Solving PDEs
Organizations: University of Science and Technology Beijing · Beijing Engineering Research Center of Industrial Spectrum Imaging
Abstract
Aggregating observations into a limited number of latent tokens and learning physical interactions in the latent space has emerged as an effective approach for Transformer-based neural operators to solve Partial Differential Equation (PDE) problems involving large-scale observations. However, existing methods typically compress observations using a single aggregation pattern, making physical information not retained by that pattern invisible to the latent space and thereby constraining physical interactions within it. To address this limitation, we propose PArallel Latent Space Neural Operator (PANO), a novel neural operator for solving PDEs that formulates physical interactions among observations as an additive combination of interaction results from parallel latent spaces with shared parameters. Specifically, PANO transforms observations into multiple distinct compressed representations through independent mappings, learns physical interactions in parallel based on these representations, and finally fuses the decoded results to predict the target physical field. Within PANO, we further propose Coupled Assignment Projection (CAP) to construct bidirectionally consistent encoding and decoding mappings, and Pairwise Latent Space Attention Layer (PaLA) to jointly update latent representations of each latent space through adaptive cross-space interactions. Extensive experiments on nine benchmarks spanning diverse physical systems, geometries, and observation scales demonstrate that PANO achieves state-of-the-art performance. Notably, PANO reduces equivalent plastic strain prediction errors on Unilateral Stamping by 28.3% (or more) and lift-coefficient prediction errors on AirCraft by 21.4% (or more) compared with existing methods. Code is available at https://github.com/ZijiangY1116/PANO.
Figures & tables
| Method | Darcy85 | Darcy141 | Darcy211 |
| LNO | 0.0081 | 0.0074 | 0.0073 |
| Transolver | 0.0055 | 0.0064 | 0.0063 |
| LinearNO | 0.0050 | 0.0053 | 0.0053 |
| PANO-P (ours) | 0.0055 | 0.0042 | 0.0041 |
| PANO-D (ours) | 0.0050 | 0.0039 | 0.0039 |
| Method | Standard Setting | OOD Reynolds | OOD Angles | |||||
| Vol. | Surf. | |||||||
| GNOT ( Hao et al., 2023 ) | 0.0049 | 0.0152 | 0.1992 | 0.9942 | 0.3268 | 0.9865 | 0.3497 | 0.9868 |
| GINO ( Li et al., 2023c ) | 0.0297 | 0.0482 | 0.1821 | 0.9958 | 0.4180 | 0.9645 | 0.2583 | 0.9923 |
| LNO ( Wang and Wang, 2024 ) | 0.0214 | 0.0268 | 0.1480 | 0.9744 | 0.7440 | 0.9399 | 0.3732 | 0.9905 |
| Transolver ( Wu et al., 2024 ) | 0.0023 | 0.0085 | 0.1230 | 0.9978 | 0.1622 | 0.9904 | 0.2438 | 0.9948 |
| LinearNO ( Hu et al., 2026 ) | 0.0011 | 0.0077 | 0.0491 | 0.9992 | 0.2400 | 0.9951 | 0.0987 | 0.9963 |
| Method | Geometry | Stress | PEEQ | |||
| Metal | Rubber | Metal | Rubber | Metal | Rubber | |
| OFormer ( Li et al., 2022 ) | 0.0119 | 0.0098 | 0.1103 | 0.0981 | 0.3269 | 0.2675 |
| ONO ( Xiao et al., 2023 ) | 0.0433 | 0.0437 | 0.1377 | 0.1311 | 0.5140 | 0.4748 |
| MeshGraphNet ( Pfaff et al., 2020 ) | 0.0391 | 0.0383 | 0.1252 | 0.1417 | 0.5805 | 0.5694 |
| Transolver ( Wu et al., 2024 ) | 0.0079 | 0.0075 | 0.0839 | 0.0862 | 0.2938 | 0.2693 |
| Unisoma ( Tao et al., 2025 ) | 0.0071 | 0.0063 | 0.0807 | 0.0773 | 0.2591 | 0.2114 |
| Method | Vol. | Surf. |
| LNO | 0.0269 | 0.0870 |
| Transolver | 0.0221 | 0.0785 |
| Transolver++ | 0.0226 | 0.0800 |
| LinearNO | 0.0194 | 0.0754 |
| PANO-P (ours) | 0.0214 | 0.0817 |
| PANO-D (ours) | 0.0187 | 0.0784 |
| Model | Params. (M) | GFLOPs | Forward (ms) |
| Transolver | 3.86 | 402.03 | 111.20 |
| Transolver++ | 3.34 | 356.36 | 127.19 |
| LinearNO | 3.32 | 362.70 | 154.76 |
| PANO-P | 1.99 | 171.78 | 61.20 |
| PANO-D | 8.31 | 396.21 | 75.81 |
| Params. | Geometry | Stress | PEEQ | |||||||
| (M) | Metal | Rubber | Metal | Rubber | Metal | Rubber | ||||
| 512 | 1 | 512 | 96 | 1.26 | 0.0066 | 0.0057 | 0.0769 | 0.0713 | 0.2256 | 0.1876 |
| 512 | 2 | 256 | 96 | 1.54 | 0.0065 | 0.0056 | 0.0772 | 0.0709 | 0.2192 | 0.1814 |
| 512 | 4 | 128 | 96 | 1.69 | 0.0064 | 0.0055 | 0.0761 | 0.0710 | 0.2224 | 0.1824 |
| 2048 | 1 | 2048 | 96 | 1.56 | 0.0062 | 0.0054 | 0.0765 | 0.0725 | 0.2132 | 0.1805 |
| 2048 | 1 | 2048 | 192 | 5.42 | 0.0064 | 0.0054 | 0.0750 | 0.0691 | 0.2157 | 0.1770 |
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
| Notation | Description |
| Domain, dimensions, and observations | |
| Physical domain of the PDE problem. | |
| Number of observation points. | |
| Input observation dimension at each point. | |
| Feature dimension of point tokens and latent tokens. | |
| Number of parallel latent spaces in each PANO-Block. | |
| Notation | Description |
| Probability measure on the physical domain . | |
| Bounded measurable point-token field. | |
| Spatial location at which the output field is evaluated. | |
| Integration variable over the physical domain. | |
| Operator that averages the decoded outputs of parallel latent spaces using CAP and shared-parameter self-attention. | |
| Input-dependent integral kernel induced by CAP encoding, self-attention, and CAP decoding in space . |
| Block | Space 1 | Space 2 | Space 3 | Space 4 | Jointly |
| Block 1 | 7.88% | 0.09% | 0.55% | 10.22% | 0.000014% |
| Block 2 | 0.11% | 0.66% | 0.18% | 0.69% | 0.000083% |
| Block | Target | Space 1 | Space 2 | Space 3 | Space 4 |
| Block 1 | 1 | – | 2.7521% | 1.2822% | 10.8355% |
| 2 | 0.0096% | – | 0.0001% | 0.0055% | |
| 3 | 0.1095% | 0.0443% | – | 0.1870% | |
| 4 | 12.7288% | 3.9956% | 1.7093% | – | |
| Block 2 | 1 | – | 0.0012% | 0.0013% | 0.0002% |
| 2 | 0.0232% | – | 0.0106% | 0.0046% |
| Block | Space 1 | Space 2 | Space 3 | Space 4 |
| Block 1 | 6.1682% | 3.6895% | 2.9733% | 5.6461% |
| Block 2 | 0.9488% | 0.8416% | 1.0126% | 1.0638% |
| Method | Airfoil | Plasticity | NS2D | Elasticity |
| FNO ( Li et al., 2020a ) | - | - | 0.1556 | - |
| GEO-FNO ( Li et al., 2023b ) | 0.0138 | 0.0074 | 0.1556 | 0.0229 |
| F-FNO ( Tran et al., 2021 ) | 0.0078 | 0.0047 | 0.2322 | 0.0263 |
| Galerkin ( Cao, 2021 ) | 0.0118 | 0.0120 | 0.1401 | 0.0240 |
| OFormer ( Li et al., 2022 ) | 0.0183 | 0.0017 | 0.1705 | 0.0183 |
| GNOT ( Hao et al., 2023 ) | 0.0076 | 0.0336 | 0.1380 | 0.0086 |
| Method | Vol. | Surf. | ||
| MLP | 0.0081 | 0.0200 | 0.2108 | 0.9932 |
| PointNet ( Qi et al., 2017 ) | 0.0253 | 0.0996 | 0.1973 | 0.9919 |
| Graph U-Net ( Bronstein et al., 2016 ) | 0.0076 | 0.0144 | 0.1677 | 0.9949 |
| MeshGraphNet ( Pfaff et al., 2020 ) | 0.0214 | 0.0387 | 0.2252 | 0.9945 |
| GNO ( Li et al., 2020b ) | 0.0269 | 0.0405 | 0.2016 | 0.9938 |
| GEO-FNO ( Li et al., 2023b ) | 0.0361 | 0.0301 | 0.6161 | 0.9257 |
| Method | OOD Reynolds | OOD Angles | ||||||
| Vol. | Surf. | Vol. | Surf. | |||||
| MLP | 0.0669 | 0.1153 | 0.6205 | 0.9578 | 0.1309 | 0.3311 | 0.4128 | 0.9572 |
| PointNet ( Qi et al., 2017 ) | 0.0838 | 0.1403 | 0.3836 | 0.9806 | 0.2021 | 0.4649 | 0.4425 | 0.9784 |
| Graph U-Net ( Bronstein et al., 2016 ) | 0.0538 | 0.1168 | 0.4664 | 0.9645 | 0.0979 | 0.2391 | 0.3756 | 0.9816 |
| MeshGraphNet ( Pfaff et al., 2020 ) | 0.2789 | 0.2382 | 1.7718 | 0.7631 | 0.4902 | 1.1071 | 0.6525 | 0.8927 |
| GNO ( Li et al., 2020b ) | 0.0833 | 0.1562 | 0.4408 | 0.9878 | 0.1626 | 0.2359 | 0.3038 | 0.9884 |
| Method | Geometry | Stress | PEEQ | |||
| Metal | Rubber | Metal | Rubber | Metal | Rubber | |
| GEO-FNO ( Li et al., 2023b ) | 0.0109 | 0.0154 | 0.1113 | 0.1124 | 0.3771 | 0.3574 |
| GINO ( Li et al., 2023c ) | 0.0188 | 0.0219 | 0.1262 | 0.1438 | 0.4651 | 0.4885 |
| GNO ( Li et al., 2020b ) | 0.0426 | 0.0424 | 0.1351 | 0.1814 | 0.6413 | 0.6859 |
| LSM ( Wu et al., 2023 ) | 0.0367 | 0.0357 | 0.1327 | 0.1335 | 0.5889 | 0.6026 |
| Galerkin ( Cao, 2021 ) | 0.1007 | 0.1010 | 0.1725 | 0.2794 | 0.7986 | 0.8555 |
| Method | Vol. | Surf. | ||
| MLP | 0.0512 | 0.1304 | 0.0307 | 0.9496 |
| PointNet ( Qi et al., 2017 ) | 0.0494 | 0.1104 | 0.0298 | 0.9583 |
| Graph U-Net ( Bronstein et al., 2016 ) | 0.0471 | 0.1102 | 0.0226 | 0.9725 |
| GNO ( Li et al., 2020b ) | 0.0383 | 0.0815 | 0.0172 | 0.9834 |
| GEO-FNO ( Li et al., 2023b ) | 0.1670 | 0.2378 | 0.0664 | 0.8280 |
| Galerkin ( Cao, 2021 ) | 0.0339 | 0.0878 | 0.0179 | 0.9764 |
| Method | Surf. | ||
| PointNet ( Qi et al., 2017 ) | 0.169 | 0.095 | 0.982 |
| Graph U-Net ( Bronstein et al., 2016 ) | 0.161 | 0.063 | 0.953 |
| MeshGraphNet ( Pfaff et al., 2020 ) | 0.113 | 0.038 | 0.993 |
| GNO ( Li et al., 2020b ) | 0.129 | 0.031 | 0.991 |
| Galerkin ( Cao, 2021 ) | 0.118 | 0.069 | 0.879 |
| GEO-FNO ( Li et al., 2023b ) | 0.395 | 0.243 | 0.903 |
| Method | Standard Benchmarks | Darcy | AirCraft | |||||||
| Airfoil | Plasticity | NS2D | Elasticity | 85 | 141 | 211 | Surf. | |||
| MoNo ( Yang et al., 2026 ) | 0.0048 | 0.0006 | 0.0522 | 0.0033 | 0.0059 | 0.0049 | 0.0046 | 0.077 | 0.016 | 0.999 |
| PANO-P (ours) | 0.0044 | 0.0006 | 0.0720 | 0.0040 | 0.0055 | 0.0042 | 0.0041 | 0.063 | 0.016 | 1.000 (0.9995) |
| PANO-D (ours) | 0.0044 | 0.0004 | 0.0395 | 0.0029 | 0.0050 | 0.0039 | 0.0039 | 0.056 | 0.011 | 1.000 (0.9996) |
| Benchmark | Training Set | Test Set |
| Airfoil | 1000 | 200 |
| Plasticity | 900 | 80 |
| NS2D | 1000 | 200 |
| Elasticity | 1000 | 200 |
| Darcy | 1000 | 200 |
| AirfRANS (Standard Setting) | 720 | 200 |
| Split | OOD Reynolds: Reynolds Number Range | OOD Angles: AoA Range |
| Training Set | ||
| Test Set |
| Benchmark | Epochs | Peak LR | Batch size | Optimizer | Scheduler |
| Airfoil | 500 | 4 | AdamW | OneCycle | |
| Plasticity | 500 | 8 | AdamW | OneCycle | |
| NS2D | 500 | 2 | AdamW | OneCycle | |
| Elasticity | 500 | 4 | AdamW | OneCycle | |
| Darcy85 | 500 | 4 | AdamW | OneCycle | |
| Darcy141 | 500 | 4 | AdamW | OneCycle |
PDE Operator Learning
Partial differential equation (PDE) operator learning builds models that map inputs such as coefficients and initial conditions to solution fields.
See all 250 papers