PDE-State Conditioning for Learned Optimization Using Physics-Informed Neural Networks
Organizations: Department of Computer Science, University of Houston, Houston, TX, USA · Center for Science, University of Austin, Austin, TX, USA
Abstract
Physics-informed neural network (PINN) optimization depends on global properties of the underlying PDE and how the physics residual evolves across the domain. Standard optimizers such as Adam use fixed gradient-based statistics, while learned PINN optimizers replace the fixed rule with a learned update built from local parameters and optimization-history features. However, none explicitly use PDE-level information. We study two coupled design questions: what PDE-level information should a learned optimizer observe, and how should that information influence its update? We introduce an 11-dimensional PDE state-capturing operator structure, residual geometry, and short-horizon update response, and use it to condition a structured meta-learned optimizer through bounded feature-wise modulation. Repeated design studies identify the compact residual-geometry state G2 and bounded FiLM as the strongest overall choices among those tested. The resulting PDE-State-conditioned optimizer improves all evaluation metrics (relative-L2, L1, solution MSE, and physics MSE) over Adam and the unconditioned learned optimizer on Linear Advection, KdV, and Burgers.
Figures & tables
| Group | Dim. | Role in optimizer state |
|---|---|---|
| G1 | 7 | Operator structure: time dependence, derivative orders, nonlinearity, advection, diffusion, dispersion. |
| G2 | 2 | Residual geometry: concentration of squared PDE residual energy and local residual roughness. |
| G3 | 2 | Update response: PDE and IC residual response to the previous learned black-box displacement. |
| Method | PDE | Rel. | Solution MSE | Physics MSE | |
|---|---|---|---|---|---|
| Adam | LA | .5200 .0224 | .3007 .0130 | .14088 .00840 | .04156 .00182 |
| KdV | .4648 .0147 | .2211 .0057 | .10952 .00685 | .03523 .00169 | |
| Burgers | .1286 .0017 | .0425 .0019 | .00657 .00023 | .01102 .00103 | |
| Unconditioned LO | LA | .4249 .1808 | .2468 .1058 | .11532 .07294 | .03451 .01669 |
| KdV | .4392 .1151 | .2110 .0475 | .10337 .05361 | .03378 .01347 | |
| Burgers | .1570 .1280 | .0582 .0493 | .01451 .02044 | .02311 .02727 |
| PDE state | Conditioning | LA | KdV | Burgers |
|---|---|---|---|---|
| All11 | PlainFiLM | .3445 .0320 | ||
| ContrastOnly | .2933 .1569 | .3957 .0565 | .2120 .1485 | |
| RawScale96 | .2281 .2294 | .1746 .1134 | ||
| G2 | PlainFiLM | |||
| ContrastOnly | .2890 .1659 | .4083 .0561 | .2079 .0930 | |
| RawScale96 | .2962 .1548 | .3829 .0451 | .1256 .0087 |
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Indices | Group | Coordinates |
|---|---|---|
| 0–6 | G1 | time dependence; maximum temporal derivative order; maximum spatial derivative order; nonlinearity; advection; diffusion; dispersion |
| 7–8 | G2 | (residual concentration), (residual roughness) |
| 9–10 | G3 | , (response to the previous black-box correction) |
| Equation | Time | Nonlinear | Adv. | Diff. | Disp. | ||
|---|---|---|---|---|---|---|---|
| Linear Advection | 1 | 1 | 1 | 0 | 1 | 0 | 0 |
| Burgers | 1 | 1 | 2 | 1 | 1 | 1 | 0 |
| KdV | 1 | 1 | 3 | 1 | 1 | 0 | 1 |
| Meta seed | Active state | LA | KdV | Burgers |
|---|---|---|---|---|
| 1234 | G1 | .0962 .0426 | .3176 .0496 | .1601 .1039 |
| G2 | .1188 .0594 | |||
| G3 | .1143 .0641 | .3013 .0218 | .1529 .0575 | |
| G1+G2 | .1216 .0610 | .3080 .0290 | .1151 .0454 | |
| G1+G3 | .1185 .0453 | .3072 .0298 | .1035 .0431 | |
| G2+G3 | .3309 .0563 | .1065 .0809 |
| State | PDE | Rel. | L1 | Solution MSE | Physics MSE |
|---|---|---|---|---|---|
| G2 | LA | ||||
| KdV | .0211 .0017 | ||||
| Burgers | .1192 .0277 | ||||
| G1+G3 | LA | .1551 .0646 | .0893 .0377 | .0169 .0132 | .0105 .0042 |
| KdV | .3319 .0419 | .1647 .0164 | .0575 .0148 | ||
| Burgers | .1178 .0300 | .0331 .0049 | .0066 .0041 | .0095 .0037 |
| Meta seed | Variant | PDE | Rel. | L1 | Solution MSE | Physics MSE |
| 1234 | G2 | LA | ||||
| KdV | ||||||
| Burgers | ||||||
| 1234 | G2-ext | LA | .2422 .1471 | .1399 .0847 | .0380 .0355 | .0177 .0119 |
| KdV | .3375 .0423 | .1685 .0191 | .0579 .0135 | .0217 .0031 | ||
| Burgers | .1317 .0954 | .0325 .0143 | .0093 .0137 | .0108 .0090 |
| Meta seed | Method | PDE | Rel. | Solution MSE | Physics MSE | |
|---|---|---|---|---|---|---|
| 1234 | Adam | LA | .5451 | .3155 | .14959 | .04365 |
| KdV | .4485 | .2147 | .10204 | .03357 | ||
| Burgers | .1267 | .0410 | .00630 | .01025 | ||
| Unconditioned LO | LA | .5969 | .3480 | .18567 | .05058 | |
| KdV | .5677 | .2642 | .16417 | .04930 | ||
| Burgers | .3045 | .1150 | .03811 | .05450 |
| Conditioning | PDE | Rel. | Solution MSE | Physics MSE | |
|---|---|---|---|---|---|
| PlainFiLM | LA | ||||
| KdV | |||||
| Burgers | |||||
| ContrastOnly | LA | .2890 .1659 | .1663 .0966 | .05596 .05199 | .02192 .01305 |
| KdV | .4083 .0561 | .1976 .0228 | .08889 .02495 | .03164 .01035 | |
| Burgers | .2079 .0930 | .0650 .0438 | .02350 .02372 | .02652 .02899 |
| Conditioning | PDE | Rel. | L1 | Solution MSE | Physics MSE |
|---|---|---|---|---|---|
| PlainFiLM | LA | ||||
| KdV | |||||
| Burgers | |||||
| ContrastOnly | LA | ||||
| KdV | |||||
| Burgers |
| State | Method | LA | KdV | Burgers |
|---|---|---|---|---|
| G1 | PlainFiLM | |||
| NoContext | .1622 .0572 | .5864 .2750 | .3224 .1020 | |
| G2 | PlainFiLM | .1188 .0594 | ||
| NoContext | .6041 .1874 | .6442 .2613 | ||
| G3 | PlainFiLM | |||
| NoContext | .3479 .2817 | .7903 .5344 | .6241 .2992 |
| Method | PDE | Rel. | L1 | Solution MSE | Physics MSE |
|---|---|---|---|---|---|
| PlainFiLM | LA | .0969 .0614 | .0542 .0363 | .0062 .0073 | .0078 .0048 |
| KdV | |||||
| Burgers | .0871 .0149 | .0279 .0050 | .0029 .0009 | .0122 .0094 | |
| Gamma-only | LA | .1691 .1689 | .0971 .0972 | .0257 .0373 | .0114 .0130 |
| KdV | .3414 .0659 | .1692 .0264 | .0603 .0236 | .0217 .0059 | |
| Burgers | .0987 .0521 | .0264 .0051 | .0045 .0049 |
| Method | LA | KdV | Burgers |
|---|---|---|---|
| RawScale32 | .1171 .0621 | ||
| ContrastOnly32 | .1188 .0807 | .3742 .0421 |
| Meta seed | Method | LA | KdV | Burgers |
|---|---|---|---|---|
| 1234 | PlainFiLM-All11 | .1698 .1517 | .3526 .0537 | .1061 .0543 |
| Groupwise FiLM | ||||
| 2024 | PlainFiLM-All11 | .1557 .0802 | ||
| Groupwise FiLM | .5705 .0575 | .4807 .0383 | .1055 .0098 | |
| Groupwise FiLM (same norm) | .5623 .0732 | .4835 .0388 |
| Meta seed | Method | LA | KdV | Burgers |
|---|---|---|---|---|
| 1234 | PlainFiLM | .0411 | .3459 | .0971 |
| ContrastOnly | .2547 | .3604 | .0988 | |
| RawScale96 | ||||
| 2024 | PlainFiLM | |||
| ContrastOnly | .1592 | .3657 | .3801 | |
| RawScale96 | .1784 | .3422 | .3014 |
Learning to Optimize
Learning to Optimize uses data to adapt algorithms, solver decisions, and solution maps for faster or better optimization.
See all 152 papers