TangentPDE: Learning Local Operator Responses with Sparse Finite-Difference Supervision
Organizations: Sichuan Agricultural University, China · Harbin Engineering University, China · University of Southampton, United Kingdom · Wenzhou-Kean University, China ·
Abstract
Neural PDE operators are often optimized through state reconstruction, which largely constrains operator values at observed inputs but does not fully characterize their sensitivity to local perturbations. A more faithful approximation of operator behavior requires not only accurate state predictions, but also correct variations along local perturbation directions. We present TangentPDE, which uses finite differences to approximate directional derivatives and constructs task-relevant finite-step responses from the state outputs of a reference solver at sparse paired inputs. Its objective jointly constrains state fidelity, response-direction alignment, and response-magnitude calibration, thereby learning both operator values and their local variations. Because the response labels are constructed solely from state outputs at paired inputs, the method requires neither explicit derivative labels, linearized or adjoint PDE solves, nor full Jacobians, and remains applicable to nonlinear solvers without derivative interfaces. Building on this, we additionally propose a selectively executed adaptation mechanism to strengthen local operator representations under finite-difference supervision. Budget-matched and correspondence-perturbed controls isolate the effect of relational supervision from improvements attributable to additional state observations or increased data volume. Designed as a plug-and-play training framework, TangentPDE integrates with diverse neural-operator backbones. Across four PDE systems and 20 task-backbone settings, TangentPDE lowers the three-seed mean state error in 18 settings and the corresponding Rollout@20 error in 18, while intrinsic finite-step response gains are strongest on DiffReact and remain task- and backbone-dependent elsewhere.
Figures & tables
| Task | Backbone | Aug-only | TangentPDE-Core | ||||
|---|---|---|---|---|---|---|---|
| State | Resp. cos. | Roll.@20 | State | Resp. cos. | Roll.@20 | ||
| DiffReact | FNO | 0.007529 | 0.4367 | 0.00413 | 0.005939 | 0.9358 | 0.00336 |
| CViT | 0.011589 | 0.1411 | 0.00690 | 0.009398 | 0.4721 | 0.00590 | |
| LaMO | 0.007906 | 0.2455 | 0.00344 | 0.007188 | 0.4855 | 0.00348 | |
| MambaNO | 0.012105 | 0.0722 | 0.00830 | 0.010433 | 0.9345 | 0.00740 | |
| RNO | 0.008452 | 0.5431 | 0.00580 | 0.006316 | 0.9630 | 0.00440 | |
| Task | Step | Pairs | |
|---|---|---|---|
| DiffReact | 200/50/100 | 100% | |
| Kolmogorov | 200/50/100 | 100% | |
| Burgers | 1638/204/206 | 16.38% | |
| Advection | 1638/204/206 | 16.38% |
| Metric | MSE-1 | MSE-2N | Two-point | Core |
|---|---|---|---|---|
| DiffReact | ||||
| State | 0.009452 | 0.009445 | 0.009223 | 0.005908 |
| Resp. cos. | 0.2935 | 0.2944 | 0.2960 | 0.9308 |
| Resp. rel. | 0.9773 | 0.9768 | 0.9733 | 0.3758 |
| Mag. err. | 1.104 | 1.102 | 1.050 | 0.0997 |
| Roll.@20 | 0.004679 | 0.004687 | 0.004440 | 0.003573 |
| Setting | State | Resp. cos. | Roll.@20 |
|---|---|---|---|
| Objective composition: DiffReact | |||
| State only | 0.008536 | 0.5360 | 0.0059 |
| State + Direct Resp. MSE | 0.008095 | 0.5953 | 0.0050 |
| State + Dir. | 0.007107 | 0.9640 | 0.0044 |
| State + Mag. | 0.008715 | 0.4311 | 0.0065 |
| Full Core | 0.006316 | 0.9630 | 0.0044 |
| Variant | State | Intr. resp. | Aux. resp. | Roll. |
|---|---|---|---|---|
| DiffReact | ||||
| Static | 0.0059 | 0.9433 | – | 0.0035 |
| State FT | 0.0032 | 0.9410 | – | 0.0018 |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 |
| LSTH | 0.0036 | 0.9431 | 0.9779 | 0.0019 |
| Kolmogorov | ||||
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
| Backbone | Input | Output | Blocks | Width | Backbone-specific setting | History | Prediction | Params. D/K (M) |
|---|---|---|---|---|---|---|---|---|
| FNO | 3 | 32 | Fourier modes | 1 | Parallel | 0.141/0.135 | ||
| CViT | 3 | 32 | Patch ; 4 heads | 1 | Parallel | 0.043/0.039 | ||
| LaMO | 3 | 32 | Latent width 32; stride 2 | 1 | Parallel | 0.078/0.075 | ||
| MambaNO | 3 | 32 | State size 16; convolution width 4 | 1 | Parallel | 0.014/0.011 | ||
| RNO | 3 | 32 | Three ConvGRU recurrent blocks | 1 | Recurrent | 0.176/0.176 |
| Backbone | Optimizer | LR | Batch | Epochs | Scheduler | Weight decay | Grad. clip | Precision | Checkpoint selection |
|---|---|---|---|---|---|---|---|---|---|
| FNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| CViT | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| LaMO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| MambaNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative | |
| RNO | Adam | 4 | 50 | None | 0 | None | FP32 | Lowest validation state relative |
| Task | Total | Valid | Excluded | Valid ratio | Min | Median | P95 |
|---|---|---|---|---|---|---|---|
| DiffReact | 100 | 100 | 0 | 1.0000 | 102.7 | 155.3 | 260.8 |
| Kolmogorov | 100 | 100 | 0 | 1.0000 | 9.059 | 37.39 | 150 |
| Burgers | 206 | 206 | 0 | 1.0000 | 1.253 | 3.041 | 7.492 |
| Advection | 206 | 206 | 0 | 1.0000 | 15.7 | 53.24 | 217.7 |
| Model | Method | State | Resp. | Roll. |
|---|---|---|---|---|
| CViT | Aug-only | 0.011589 | 0.1411 | 0.00690 |
| TangentPDE-Core | 0.009398 | 0.4721 | 0.00590 | |
| FNO | Aug-only | 0.007529 | 0.4367 | 0.00413 |
| TangentPDE-Core | 0.005939 | 0.9358 | 0.00336 | |
| LaMO | Aug-only | 0.007906 | 0.2455 | 0.00344 |
| TangentPDE-Core | 0.007188 | 0.4855 | 0.00348 |
| Model | Method | State | Resp. | Roll. |
|---|---|---|---|---|
| CViT | Aug-only | 0.414042 | 0.3075 | 0.43090 |
| TangentPDE-Core | 0.353682 | 0.3192 | 0.26900 | |
| FNO | Aug-only | 0.237184 | 0.7392 | 0.22747 |
| TangentPDE-Core | 0.237143 | 0.7798 | 0.22030 | |
| LaMO | Aug-only | 0.213051 | 0.4187 | 0.38450 |
| TangentPDE-Core | 0.200136 | 0.4000 | 0.36410 |
| Model | Method | State | Resp. | Roll. |
|---|---|---|---|---|
| CViT | Aug-only | 0.293270 | 0.2084 | 0.25000 |
| TangentPDE-Core | 0.265638 | 0.2191 | 0.24610 | |
| FNO | Aug-only | 0.156602 | 0.3901 | 0.15502 |
| TangentPDE-Core | 0.157755 | 0.4533 | 0.15108 | |
| LaMO | Aug-only | 0.287690 | 0.2064 | 0.25420 |
| TangentPDE-Core | 0.286906 | 0.2056 | 0.25270 |
| Model | Method | State | Resp. | Roll. |
|---|---|---|---|---|
| CViT | Aug-only | 0.546122 | 0.0471 | 0.06160 |
| TangentPDE-Core | 0.483750 | 0.0454 | 0.05960 | |
| FNO | Aug-only | 0.129336 | 0.0899 | 0.03090 |
| TangentPDE-Core | 0.135071 | 0.0881 | 0.06828 | |
| LaMO | Aug-only | 0.557364 | 0.0492 | 0.09400 |
| TangentPDE-Core | 0.555104 | 0.0475 | 0.07730 |
| Model | Method | State | Resp. | Supp. mag. | Roll. |
|---|---|---|---|---|---|
| CViT | Aug-only | 0.011589 | 0.1411 | 0.3249 0.0305 | 0.00690 |
| TangentPDE-Core | 0.009398 | 0.4721 | 0.3281 0.0111 | 0.00590 | |
| FNO | Aug-only | 0.007529 | 0.4367 | 0.8152 0.0646 | 0.00413 |
| TangentPDE-Core | 0.005939 | 0.9358 | 0.3167 0.3682 | 0.00336 | |
| LaMO | Aug-only | 0.007906 | 0.2455 | 1.0623 0.0634 | 0.00344 |
| TangentPDE-Core | 0.007188 | 0.4855 | 0.3807 0.6121 | 0.00348 |
| Model | Method | State | Resp. | Supp. mag. | Roll. |
|---|---|---|---|---|---|
| CViT | Aug-only | 0.414042 | 0.3075 | 1.7635 0.0715 | 0.43090 |
| TangentPDE-Core | 0.353682 | 0.3192 | 0.6632 0.3295 | 0.26900 | |
| FNO | Aug-only | 0.237184 | 0.7392 | 1.0413 0.0058 | 0.22747 |
| TangentPDE-Core | 0.237143 | 0.7798 | 1.0397 0.0043 | 0.22030 | |
| LaMO | Aug-only | 0.213051 | 0.4187 | 1.2073 0.0397 | 0.38450 |
| TangentPDE-Core | 0.200136 | 0.4000 | 1.2221 0.0345 | 0.36410 |
| Model | Method | State | Resp. | Supp. mag. | Roll. |
|---|---|---|---|---|---|
| CViT | Aug-only | 0.293270 | 0.2084 | 0.5475 0.0229 | 0.25000 |
| TangentPDE-Core | 0.265638 | 0.2191 | 0.3839 0.0098 | 0.24610 | |
| FNO | Aug-only | 0.156602 | 0.3901 | 0.4872 0.0172 | 0.15502 |
| TangentPDE-Core | 0.157755 | 0.4533 | 0.2572 0.0112 | 0.15108 | |
| LaMO | Aug-only | 0.287690 | 0.2064 | 0.8402 0.0204 | 0.25420 |
| TangentPDE-Core | 0.286906 | 0.2056 | 0.4180 0.0041 | 0.25270 |
| Model | Method | State | Resp. | Supp. mag. | Roll. |
|---|---|---|---|---|---|
| CViT | Aug-only | 0.546122 | 0.0471 | 2.4749 0.0164 | 0.06160 |
| TangentPDE-Core | 0.483750 | 0.0454 | 2.2433 0.0135 | 0.05960 | |
| FNO | Aug-only | 0.129336 | 0.0899 | 2.5620 0.0116 | 0.03090 |
| TangentPDE-Core | 0.135071 | 0.0881 | 2.5603 0.0095 | 0.06828 | |
| LaMO | Aug-only | 0.557364 | 0.0492 | 3.1964 0.0715 | 0.09400 |
| TangentPDE-Core | 0.555104 | 0.0475 | 1.5833 0.0871 | 0.07730 |
| Metric | Improved | Mean | Median | Bootstrap 95% CI | Raw | Holm | ||
|---|---|---|---|---|---|---|---|---|
| State | 18/20 | 0.017041 | 0.003352 | [0.007986, 0.026833] | 195.0 | 0.000131 | 0.000392 | 0.857 |
| Response cosine | 12/20 | 0.116285 | 0.011200 | [0.026504, 0.226892] | 160.0 | 0.020000 | 0.020000 | 0.524 |
| Rollout@20 | 18/20 | 0.020988 | 0.005555 | [0.006057, 0.039931] | 193.0 | 0.000197 | 0.000395 | 0.838 |
| Metric | MSE-1 | MSE-2N | Two-point | Core |
|---|---|---|---|---|
| DiffReact | ||||
| State | 0.009452 | 0.009445 | 0.009223 | 0.005908 |
| Resp. cos. | 0.2935 | 0.2944 | 0.2960 | 0.9308 |
| Resp. rel. | 0.9773 | 0.9768 | 0.9733 | 0.3758 |
| Mag. err. | 1.104 | 1.102 | 1.050 | 0.0997 |
| Roll.@20 | 0.004679 | 0.004687 | 0.004440 | 0.003573 |
| Setting | State | Resp. cos. | Roll.@20 |
|---|---|---|---|
| Objective composition: DiffReact | |||
| State only | 0.008536 | 0.5360 | 0.0059 |
| State + Direct Resp. MSE | 0.008095 | 0.5953 | 0.0050 |
| State + Dir. | 0.007107 | 0.9640 | 0.0044 |
| State + Mag. | 0.008715 | 0.4311 | 0.0065 |
| Full Core | 0.006316 | 0.9630 | 0.0044 |
| Task | Base params | Added params | Total params | Param. overhead | Intrinsic (ms/batch) | Auxiliary (ms/batch) | Latency overhead |
|---|---|---|---|---|---|---|---|
| DiffReact | 7,526 | 1,996 | 9,522 | 26.52% | 3.7778 | 4.2479 | 12.44% |
| Kolmogorov | 6,133 | 538 | 6,671 | 8.77% | 3.6005 | 4.1841 | 16.21% |
| Variant | State | Intr. resp. | Aux. resp. | Roll. |
|---|---|---|---|---|
| DiffReact | ||||
| Static | 0.0059 | 0.9433 | – | 0.0035 |
| State FT | 0.0032 | 0.9410 | – | 0.0018 |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 |
| LSTH | 0.0036 | 0.9431 | 0.9779 | 0.0019 |
| Kolmogorov | ||||
| Variant | State | Intr. resp. | Aux. resp. | Roll. | Supp. over-corr. | 95% CI |
|---|---|---|---|---|---|---|
| DiffReact | ||||||
| Static | 0.0059 | 0.9433 | – | 0.0035 | – | – |
| State FT | 0.0032 | 0.9410 | – | 0.0018 | – | – |
| Weak FT | 0.0037 | 0.9440 | – | 0.0021 | – | – |
| LSTH | 0.0036 | 0.9431 | 0.9779 | 0.0019 | 0.0000 0.0000 | [0.0000, 0.0000] |
| Kolmogorov | ||||||
| Task | Pairs | Overall | Calls | |
|---|---|---|---|---|
| DiffReact | 200/50/100 | 100% | 100% | 350 |
| Kolmogorov | 200/50/100 | 100% | 100% | 350 |
| Burgers | 1638/204/206 | 16.38% | 20.48% | 2,048 |
| Advection | 1638/204/206 | 16.38% | 20.48% | 2,048 |
| Task | Time (min) | 95% CI hw | Peak (GiB) | 95% CI hw | GPUs |
|---|---|---|---|---|---|
| DiffReact | 6.4 2.1 | [4.8, 8.0] | 0.18 0.00 | [0.18, 0.18] | 1 |
| Kolmogorov | 7.9 2.2 | [6.2, 9.6] | 0.09 0.00 | [0.09, 0.09] | 1 |
| Task | Source | Add. calls | Time (s) | Execution |
|---|---|---|---|---|
| Advection | 10000 | 10000 | 10.0 | 1 GPU |
| Burgers | 10000 | 10000 | 24.0 | 1 GPU |
| DiffReact | 350 | 350 | 25.1 | 4 CPU workers |
| Kolmogorov | 350 | 350 | 6.8 | 4 CPU workers |
| Task | Method | State | Resp. cos. | Rel. resp. | Mag. err. | Roll.@20 |
|---|---|---|---|---|---|---|
| DiffReact | Aug-only | 0.010514 0.000405 | 0.1667 0.0784 | 1.0227 0.0643 | 1.5896 0.6075 | 0.00580 0.00133 |
| DiffReact | Two-point MSE | 0.010411 0.000500 | 0.1707 0.0680 | 1.0208 0.0652 | 1.5491 0.5881 | 0.00564 0.00146 |
| DiffReact | TangentPDE-Core | 0.008211 0.000776 | 0.8834 0.0034 | 0.6393 0.0108 | 0.2504 0.0095 | 0.00491 0.00034 |
| Kolmogorov | Aug-only | 0.286750 0.003532 | 0.5059 0.0021 | 9.8993 0.0603 | 2.3386 0.0056 | 0.31424 0.01693 |
| Kolmogorov | Two-point MSE | 0.286668 0.003640 | 0.5059 0.0021 | 9.9005 0.0588 | 2.3387 0.0055 | 0.31396 0.01725 |
| Kolmogorov | TangentPDE-Core | 0.286205 0.005883 | 0.5419 0.0016 | 9.7912 0.0321 | 2.3318 0.0031 | 0.30824 0.02328 |
| Objective | Test step | Resp. cos. | Mag. err. |
|---|---|---|---|
| Full Core | 0.9537 0.0041 | 0.0220 0.0051 | |
| Full Core | 0.9630 0.0035 | 0.0146 0.0051 | |
| Full Core | 0.9665 0.0036 | 0.0191 0.0056 | |
| Direct Resp. MSE | 0.5768 0.0616 | 0.8388 0.0452 | |
| Direct Resp. MSE | 0.5953 0.0610 | 0.8716 0.0735 | |
| Direct Resp. MSE | 0.6034 0.0599 | 0.8858 0.0878 |
PDE Operator Learning
Partial differential equation (PDE) operator learning builds models that map inputs such as coefficients and initial conditions to solution fields.
See all 251 papers