SALTO: Solver-Anchored Lifted Transfer Operators for Nonlinear Event Inference
Authors: Qi Wang, Rylie Foster
Organizations: Department of Aerospace Engineering San Diego State University San Diego, CA 92182, USA · Computational Science Research Center San Diego State University San Diego, CA 92182, USA · Department of Mechanical and Aerospace Engineering University of California, Irvine, CA 92697, USA
Learned dynamical models can accurately predict trajectories while producing unreliable sensitivities when differentiated or transposed, limiting their use in inverse problems and scientific decision-making. Classical adjoints provide exact local derivatives, but a single adjoint cannot represent the nonlinear response of a system to a family of finite-amplitude perturbations. We introduce SALTO (Solver-Anchored Lifted Transfer Operators), a Koopman-based framework for inferring finite-amplitude sources. In the lifted space, the nonlinear dynamics are approximated by a linear propagator, and its transpose carries each sensor backward in a single solve. The result is a reusable lifted adjoint that predicts the measurement produced by any candidate localized event with one encoder pass and an inner product. To give the model the simulator's exact local behavior, SALTO imposes three structural constraints: it anchors the physical-state dynamics to the simulator's linearized propagator, centers the learned observables so that they vanish to second order at the reference state, and uses a block-triangular lifted propagator. These constraints guarantee that the model reduces to the simulator's tangent dynamics and classical adjoint as the amplitude vanishes, while the learned observables carry only the finite-amplitude correction. On three nonlinear PDE systems, SALTO predicts the measurements of finite-amplitude sources to within 2-8% where the classical adjoint is in error by 41-165%, recovers source position and intensity from sparse sensors nearly as accurately as inversion through the solver, and samples the joint posterior of two interacting sources, matching the solver's posterior on reaction-diffusion at a twentieth of its cost. One trained model serves new sensor layouts and measurement times with a single backward solve.
Figures & tables
Figure 1: SALTO in one picture. Localized sources S(p) seed a nonlinear flow observed by a few probes. Offline, the solver supplies Lk and the data supply only the finite-amplitude correction, so the lifted model is exact at small amplitude. Online, one backward solve per sensor gives the lifted adjoint Zj† , and every candidate source is scored by an encoder pass and inner products.
accuracy
cost
case
model
εm (%)
δx (cells)
δI (%)
seconds
Burgers self-advection
linear
72.9
6.00
15.2
0.003
quadratic
96.1
3.00
8.9
4.1
lifted (ours)
1.8
1.00
3.6
9.6
solver
–
1.00
3.4
167
reaction–diffusion saturation
linear
164.5
0.63
40.2
0.005
Table 1: Accuracy and cost on the three systems, with the c=24 lift on Burgers and the c=4 lifts on the two-dimensional systems. Each block uses one common draw of test sources. Bold marks the best of the three models, the solver serves as the reference, and timings are wall-clock times.
Figure 2: (a–d) Space–time evolution of the base flow and of one release, from the solver, the lifted model and the linearized solver. The dashed line is the half-maximum contour of the ground truth. (e) Five representative blocks of the lift, forward in time (top) and adjoint in backward time τ (bottom). (f) Median inversion error for 800 sources at 1% noise, with δx in black and δI in red, and linear dotted, quadratic dashed and lifted solid. The dotted horizontal line marks one grid cell.
Figure 3: Two-dimensional reaction–diffusion, with the eight-channel equivariant lift. The lifted posterior reproduces the solver’s, while the classical ones spread over the whole window. (a–d) Joint posterior density of source position (Eq. 4 ) for a pair 1.4 source widths apart, on a logarithmic scale. Each model is evaluated on the combined field, and its likelihood is widened by its own discrepancy. Crosses mark the true sources and the dot marks a probe. (e) Error of the one-shot inversion against the true intensity, with δx in black and δI in red, and linear dotted, quadratic dashed and lifted solid. Shading marks where each classical model still predicts the measurements to within three per cent, and the narrow unlabelled band at the left is where the linear embedding does.
Figure 4: Kolmogorov flow. (a–d) Perturbation vorticity at ±35% of its initial maximum, time upward to 2T , for the most strongly interacting pair, from the solver, the lifted rollout, the quadratic expansion and the linear superposition. (e) The state and one learned channel along that pair’s solver trajectory (top), and the same blocks of the centre probe’s lifted adjoint Z† (circle) in backward time (bottom), each on its own scale. The state block is exactly qL† . The channel block is spread out because a measurement uses only its projection on the channels. (f) Inversion error over 600 further sources, as in Fig. 3 e on linear axes; the narrow band is where the linear embedding holds.
reaction–diffusion
Kolmogorov flow
εm (%)
εm
gradient (%)
εm (%)
εm
gradient (%)
model
top band
Is=10−3
weak
train
top band
Is=10−3
weak
train
lifted (ours)
6.0
8×10−5
0.7
21
13
9×10−5
0.6
17
unconstrained
3.2
0.23
4.9
7.8
14
0.67
16
25
centering only
4.8
3×10−2
2.9
17
14
6×10−2
10
20
anchoring only
9.2
1.9
11
23
22
8.2
17
27
Table 2: What the constraints buy (two-channel lifts). Only the anchored, centered lift is exact as the amplitude vanishes, in the measurement at Is=10−3 (a ratio) and in the gradient for Is∈[0.01,0.1) , and on Kolmogorov flow it is the most accurate learned model throughout. The neural surrogate did not reach the lift’s training loss on reaction–diffusion. Bold marks the best per column.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
εm (%), training range
εm at I=10−3
inference δI (%)
problem
ours
unconstr.
linear
ours
unconstr.
linear
ours
brute force
linear
Burgers, base flow
0.9
–
40.7
1.0×10−4
–
1.1×10−3
5.8
5.8
20.1
reaction–diffusion
3.0
1.9
76.7
8.4×10−5
2.3×10−1
6.1×10−4
12.6
10.7
35.9
Kolmogorov flow
5.7
9.7
18.4
9.0×10−5
6.7×10−1
2.3×10−4
3.8
2.7
8.3
Appendix
Table 3: The constrained lift is the most accurate model at small amplitude on all three systems, and it infers the intensity nearly as well as brute force. The columns give the measurement error εm over the training range and at I=10−3 for our lift, the unconstrained lift and the linear embedding, followed by the median intensity error of the inference. The two-dimensional rows use the two-channel lifts of Tables 5 and 6 on their own draw, so they differ from the four-channel values in Table 1 . The unconstrained lift was not trained on Burgers. Bold marks the best value in each group, excluding brute force.
Is=0.1
Is=0.3
Is=1
forward error at T , linearized solver
10.9%
31.7%
79.4%
forward error at T , lifted model
0.5%
0.5%
1.6%
Appendix
Table 4: On Burgers the lifted model matches brute force in every band, while the errors of the linear embedding grow with the intensity. Top, relative error at T of the forward models. Bottom, median position error (in length units, with grid cell 0.025 ) and relative intensity error of the one-shot inversion in each intensity band (60 sources, 1% noise), on a draw separate from the 800 sources of Table 1 .
Figure 5: Burgers about the base flow 1+sin3x . The lifted model follows the solver where the linearized solver does not. Its forward error falls steadily as the number of learned channels c grows, whereas its inversion error stops improving at four channels. (a–c) The perturbation released at xs=3 with the three test intensities, at t=T/3 (light grey) and at T (black). The solver is the broad grey band, the lifted model the solid line and the linearized solver the dotted line. (d) Relative error over time at the three test intensities for the lifted model (solid) and the linearized solver (dotted), labelled by Is . (e) Mean validation error over time for the linearized solver and for the lift before and after training, on the same vertical axis as (d). (f) Forward error at T against c for four test intensities, labelled by Is , together with the mean validation error (open circles). (g) Median inversion error in position δx (black) and intensity δI (red) against c . The dotted line marked “linear” is the linear embedding, whose two errors differ by less than one per cent and are drawn as one line, and the fine dotted line marks one grid cell.
εm (%) per intensity band Is
εm (ratio) at
inference
model
[0.1,0.5)
[0.5,1)
[1,2)
[2,3)
Is=10−3
δx
δI (%)
lifted (ours)
1.2
1.7
6.0
13.9
8.4×10−5
0.145
12.6
unconstrained lift
1.3
1.1
3.2
6.9
2.3×10−1
0.143
11.3
centering only
0.9
1.3
4.8
12.1
2.8×10−2
0.155
15.7
anchoring only
3.4
4.2
9.2
28.8
1.9×100
0.181
14.3
linear adjoint
23.3
59.8
146.9
165.2
6.1×10−4
0.147
35.9
Appendix
Table 5: On reaction–diffusion only the full lift reduces to the linear adjoint at small amplitude, while the unconstrained lift fits the strongest bands more closely. The columns give the measurement error in each intensity band, with the last band beyond the training amplitudes, and at Is=10−3 , followed by the median position error in length units (candidate lattice 0.26 ) and the relative intensity error, over 60 sources with 1% noise and two channels. The neural surrogate did not converge on this problem and is reported as trained. Bold marks the best value in each column, excluding brute force.
εm (%) per intensity band Is
εm (ratio) at
inference
model
[0.1,0.5)
[0.5,1)
[1,4)
[4,6)
Is=10−3
δx
δI (%)
lifted (ours)
1.6
3.0
12.6
20.7
9.0×10−5
0.111
3.8
unconstrained lift
9.6
5.6
14.0
26.3
6.7×10−1
0.114
4.7
centering only
4.1
3.5
13.5
22.6
5.8×10−2
0.106
4.2
anchoring only
8.1
8.3
21.9
39.6
8.2×100
0.133
5.0
linear adjoint
4.3
11.3
39.6
65.9
2.3×10−4
0.204
8.3
Appendix
Table 6: On Kolmogorov flow the full lift is the most accurate learned model in every band and at small amplitude. Columns as in Table 5 .
Figure 6: On reaction–diffusion the measurement error of the full lift falls in proportion to the source intensity, whereas that of the unconstrained lift levels off at small amplitude. (a) Measurement error in each intensity band for the six models. (b) Measurement error against the source intensity. (c) The measurement of one probe over twice the training horizon for a source of intensity 1.5 , from the solver, the lifted functional, the linear embedding and the unconstrained lift.
Figure 7: On Kolmogorov flow the full lift is the most accurate model in every intensity band, and its measurement error falls with the source intensity. Panels as in Fig. 6 , with an injection of intensity 3 in (c) .
val.
εm (%)
εm (ratio) at
inference
c
dΦ
iters
MSE
top band
beyond
Is=10−3
δx
δI (%)
reaction–diffusion
1
2,304
300
1.7×10−3
11.4
9.0
9.3×10−5
0.192
13.7
2
4,608
300
7.8×10−4
6.0
13.9
8.4×10−5
0.145
12.6
4
9,216
300
7.2×10−5
1.6
4.0
8.3×10−6
0.113
8.7
Kolmogorov flow
Appendix
Table 7: With four channels the anchored, centered lift is the most accurate of the three sizes on both two-dimensional systems. The columns give the feature dimension dΦ=cN , the number of training iterations, the best validation rollout MSE, the measurement error in the top training band and beyond it, the error at Is=10−3 , and the median inference errors. Every row uses the same schedule and training pairs, although the cost of the fit grows with c . We vary c only for the anchored lift, because the full-propagator fits of the other variants exceed the GPU memory at c=4 . Bold marks the best value in each column.
reaction–diffusion, Is band
Kolmogorov flow, Is band
model
[0.01,0.1)
[0.1,0.5)
[0.5,1)
[1,2)
[2,3)
[0.01,0.1)
[0.1,0.5)
[0.5,1)
[1,4)
[4,6)
lifted (ours)
0.7
3.3
7.5
21
73
0.6
2.6
7.9
17
35
unconstrained lift
4.9
2.9
4.7
7.8
24
16
13
18
25
37
centering only
2.9
3.1
5.3
17
60
10
8.1
11
20
29
anchoring only
11
7.0
12
23
114
17
19
19
27
54
neural surrogate
66
61
66
94
170
18
17
17
37
68
Appendix
Table 8: For the weakest sources the constrained lift gives the most accurate measurement gradient among the learned models. We difference every model and the solver in the same way, centrally, with steps of 10−3 in position and 10−3 (relative) in intensity, and reducing the steps threefold changes no entry. Each entry is the median, over twelve sources in the band, of ∥J−Jsolver∥F/∥Jsolver∥F in per cent, where J is the Jacobian of the nine probe measurements with respect to (xs,ys,Is) , made dimensionless by the source width and intensity. The models are those of Tables 5 and 6 , and the last band of each system lies beyond the training range. Bold marks the most accurate model in each band.
Figure 8: The lifted functional reproduces the finite-amplitude sensitivity map of the centre probe at the cost of one inner product per source position, whereas the linear embedding misses both its amplitude and its shape. Reaction–diffusion is shown in (a–e) and Kolmogorov flow in (f–j) , at Is=0.75Imax with the four-channel lifts. Each row shows the exact map, which costs one nonlinear solve per source position, the lifted functional and the linear embedding, followed by the difference of each approximation from the exact map. Every panel in a row is divided by the largest value of that row’s exact map, and all ten panels share one colour scale, so the differences read directly as fractions of the exact map. Values beyond ±1 saturate, and the crosses mark the probes.
case
model
δx (cells)
δI (%)
Burgers
linear
6.00 [5.00, 6.00]
15.2 [13.8, 16.5]
quadratic
3.00 [2.00, 3.00]
8.9 [8.2, 9.7]
lifted
1.00 [1.00, 1.00]
3.6 [3.3, 3.9]
solver
1.00 [1.00, 1.00]
3.4 [3.1, 3.7]
reaction–diffusion
linear
0.63 [0.51, 0.84]
40.2 [34.7, 45.5]
quadratic
0.91 [0.60, 1.21]
37.1 [23.2, 44.6]
Appendix
Table 9: The lifted inference is statistically indistinguishable from the solver in position on all three systems. Medians of Table 1 with 95% bootstrap intervals over the test sources ( 10,000 resamples).
Figure 9: The misfit landscape of the lifted functional has its minimum at the true source, as for brute force through the solver, whereas the minimum of the linear embedding is displaced on Kolmogorov flow. The panels show single-source inference with the four-channel lifts, for reaction–diffusion in (a–d) and Kolmogorov flow in (e–h) . The first three panels of each row show the misfit over the candidate lattice for the lifted functional, the linear embedding and brute force through the solver. At each position the misfit is minimized over the intensity, normalized by its own minimum and drawn on one logarithmic scale, on which red indicates a good fit. The cross marks the true source and the circle the estimate. Panels (d) and (h) show the position error of the 60 test sources against their intensity, with the candidate-lattice spacing dotted.
inter-
lifted functional
linear embedding
brute force
speed-up
problem
pairs
action (%)
δx
δI (%)
both (%)
δx
δI (%)
both (%)
δx
δI (%)
both (%)
brute/lifted
reaction–diffusion
separated
0.3
0.009
0.7
100.0
0.092
33.9
96.7
0.008
0.6
100.0
18
separated, strong
0.9
0.010
1.3
100.0
0.116
42.2
96.7
0.007
0.7
100.0
24
overlapping
16.1
0.035
9.0
83.3
0.145
37.1
63.3
0.027
6.4
93.3
24
overlapping, strong
22.6
0.045
9.4
80.0
0.208
49.4
33.3
0.024
5.8
100.0
26
Kolmogorov flow
separated
47.9
0.168
6.3
79.2
0.343
30.0
20.8
0.007
0.4
100.0
240
Appendix
Table 10: The lifted functional resolves two sources more often and more accurately than the linear embedding in every pair family. Interaction is the median departure of the exact response from the sum of the single-source responses. For each model the table gives the median per-source position error δx (lattice spacing 0.26 ), the median intensity error δI and the fraction of pairs with both sources within one lattice cell, followed by the ratio of brute-force to lifted inference time. Bold marks the better of the lifted functional and the linear embedding.
Figure 10: The lifted functional recovers both sources of a separated pair on reaction–diffusion, together with their intensities, as accurately as brute force, whereas the linear embedding underestimates every intensity. The pairs carry 0.5 – 1.0Imax per source, up to twice the training budget in total, and the frame is shifted to centre the pair shown. (a–d) The forward problem. Panel (a) shows the initial field with the 4×4 probes as crosses, (b) the exact field at T , and (c) the superposition of the linearized responses on the same colour scale as (b). Panel (d) shows the sixteen measurements, from the solver as bars and with 1% noise as dashes, together with the predictions of the lifted functional and of the linear embedding at the true parameters. (e–h) The inverse problem. Panels (e) and (f) show the misfit of the second source over the domain at its fitted intensity, given the fitted first source, for the lifted functional and for the linear embedding, with crosses at the true sources and circles at the estimates. Panel (g) shows the estimated against the true intensity of every source over all pairs, and (h) the cumulative distribution of the per-source position error, with the candidate-lattice spacing dotted.
Figure 11: Beyond the training horizon, the lifted rollout reproduces the decay of the strong separated pair of Fig. 10 , whereas the linear superposition stays twice as strong. The panels show isosurfaces of u(x,y,t) at 25% of the initial maximum, with time running vertically to 2T , the translucent plane at the training horizon T and the probes marked by crosses, for (a) the solver, (b) the lifted rollout and (c) the superposition of the linearized responses. Over the whole space–time field the lifted rollout is off by 3.5% and the linear superposition by 81% .
reaction–diffusion
Kolmogorov flow
model
3×3 , T
4×4 , T
25 rand., T
3×3 , T/2
pairs
3×3 , T
4×4 , T
25 rand., T
3×3 , T/2
pairs
top training band, Is∈[1,Imax)
lifted (ours)
1.5
1.6
1.5
1.4
0.8
6.0
11
5.8
4.4
5.2
direct readout
2.3
2.2
2.0
1.5
1.3
4.6
9.4
4.7
3.1
4.6
direct CNN
9.2
11
–
–
5.1
17
25
–
–
8.5
parameter MLP
5.8
5.5
5.3
7.8
–
4.6
15
4.9
5.0
–
Appendix
Table 11: One lift serves new sensor layouts and horizons as accurately as models refitted for each of them. The table gives the relative measurement error (%) of the four-channel lift and of models trained directly on the same simulations, for single sources and for overlapping pairs. The direct readout and the direct CNN both keep the exact linear term ⟨qL†,S⟩ and add a linear readout of centered features, so they are anchored and centered like the lift. The readout is a ridge regression on the lift’s own features, fitted to the stored training fields at each layout and horizon. The direct CNN, which has the encoder’s architecture but is trained on measurements alone, and the parameter MLP must be retrained for each layout.
reaction–diffusion
Kolmogorov flow
T
1.5T
T
1.5T
band
1
2
3
4
1
2
3
4
1
2
3
4
1
2
3
4
lifted, 400 traj.
0.2
0.3
1.5
4.2
0.3
0.8
4.7
13
1.2
1.6
6.0
12
1.4
2.2
6.6
17
readout, 400 traj.
0.5
0.7
2.3
5.2
–
–
–
–
1.1
2.1
4.6
10
–
–
–
–
anchored CNN, 400 traj.
3.4
3.3
12
67
–
–
–
–
2.3
4.0
9.1
14
–
–
–
–
lifted, 50 traj.
3.9
8.8
32
84
6.2
15
56
140
2.8
5.5
17
38
3.1
6.5
20
41
Appendix
Table 12: The lift predicts beyond its training horizon, where the linear embedding fails. The table gives the relative measurement error (%) on the 3×3 probes in the four intensity bands, at the training horizon T and at 1.5T . It compares the four-channel lift trained on 400 and on 50 trajectories, a ridge readout fitted on each lift’s own features and trajectories, an anchored and centered CNN of the lift’s architecture trained end to end on the same trajectories, and the linear embedding. The readout and the CNN have no training data beyond T .
Neural operators provide fast surrogates for partial differential equation (PDE) solvers, but their reliability can degrade for high-dimensional spatial inputs and inverse or repeated inference. State-only training constrains solution values but not the learned input--output response. We study sensitivity-constrained neural operators (SC-NOs), which augment standard training with sampled solver-derived Jacobian supervision. Selected sensitivities from differentiable solvers or discrete adjoints are matched during training, allowing response information to be amortized across minibatches without imposing the full Jacobian at every update. We evaluate SC-NO on advection--diffusion and RANS--Spalart--Allmaras benchmarks, input-dimensionality scaling tests, long-horizon autoregressive rollout, and a shallow-water Tohoku tsunami source-inversion case. Sensitivity supervision improves forward prediction and yields larger gains in gradient-based inverse reconstruction of distributed fields. Scaling experiments show an improved accuracy--cost tradeoff for high-dimensional gridded inputs, while ablations indicate that state values and Jacobian information provide complementary supervision. In the tsunami case, SC-FNO reconstructs gridded seafloor deformation from sparse early gauge observations and forecasts subsequent wave propagation in a near-real-time proof-of-concept workflow. These results support sampled sensitivity supervision as a practical way to improve neural PDE surrogates when forward accuracy, inverse stability, robustness, and computational cost must be considered together.
Abdolmehdi Behroozi, Chaopeng Shen, Daniel Kifer +1
Department of Civil and Environmental Engineering, Penn State University, University Park, PA 16802, USA · School of Electrical Engineering and Computer Science, Penn State University, University Park, PA 16802, USA
Diffusion and flow generative models sample by integrating a learned ODE, but high quality still requires many sequential model evaluations. Solver learning reduces this cost by adapting scalar coefficients, timesteps, or both, while keeping the backbone model fixed. In this work, we identify a structural bottleneck in this update family: each step remains span-limited. Since the scalar-coefficient update lies in the span of buffered velocity evaluations, it can fit only the in-span component while leaving any out-of-span residual unreachable by scalar recombination alone. We propose SpanLift, a lightweight neural solver that augments scalar-coefficient updates with a spatial residual operator. SpanLift keeps a fixed base solver as an in-span prior and learns a spatial residual operator over the state and velocity buffer. The operator is trained by endpoint teacher matching, preserves the pretrained backbone, and adds no model NFEs. Empirically, the learned correction transfers across base solvers and is predominantly out-of-span. Across pixel-space diffusion, latent flow matching, and precipitation nowcasting, SpanLift achieves state-of-the-art few-step sampling. With only 3 NFE, it improves CIFAR-10 FID from 8.16 to 5.69 and ImageNet FID from 17.37 to 11.83.
Sihyeon Kim, Seunghun Lee, Vikas Singh +1
Korea University · KAIST · University of Wisconsin–Madison
High-fidelity simulations of complex physical systems are often too expensive for repeated prediction, design, and control. Reduced-order models address this computational cost by compressing high-dimensional dynamics into low-dimensional representations that can be evaluated rapidly, but they often lose accuracy when online dynamics drift beyond the offline training data. Adaptive methods address this limitation by updating the reduced subspace online using external, out-of-span information, such as full-order corrections or sensor snapshots. We discovered that a complementary and previously unexploited in-span adaptation channel exists within the current reduced subspace. To exploit this channel, we continually update the reduced representation using the model's own predictions through an incremental singular-value decomposition with a forgetting factor. This produces a trajectory-informed spectral preconditioner in which the reduced subspace remains unchanged, while the basis is reweighted and realigned according to the directions visited by the evolving dynamics. This internal reorganization prepares the basis to absorb future out-of-span corrections more effectively, improving adaptation without requiring additional external information. We expose the mechanism through a three-dimensional spiral example and demonstrate its benefits on nonlinear partial differential equations, including viscous Burgers and Fisher--KPP dynamics. We also discuss how in-span learning can be interpreted as a dynamical-systems analogue of in-context learning. More broadly, in-span learning suggests a new principle for computational science, revealing that model-generated trajectories contain more usable information than previously recognized.
Amirpasha Hedayat, Laura Balzano, Karthik Duraisamy
Department of Aerospace Engineering · Michigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, MI, USA · Department of Electrical Engineering and Computer Science