cs.LGOct 6, 2026

Learning PDE solution operators with variable initial conditions via Latent Dynamics Networks

Authors: Stefano Maria Pizzamiglio, Stefano Pagani, Francesco Regazzoni

Organizations: MOX Laboratory, Department of Mathematics, Politecnico di Milano, Italy

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In many-query scenarios, data-driven surrogate models provide an efficient alternative to high-fidelity solvers for simulating physical systems governed by Partial Differential Equations (PDEs). In this context, the Latent Dynamics Network (LDNet) has recently demonstrated remarkable performance in predicting the response of spatio-temporal systems, combining Neural Ordinary Differential Equations with nonlinear dimensionality reduction. However, the original formulation assumes a fixed initial condition, limiting its applicability to many real-world applications where a system evolves from varying starting states. In this work, we overcome this limitation while keeping the end-to-end training procedure of the original LDNet and its encoder-free nature, which preserves its intrinsic independence from spatial resolution and grid topology. We infer the initial latent state directly from a small set of early-time observations, treating latent-state initialization as an adaptation problem, and investigate two strategies: an auto-decoding formulation and a meta-learning approach in which the initial latent state acts as a task-specific context variable. We demonstrate the accuracy of the proposed methods across diverse physical phenomena, spanning advection-diffusion, fluid dynamics, and solid mechanics. Meta-learning markedly accelerates latent-state inference and induces smoother, better-conditioned optimization landscapes, and spontaneously organizes the latent space into a structured representation that reflects physically meaningful features of the underlying dynamics. The coordinate-based decoder enables training from spatially subsampled data while recovering high-resolution solution fields at inference. The resulting approach provides an efficient and resolution-independent surrogate modeling framework for many-query simulations of time-dependent PDEs with varying initial conditions.

Figures & tables

Test caseArchitectureNRMSE (Test set)
ADRAD-IC-LDNet3.5×10−33.5\times 10^{-3}
Meta-IC-LDNet2.9×10−32.9\times 10^{-3}
  ppuuvv
Static cylinderAD-IC-LDNet4.1×10−34.1\times 10^{-3}4.7×10−34.7\times 10^{-3}7.1×10−37.1\times 10^{-3}
Meta-IC-LDNet2.6×10−32.6\times 10^{-3}2.7×10−32.7\times 10^{-3}4.1×10−34.1\times 10^{-3}
  ppuuvv
Table 1: Comparison of per-component test-set range-Normalized Root Mean Square Error (NRMSE) across the four test cases.
Test caseArchitectureHardware (Training budget)Inference mini-batch sizeInference time a
ADRAD-IC-LDNetHPC cluster ( << 24 h) b1001.14 s/ex.
Meta-IC-LDNetHPC cluster (24 h)1000.04 s/ex.
Static cylinderAD-IC-LDNetHPC cluster (24 h)621.11 s/ex.
Meta-IC-LDNetHPC cluster (24 h)60.23 s/ex.
Rotating cylinderMeta-IC-LDNetHPC cluster (24 h) c20.87 s/ex.
Nonlinear beamMeta-IC-LDNetWorkstation (Convergence) d80.15 s/ex.
Table 2: Training hardware and inference speed for different test cases and architectures.
Figure 1 : ADR test case: inference on the test set (AD-IC-LDNet). Predicted solution, absolute error, and true solution in the space-time domain for the test-set samples with the lowest (a) and highest (b) NRMSE.
Figure 2 : ADR test case: initial latent states (AD-IC-LDNet). Test-set initial codes, colored by the amplitude (left) and the phase (right) of the corresponding initial conditions.
Figure 3 : ADR test case: latent trajectories (AD-IC-LDNet). Test-set latent trajectories, translated to start from the origin, colored by μ1π2+μ3\mu_{1}\pi^{2}+\mu_{3} , which combines diffusion and reaction (left), and by the advection coefficient μ2\mu_{2} (right).
Figure 4 : ADR test case: inference on the test set (Meta-IC-LDNet). Predicted solution, absolute error, and true solution in the space-time domain for the test-set samples with the lowest (a) and highest (b) NRMSE.
Figure 5 : ADR test case: latent space (Meta-IC-LDNet). (a) Test-set initial codes, colored by the amplitude (left) and the phase (right) of the corresponding initial conditions. (b) Latent trajectory (red) of an advection-dominated case with the reported PDE coefficients, integrated over a longer time horizon than in training (i.e, T=50T=50 ), superimposed on the test-set initial codes (gray).
Figure 6 : ADR test case: initial latent velocity against PDE parameters (Meta-IC-LDNet). For each test-set sample, the initial latent velocity is expressed in polar coordinates (ρ,ϑ)(\rho,\vartheta) with respect to the center of the initial codes. The radial velocity, normalized by the initial radius ρ0\rho_{0} , is plotted against μ1π2+μ3\mu_{1}\pi^{2}+\mu_{3} (left), and the angular velocity, normalized by π\pi , against μ2\mu_{2} (right). The red dashed lines are linear fits, with the corresponding coefficient of determination r2r^{2} .
Figure 7 : ADR test case: initial-code fitting on the test set. Each panel refers to one of the first nine test-set samples (same in (a) and (b)) and shows the contours of the MSE loss on the initial condition as a function of the initial latent code (s1,s2)(s_{1},s_{2}) . The red line is the optimization path and the orange star is the final code (i.e., 10410^{4} Adam iterations from a random initialization close to the origin for the AD-IC-LDNet, and three gradient-descent steps from s0=0\mathbf{s}_{0}=\mathbf{0} for the Meta-IC-LDNet).
Figure 8 : ADR test case: evolution of the inner-loop loss function (Meta-IC-LDNet). Contours of the inner-loop loss as a function of the initial latent code (s1,s2)(s_{1},s_{2}) , for the same validation-set sample at selected training epochs. The corresponding training and validation losses are shown in Figure 28 .
Figure 9 : 2D flow around a circular cylinder: domain and boundary conditions (not to scale). The dataset fields are sampled in the region of interest (RoI, red dashed box).
Figure 10 : 2D flow around a circular cylinder: force coefficients from the OpenFOAM simulation at ReD=100\mathrm{Re}_{D}=100 . (a) Time history of the lift and drag coefficients. (b) FFT magnitude of the lift coefficient for the developed flow. The dashed line identifies the shedding frequency, corresponding to St=0.167\mathrm{St}=0.167 .
Figure 11 : Static-cylinder test case: final-time normalized test-set fields (AD-IC-LDNet). Predicted (top), absolute error (middle), and true (bottom) pressure, xx -velocity, and yy -velocity at the last time step of test-set samples 1 (a) and 3 (b), numbered as in Figure 13 . The fields are normalized to [−1,1][-1,1] using the training-set minimum and maximum values.
Figure 12 : Static-cylinder test case: final-time test-set vorticity (AD-IC-LDNet). Predicted (top), absolute error (middle), and true (bottom) vorticity at the last time step of test-set samples 1 (a) and 3 (b). The predicted vorticity is obtained by automatic differentiation of the predicted velocity field, while the true one is computed by OpenFOAM.
Figure 13 : Static-cylinder test case: initial latent states (AD-IC-LDNet). Training-set initial codes, colored by the index of the initial time step within the shedding period, and test-set initial codes (crosses), numbered for reference in Figures 11 and 12 .
Figure 14 : Static-cylinder test case: physical interpretation of initial latent states (Meta-IC-LDNet). Initial codes of the training, validation, and test sets combined, colored by the lift coefficient CLC_{\mathrm{L}} (left) and its time derivative dCL/dt\mathrm{d}C_{\mathrm{L}}/\mathrm{d}t (right) at the initial time, both min-max normalized to [−1,1][-1,1] .
Figure 17 : 2D flow around a rotating circular cylinder: domain and boundary conditions (not to scale). Same configuration as in Figure 9 , with the cylinder rotating at a time-varying angular velocity γ\gamma and a smaller region of interest.
Figure 18 : 2D flow around a rotating circular cylinder: force coefficients. Lift and drag coefficients from the OpenFOAM simulation over the time window used to build the dataset, together with the angular velocity of the cylinder. To match the scale of the lift and drag coefficients, the angular velocity is reported as the equivalent potential-flow CLC_{\mathrm{L}} .
Figure 19 : Rotating-cylinder test case: latent-space PCA (Meta-IC-LDNet). The principal components sI,…,sIVs_{\mathrm{I}},\dots,s_{\mathrm{IV}} are computed on the latent trajectories of the training, validation, and test sets combined. (a) Initial codes, colored by C~L\tilde{C}_{\mathrm{L}} (i.e., the lift coefficient minus its sliding average over 30 time steps) and by dCL/dt\mathrm{d}C_{\mathrm{L}}/\mathrm{d}t . (b) Third and fourth principal components along twelve consecutive sequences (separated by dashed lines), compared with the angular velocity γ\gamma and its time derivative, delayed by the lag (in time steps) that maximizes the Pearson correlation. The colors in (a) and the curves in (b) are min-max normalized to [−1,1][-1,1] .
Figure 20 : Rotating-cylinder test case: final-time normalized test-set fields (Meta-IC-LDNet). Predicted (top), absolute error (middle), and true (bottom) normalized fields at the last time step of the first test-set sample, arranged as in Figure 11 .
Figure 22 : Rotating-cylinder test case: force coefficients (Meta-IC-LDNet). Lift and drag coefficients of the first test-set sample, computed from the predicted fields via surface integration (S.I.) on the cylinder wall and via momentum balance on a control volume (C.V.) around the cylinder, and compared with the OpenFOAM reference (dashed lines). (a) Original model. (b) Model fine-tuned on a denser grid of points, including an O-grid around the cylinder.
Figure 23 : Rotating-cylinder test case: optimal control (Meta-IC-LDNet). The fine-tuned model is used to recover the angular velocity that reproduces the lift coefficient of the first test-set sample. (a) Angular velocity obtained by solving the optimal-control (OC) problem, compared with the original one. (b) Lift coefficient predicted by the surrogate model with the OC and with the original angular velocity, compared with the OpenFOAM target.
Figure 24 : Nonlinear solid dynamics of a 3D cantilever beam: domain and boundary conditions (not to scale). The beam is clamped at x=0x=0 , while axial ( fx\mathbf{f}_{x} ) and shear ( fy\mathbf{f}_{y} , fz\mathbf{f}_{z} ) tractions and a twisting moment ( q\mathbf{q} ) are applied on the free end.
Figure 25 : Nonlinear-beam test case: inference on the test set (Meta-IC-LDNet). Predicted and true displacement at the center of the free-end cross-section (left) and rotation angles of the cross-section about the xx , yy , and zz axes, computed with the Kabsch algorithm (right), for the test-set samples with the lowest (a) and highest (b) NRMSE.
Figure 26 : Nonlinear-beam test case: effect of latent-space principal components (Meta-IC-LDNet). Each panel shows the free-end cross-section in the yzyz plane, decoded from latent states where a single principal component (PC) varies between its 1st and 99th percentiles over all latent states (from light gray to dark blue), while the remaining components are fixed at their mean values. The red dashed square corresponds to all components at their mean values. Displacements are amplified by the indicated scale factor for visualization.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Test case: ADR
ParameterAD-IC-LDNetMeta-IC-LDNet
Architecture
Latent dimension dsd_{s}2
NNdyn\mathcal{NN}_{\mathrm{dyn}} architecture (activ.)FCNN (tanh)FCNN (SiLU)
NNdyn\mathcal{NN}_{\mathrm{dyn}} hidden layers (width)2 (9)
NNrec\mathcal{NN}_{\mathrm{rec}} architecture (activ.)FCNN (tanh)FCNN (SiLU)
Appendix
Table 3 : ADR test case: hyperparameters (AD-IC-LDNet and Meta-IC-LDNet).
Test case: static cylinder
ParameterAD-IC-LDNetMeta-IC-LDNet
Architecture
Latent dimension dsd_{s}2
NNdyn\mathcal{NN}_{\mathrm{dyn}} architecture (activ.)FCNN (SiLU)
NNdyn\mathcal{NN}_{\mathrm{dyn}} hidden layers (width)2 (16)
NNrec\mathcal{NN}_{\mathrm{rec}} architecture (activ.)FiLM-modulated SIREN (sin)
Appendix
Table 4 : Static-cylinder test case: hyperparameters (AD-IC-LDNet and Meta-IC-LDNet).
Test case: rotating cylinder
ParameterMeta-IC-LDNet
Architecture
Latent dimension dsd_{s}4
NNdyn\mathcal{NN}_{\mathrm{dyn}} architecture (activ.)FCNN (SiLU)
NNdyn\mathcal{NN}_{\mathrm{dyn}} hidden layers (width)2 (32)
NNrec\mathcal{NN}_{\mathrm{rec}} architecture (activ.)FiLM-modulated SIREN (sin)
Appendix
Table 5 : Rotating-cylinder test case: hyperparameters (Meta-IC-LDNet).
Test case: nonlinear beam
ParameterMeta-IC-LDNet
Architecture
Latent dimension dsd_{s}12
NNdyn\mathcal{NN}_{\mathrm{dyn}} architecture (activ.)FCNN (SiLU)
NNdyn\mathcal{NN}_{\mathrm{dyn}} hidden layers (width)2 (32)
NNrec\mathcal{NN}_{\mathrm{rec}} architecture (activ.)FCNN (SiLU)
Appendix
Table 6 : Nonlinear-beam test case: hyperparameters (Meta-IC-LDNet).
Figure 27 : ADR test case: training, validation and testing (AD-IC-LDNet).
Figure 28 : ADR test case: training and validation losses against epochs (Meta-IC-LDNet).
Figure 29 : Static-cylinder test case: training, validation and testing (AD-IC-LDNet).
Figure 30 : Static-cylinder test case: training and validation losses against epochs (Meta-IC-LDNet).
Figure 31 : Rotating-cylinder test case: training and fine-tuning (Meta-IC-LDNet).
Figure 32 : Nonlinear-beam test case: training and validation losses against epochs (Meta-IC-LDNet).

Explore similar work

CardsList
Mar 13, 2026cs.LG

Disentangled Latent Dynamics Manifold Fusion for Solving Parameterized PDEs

Generalizing neural surrogate models across different PDE parameters remains difficult because changes in PDE coefficients often make learning harder and optimization less stable. The problem becomes even more severe when the model must also predict beyond the training time range. Existing methods usually cannot handle parameter generalization and temporal extrapolation at the same time. Standard parameterized models treat time as just another input and therefore fail to capture intrinsic dynamics, while recent continuous-time latent methods often rely on expensive test-time auto-decoding for each instance, which is inefficient and can disrupt continuity across the parameterized solution space. To address this, we propose Disentangled Latent Dynamics Manifold Fusion (DLDMF), a physics-informed framework that explicitly separates space, time, and parameters. Instead of unstable auto-decoding, DLDMF maps PDE parameters directly to a continuous latent embedding through a feed-forward network. This embedding initializes and conditions a latent state whose evolution is governed by a parameter-conditioned Neural ODE. We further introduce a dynamic manifold fusion mechanism that uses a shared decoder to combine spatial coordinates, parameter embeddings, and time-evolving latent states to reconstruct the corresponding spatiotemporal solution. By modeling prediction as latent dynamic evolution rather than static coordinate fitting, DLDMF reduces interference between parameter variation and temporal evolution while preserving a smooth and coherent solution manifold. As a result, it performs well on unseen parameter settings and in long-term temporal extrapolation. Experiments on several benchmark problems show that DLDMF consistently outperforms state-of-the-art baselines in accuracy, parameter generalization, and extrapolation robustness.

Zhangyong Liang, Huanhuan Gao

National Center for Applied Mathematics, Tianjin University, Tianjin, 300072, PR China · School of Mechanical and Aerospace Engineering, Jilin University, Changchun, 130025, China

Neural Partial Differential Equation SolversPartial Differential EquationsNeural Ordinary Differential EquationsLatent DynamicsSurrogate Models
AskPDF
May 13, 2026cs.LG

Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations

Partial differential equations (PDEs) are fundamental for modeling complex natural and physical phenomena. In many real-world applications, however, observational data are extremely sparse, which severely limits the applicability of both classical numerical solvers and existing neural approaches. While neural methods have shown promising results under moderately sparse observations, their inference efficiency at high resolutions is limited, and their accuracy degrades substantially in the extremely sparse regime. In this work, we propose the Di-BiLPS, a unified neural framework that effectively handle both forward and inverse PDE problems under extremely sparse observations. Di-BiLPS combines a variational autoencoder to compress high-dimensional inputs into a compact latent space, a latent diffusion module to model uncertainty, and contrastive learning to align representations. Operating entirely in this latent space, the framework achieves efficient inference while retaining flexible input-output mapping. In addition, we introduce a PDE-informed denoising algorithm based on a variance-preserving diffusion process, which further improves inference efficiency. Extensive experiments on multiple PDE benchmarks demonstrate that Di-BiLPS consistently achieves SOTA performance under extremely sparse inputs (as low as 3%), while substantially reducing computational cost. Moreover, Di-BiLPS enables zero-shot super-resolution, as it allows predictions over continuous spatial-temporal domains.

Zhonghao Li, Chaoyu Liu, Qian Zhang

School of Science, Harbin Institute of Technology, Shenzhen, China · Department of Theoretical Physics and Applied Mathematics, University of Cambridge, Cambridge, United Kingdom

Neural Partial Differential Equation SolversBayesian Inverse ProblemsDeep Unfolding NetworksGenerative Super-ResolutionLatent Diffusion Model
AskPDF
Jun 25, 2026cs.CE

Latent Diffusion Posterior Sampling with Surrogate Likelihood Guidance for PDE Inverse Problems

We propose latent-space diffusion posterior sampling (L-DPS), an approximate Bayesian framework for high-dimensional inverse problems governed by partial differential equations (PDEs). The method addresses three challenges in PDE-constrained inversion: implicit sample-based priors without tractable densities, high-dimensional spatially distributed parameters, and the high cost of repeated forward-model evaluations during posterior sampling. L-DPS combines a variational autoencoder, an unconditional latent diffusion model, diffusion posterior sampling, and a differentiable neural surrogate. The VAE maps the parameter field to a lower-dimensional latent space, the diffusion model learns an implicit prior score in this latent space, and DPS combines this learned prior with likelihood-based guidance. The likelihood gradient is evaluated through the decoder-surrogate composition, avoiding repeated calls to the full numerical PDE solver. We evaluate the method on an inverse Darcy flow problem with an unknown spatially distributed permeability field inferred from sparse and noisy pressure observations. L-DPS produces accurate and robust inverse solutions, reduces inference cost relative to full-space DPS, and outperforms amortized inverse baselines such as conditional latent diffusion and inverse FNO in sparse and noisy regimes. We further compare L-DPS with a KLE-MAP baseline and study mixed-prior generalization and the sensitivity of inversion accuracy to surrogate forward-model error.

Yuanzhe Wang, Alexandre M. Tartakovsky

Department of Civil and Environmental Engineering University of Illinois Urbana-Champaign Urbana, IL 61801

Bayesian Inverse ProblemsPosterior Sampling
AskPDF