Stable and Calibrated Latent Force ODE Filters

Paul Fischer; Jonathan Schmidt; Philipp Hennig

cs.LGOct 24, 2026

Stable and Calibrated Latent Force ODE Filters

Authors: Paul Fischer, Jonathan Schmidt, Philipp Hennig

Organizations: Tübingen AI Center University of Tübingen

Abstract

Estimating unobserved forces in mechanistic systems is a central task of scientific machine learning. Approximate Gaussian filtering methods that synthesize mechanistic and empirical information have shown remarkable capacity in jointly identifying latent processes together with the system state in a single forward-backward pass. The resulting latent force ODE filter framework channels information from observational data specifically into the unknown parts of the system. This work addresses open questions about calibration and stability of the method. We show that the joint posterior over state and force is the reweighted conventional latent force posterior. This uncovers a crucial dependence of the stability of the method on the prior variance of the system state, which effectively controls the flow of information from observations to the latent force through the mechanistic system. From this insight, we construct an improved algorithm that provides stable and calibrated plug-and-play inference. We demonstrate the improved framework on several simulated experimental settings.

Figures & tables

Figure 1: The state diffusion controls the latent force inference. Latent force ODE filter on the Duffing oscillator of Eq. ( 16 ) with the position observed (Appendix D ). Center: the diffusion

\sigma_{x}^{\star 2}

that maximizes

I(U;Y\mid Z)

(Sec. 3.4 ). Left and right:

10^{8}

times smaller and

10^{6}

times larger. Bands: mean

\pm 2

standard deviations of the classical latent force model with the probabilistic numerical solver,

\pi(X,U)

, and of the full latent force ODE filter posterior

p(X,U\mid Y,Z)

Panel of Figure 2 — Figure 2: Directed graphical model of the latent force ODE filter. State and force are coupled only through the residual pseudo-observation and the initial state. $\varphi_{x}$ , $\varphi_{u}$ : prior hyperparameters. Shaded: observed. Plates: $N$ grid nodes and $K\leq N$ observation times. : mechanism $f$ , replaced at inference by $Z$ .

Criterion $\mathcal{J}$	Definition	Passes
$\mathcal{L}_{\mathrm{OBS}}$	Eq. ( 8 )	1
$\log p(Y\mid Z)$	$\mathcal{L}_{\mathrm{ODE}}+\mathcal{L}_{\mathrm{OBS}}-\log p(Z)$	2
$D_{\xi}$	Eq. ( 14 )	2
$I_{\xi}(U;Y\mid Z)$	Eq. ( 13 )	4

Table 1: Criteria

\mathcal{J}

for the state diffusion (line 8 of Algorithm 1 ): evidence (top) and information (bottom), causal member first, with the cost of one evaluation in filter passes.

Figure 3: The force error is smallest where $I(U;Y\mid Z)$ is largest. (a) Relative force error and (b) mutual information $I(U;Y\mid Z)$ against the state diffusion $\sigma_{x}^{2}$ on Duffing (Appendix D ), with the IEKS and $\varphi_{u}$ at the truth. One line per grid density $\kappa$ (color bar). Per grid, marks the maximizer of $I(U;Y\mid Z)$ and that of $\mathcal{L}_{\mathrm{ODE}}$ .

Panel of Figure 5 — Algorithm 1 Latent force ODE filter with the diffusion selected by a criterion $\mathcal{J}$ (Table 1 ).

Figure 4: Error ratio, force

\chi^{2}

and roughness ratio of the criteria and the classical LFM on the

60

problems. Columns: Duffing, FitzHugh–Nagumo and Lorenz-96, one dot per problem and method. Top: error ratio, the relative force error over the smallest relative force error of any method on the same problem, with the miss threshold

1.5

dashed. Fits above it are outlined, also in the rows below. Middle: force

\chi^{2}

, with the 95 % interval of a calibrated posterior whose errors are correlated as the true force prior in gray (Appendix E.6 ). Bottom: roughness ratio

(\sigma_{u}^{2}/\ell_{u}^{5})/(\sigma_{u}^{\star 2}/\ell_{u}^{\star 5})

of the fitted force prior to the true one, below one for a force that is too smooth and above one for a force that is too rough.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A.1: The

4

misses of

\mathcal{L}_{\mathrm{OBS}}

. Force (top) and the scored state with its data (bottom) against

t/T

. On Duffing the acceleration is observed, so no data are shown. Bands: mean

\pm 2

standard deviations. Purple (outline):

\sigma_{x}^{2}

\mathcal{L}_{\mathrm{OBS}}

. Blue:

\sigma_{x}^{2}

I(U;Y\mid Z)

. Navy (dotted): classical LFM. Dashed: truth.

Figure A.2: The

4

misses of

D

, all on Lorenz-96. Force (top) and the observed site

x_{0}

with its data (bottom) against

t/T

. Bands: mean

\pm 2

standard deviations. Purple (outline):

\sigma_{x}^{2}

D(U;Y\mid Z)

. Blue:

\sigma_{x}^{2}

I(U;Y\mid Z)

. Navy (dotted): classical LFM. Dashed: truth.

Figure A.3: The miss of

\log p(Y\mid Z)

(left) and of the classical model (right). Force (top) and the scored state (bottom) against

t/T

. Bands: mean

\pm 2

standard deviations. Purple (outline): the failing method. Blue:

\sigma_{x}^{2}

I(U;Y\mid Z)

. Navy (dotted): classical LFM. Dashed: truth.

Figure A.4: Error ratio and force

\chi^{2}

off the model. One dot per problem and method, for truth forces outside the model’s prior (pooled over Duffing, FitzHugh–Nagumo and Lorenz-96,

10

problems each) and for the SIR model. Top: error ratio, with the miss threshold

1.5

dashed. Fits above it are outlined, also in the row below. Bottom: force

\chi^{2}

, with the 95 % interval of a calibrated posterior whose errors are correlated as the model’s force prior in gray (Appendix E.6 ), one interval per pooled system, drawn on top of each other.

Figure A.5: Example posteriors off the model. One problem per kind of truth force (columns), chosen as in the text. Force (top) and the scored state with its data (bottom) against

t/T

. Bands: mean

\pm 2

standard deviations. Blue:

\sigma_{x}^{2}

I(U;Y\mid Z)

. Purple (outline):

\sigma_{x}^{2}

\mathcal{L}_{\mathrm{OBS}}

. Navy (dotted): classical LFM. Dashed: truth. The posteriors nearly coincide, and for the piecewise-constant force all of them round off the jumps, which a Matérn-

5/2

prior cannot represent.

	$\sigma_{x}^{2}\to 0$	$\sigma_{x}^{2}\to\infty$
$L_{x}$	$L_{x\mid u}(I-\Pi)$	$L_{x\mid u}$
$L_{u}$	$P_{uu}^{-}B^{\top}S_{u}^{+}\Pi$	$P_{uu}^{-}B^{\top}S_{x}^{-1}\to 0$
$\mu_{x}^{\mathrm{+}}$	$-L_{x\mid u}(I-\Pi)\hat{z}$	$-L_{x\mid u}\hat{z}$
$\mu_{u}^{\mathrm{+}}$	$-P_{uu}^{-}B^{\top}S_{u}^{+}\Pi\hat{z}$	$0$
$P_{xx}^{\mathrm{+}}$	$P_{xx}^{-}-L_{x\mid u}(I-\Pi)CP_{xx}^{-}\to 0$	$P_{x\mid u}^{\mathrm{+}}\to\infty$
$P_{xu}^{\mathrm{+}}$	$-P_{xx}^{-}C^{\top}\Pi^{\top}S_{u}^{+}BP_{uu}^{-}\to 0$	$-L_{x\mid u}BP_{uu}^{-}$

Appendix

Table C.1: Limits of the single update as

\sigma_{x}^{2}\to 0

and

\sigma_{x}^{2}\to\infty

. Leading order of the single residual update (top), of the innovation covariance and gains of the subsequent observation update (middle), and of the posterior after both updates (bottom), as

\sigma_{x}^{2}\to 0

and as

\sigma_{x}^{2}\to\infty

. Blue : the entry vanishes in the limit. Orange : it diverges in proportion to

\sigma_{x}^{2}

. Black: it tends to a finite limit that does not depend on the diffusion.

P_{xx}^{-}

S_{x}

and

P_{x\mid u}^{\mathrm{+}}

are proportional to

\sigma_{x}^{2}

. If the force reaches every component of the residual,

\Pi=I

, and

L_{x}

and the state means vanish in proportion to

\sigma_{x}^{2}

for

\sigma_{x}^{2}\to 0

Figure D.1: Relative error and $\chi^{2}$ of force and state against the state diffusion. Relative error of force and state (top rows) and their $\chi^{2}$ (bottom rows) against $\sigma_{x}^{2}$ on Duffing, FitzHugh–Nagumo and Lorenz-96 (columns) with $\varphi_{u}$ at the truth, one line per grid density $\kappa$ , computed with the IEKS iterated to convergence. Only converged points are drawn. Diamonds: the $\sigma_{x}^{2}$ maximizing $I(U;Y\mid Z)$ on each grid (Fig. D.2 ), placed on the curves by interpolation. Squares: the $\sigma_{x}^{2}$ maximizing $\mathcal{L}_{\mathrm{ODE}}$ . Both in the grid’s color.

Figure D.2: Force error and the four criteria against the state diffusion on three problems that $\mathcal{L}_{\mathrm{OBS}}$ misses. Relative force error, $\mathcal{L}_{\mathrm{OBS}}$ and $\log p(Y\mid Z)$ below their maxima, $D(U;Y\mid Z)$ and $I(U;Y\mid Z)$ (rows) against $\sigma_{x}^{2}$ on the first FitzHugh–Nagumo, the 11th Duffing and the 10th Lorenz-96 problem of Sec. 4.2 (columns), with the IEKS and $\varphi_{u}$ at the truth, one line per grid density $\kappa$ . Only converged points are drawn. Each criterion’s row marks its own maximizer: stars for $\mathcal{L}_{\mathrm{OBS}}$ , plus signs for $\log p(Y\mid Z)$ , triangles for $D(U;Y\mid Z)$ and diamonds for $I(U;Y\mid Z)$ .

Bayesian Filtering

Bayesian filtering recursively estimates hidden states and their uncertainty from noisy, sequential observations in changing systems.

See all 59 papers

Jun 30, 2026

A Bayesian Filtering Approach for Learning Lagrangian Dynamics from Noisy Measurements

This paper proposes a Bayesian filtering-based approach for learning the dynamics of a physical system from partial, noisy measurements. We model the system dynamics using a Lagrangian mechanics formulation. As in Lagrangian neural networks (LNNs), we parameterize the kinetic and potential energies with neural networks. The unknown external forces in the Lagrangian formulation are modeled as white Gaussian noise. The corresponding Euler--Lagrange equations then yield a continuous-time stochastic state-space model (SSM) that describes the system dynamics. The neural network parameters and system states are then jointly learned via a maximum-likelihood method using Gaussian-approximation-based Bayesian filters. The effectiveness of the proposed method is demonstrated on pendulum and Duffing oscillator examples, and its performance is compared with conventional LNNs and with approximate Bayesian filters using known system models.

Kundan Kumar, Shreya Das, Simo Särkkä

K. Kumar is with the School of Engineering and Applied Science, Ahmedabad University, India · K. Kumar and S. Särkkä are with the ELLIS Institute Finland and the Department of Electrical Engineering and Automation, Aalto University, Finland · S. Das is with the Department of Electrical Engineering, Indian Institute of Technology (BHU) Varanasi, India

Bayesian Filtering Nonlinear System Identification Lagrangian Neural Networks

Ask PDF

Oct 23, 2026

ZENO: Nonlinear filtering with Zakai Equation Neural Operators

State inference and parameter learning for nonlinear filters is a core computation for scientific machine learning with varied applications in engineering and neuroscience. The unnormalized filtering distribution is governed by the Zakai equation, a stochastic partial differential equation (SPDE) that combines the forward Kolmogorov operator (known as the Fokker-Planck equation) with observed likelihood updates. Although solving these high-dimensional SPDEs is often impossible analytically and intractable numerically, advances in operator learning and physics-informed machine learning have made it feasible. Harnessing these advances, we introduce the Zakai Equation Neural Operator (ZENO), a model that optimizes a neural operator against a Zakai Equation based physics-informed loss to approximate the exact unnormalized filtering distribution for latent stochastic dynamical systems. We use this ZENO-inferred posterior distribution in an expectation-maximization (EM) framework to perform parameter learning and state inference. We demonstrate ZENO's ability to accurately infer the posterior distribution and learn its corresponding SDE parameters in three simulated stochastic dynamical systems (Double-Well bistable system, Van Der Pol (VDP) oscillator, and Lorenz system) and compare its performance against established methods for latent SDE inference and learning. We show that ZENO recovers previously observed limit cycles representing forward and backward locomotion from a whole-brain imaging dataset of immobilized C. elegans nematodes. ZENO is a novel approach to the problem of nonlinear filtering and dynamical inference through the lens of partial differential equations and operator theory.

Aaditya Prasad, Steven W. Flavell, Brian DePasquale

Picower Institute for Learning & Memory, Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Howard Hughes Medical Institute, Picower Institute for Learning & Memory, Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA

Latent Dynamics Modeling Stochastic Differential Equations Physics-Informed Neural Operators

Ask PDF

Sep 24, 2026

Learning to Bias: Machine Learning-Enhanced Particle Filters

Sequential inference estimates latent states from noisy and incomplete observations. Particle Filters (PFs), a class of Monte Carlo methods based on importance sampling, provide a flexible framework for this task, but often suffer from poor sample efficiency and unfavorable scaling with dimension, partly due to suboptimal proposal distributions. We address these challenges by integrating learned proposals into the PF framework. We introduce Neural Optimal Particle Filters (NOPFs), which learn an amortized approximation to the optimal proposal from offline simulated one-step conditioning tuples. The learned proposal is used as a drop-in replacement in standard PF updates, with samples corrected by standard importance weights so that the method asymptotically targets the same filtering distribution under standard support and density-evaluation assumptions. Across stochastic nonlinear benchmarks of varying inference complexity, NOPFs improve sample efficiency and distributional accuracy over standard PF baselines with modest computational overhead. The approach integrates data-driven proposal learning into classical inference without altering the underlying filtering objective.

Apoorv Srivastava, Eric Darve

Stanford University Stanford, CA 94305

Particle Filtering Bayesian Filtering Amortized Inference Importance Sampling

Ask PDF

Abstract

Bayesian Filtering

Explore similar work

A Bayesian Filtering Approach for Learning Lagrangian Dynamics from Noisy Measurements

ZENO: Nonlinear filtering with Zakai Equation Neural Operators

Learning to Bias: Machine Learning-Enhanced Particle Filters