cs.LGNov 16, 2026

Conditional Diffusion Models for Amortized Bayesian System Identification

Authors: Mike Hänseroth, Niklas Wahlström, Thomas B. Schön, Thomas Seel, Jan-Hendrik Ewering

Organizations: Institute of Mechatronic Systems, Leibniz University Hannover, Germany · Dep. of Information Technology, Uppsala University, Sweden · L3S Research Center, Leibniz University Hannover, Germany

Abstract

Generative diffusion models rapidly solve Bayesian inverse problems, such as image restoration and inpainting, with a single trained model and without retraining for each new observation or task. Bayesian system identification, which typically involves inferring the hidden states and parameters of a dynamical system from measurements, poses a closely related inverse problem. Yet it is often addressed with computationally heavy Monte Carlo methods that must be rerun for every new measurement record. Moreover, these methods are restricted to a single inference task, and their cost grows with the state dimension and the record length. To alleviate these restrictions, we interpret Bayesian system identification in hiddem Markov models (HMMs) as a general Bayesian inverse problem, taking advantage of conditional diffusion models. Specifically, a transformer-based diffusion model is trained once on short segments of simulated trajectories to approximate the joint posterior of states and parameters. Through an attention mask induced by the Markov property and a relative time encoding, it can then be deployed on any new measurement record of custom length. At inference time, the same model can flexibly and rapidly be conditioned for joint state and parameter inference, state smoothing, parameter inference, or reconstructing missing measurements, without retraining. We demonstrate these capabilities on four benchmark systems and examine how the length of the training segments affects state and parameter inference. On records five times longer than the training segments, the model maintains its state accuracy, whereas comparison methods either show a more than tenfold error increase or require ten times as much sampling time.

Figures & tables

Explore similar work

Jun 19, 2026

Diffusion-Driven State Space Models

In many domains, practitioners seek models that produce accurate forecasts while faithfully capturing latent system dynamics. Existing approaches typically sacrifice one of these goals: deep state space models often assume Gaussian latent transitions, limiting fit and forecasting, while diffusion models are highly expressive but lack principled inference for the underlying dynamics. To combine the strengths of both, we introduce the Diffusion-Driven State Space Model (DDSSM), which replaces the conventional Gaussian transition distribution with a diffusion model. Our DDSSM resolves the open problem of how to jointly train an autoencoder and a diffusion model on sequential data, thereby extending the literature on latent diffusion models for time series. Moreover, we find that the DDSSM empirically outperforms a state-of-the-art deep SSM at fitting and forecasting a simulated time series with multimodal transitions.
Apr 19, 2026

Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification

Machine learning is becoming increasingly important for nonlinear system identification, including dynamical systems with spatially distributed outputs. However, classical identification and forecasting approaches become markedly less reliable in turbulent-flow regimes, where the dynamics are high-dimensional, strongly nonlinear, and highly sensitive to compounding rollout errors. Diffusion-based models have recently shown improved robustness in this setting and offer probabilistic inference capabilities, but many current implementations inherit target parameterizations from image generation, most commonly noise or velocity prediction. In this work, we revisit this design choice in the context of nonlinear spatiotemporal system identification. We consider a simple, self-contained patch-based transformer that operates directly on physical fields and use turbulent flow simulation as a representative testbed. Our results show that clean-state prediction consistently improves rollout stability and reduces long-horizon error relative to velocity- and noise-based objectives, with the advantage becoming more pronounced as the per-token dimensionality increases. These findings identify target parameterization as a key modeling choice in diffusion-based identification of nonlinear systems with spatial outputs in turbulent regimes.
Oct 27, 2026

Deep Bayesian Filtering for Nonlinear Structural System Identification: A Tutorial on Structural and Physical Inductive Biases

Nonlinear structural system identification remains challenging when governing physics is only partially known and measurements are sparse. Deep Bayesian filtering addresses these challenges by combining learnable dynamical representations with probabilistic state-space models, physical inductive biases, and uncertainty-aware inference. This paper presents, for the first time, a unified tutorial treatment of deep Bayesian filtering for nonlinear structural system identification, connecting model formulation, learning, posterior inference, and evaluation. Amortized variational inference and differentiable Kalman filtering are introduced as complementary strategies within this broader framework. The Deep Markov Model (DMM) provides a data-driven reference, while the Physics-guided Deep Markov Model (PgDMM) and Neural Extended Kalman Filter (Neural EKF) serve as illustrative instances through which methodological choices and their implications are examined. Dynamic state estimation and joint state and parameter estimation are demonstrated on a three-dimensional shear-frame benchmark with hysteretic Bouc-Wen connections and sparse acceleration measurements. The comparative analysis yields new observations concerning the relationships among reconstruction accuracy, predictive fidelity, physical interpretability, and estimation robustness. Physical structure improves latent-state interpretability without necessarily ensuring faithful predictive dynamics, while comparable typical reconstruction accuracy can conceal substantial differences in estimation robustness. These findings motivate practical guidelines for model construction, training, and evaluation that distinguish reconstruction, forecasting, state recovery, and parameter identification. The tutorial concludes by outlining challenges in uncertainty calibration, identifiability, and validation on real structures.