System identification, namely the inference of mathematical representations of dynamical systems from measured data, is fundamental to engineering diagnostics, prognostics, and decision-making. Bayesian filtering offers a principled framework for probabilistic state estimation but traditionally relies on prescribed transition and observation models. Recent advances in deep Bayesian filtering address this limitation by embedding learnable representations within probabilistic state-space formulations, enabling data-driven identification of unknown dynamics while retaining physical structure and uncertainty-aware inference. This tutorial provides a systematic introduction to deep Bayesian filtering for nonlinear structural system identification, focusing on the interplay between learned dynamics, physical inductive biases, and posterior inference. The Deep Markov Model serves as a data-driven reference, while the Physics-guided Deep Markov Model (PgDMM) and Neural Extended Kalman Filter (Neural EKF) illustrate complementary approaches based on amortized variational inference and differentiable Kalman filtering, respectively. Their capabilities are examined through dynamic state estimation and joint dynamic state and parameter estimation on a three-dimensional shear-frame benchmark with hysteretic Bouc-Wen connections and sparse acceleration measurements. The results highlight important distinctions between measurement-conditioned reconstruction and open-loop forecasting, demonstrate how physical structure can improve latent-state interpretability without necessarily ensuring predictive fidelity, and reveal differences in the robustness of response and parameter estimation. Practical guidelines for model construction, training, evaluation, and deployment are provided, together with an outlook on remaining challenges in uncertainty calibration, identifiability, and real-world validation.
Figures & tables
Figure 1: Common probabilistic state-space skeleton and method-specific realizations of the three deep Bayesian filtering frameworks considered in this work. The top panel shows the latent state zk evolving under the transition model fν and generating observations xk through the emission model hϕ , with the dynamics driven by inputs uk . Inference recovers the latent state from the observation sequence through an approximate posterior q . The bottom panels arrange the three frameworks according to increasing physics-based inductive bias. The DMM constitutes the fully data-driven baseline, with learned transition, emission, and amortized inference models and no prescribed physical organization of the latent state. The Neural EKF introduces an intermediate level of structure by replacing the amortized encoder with the EKF predict-update recursion and, in its structural specialization, by organizing the latent state into displacement- and velocity-like coordinates, while retaining learnable transition and emission models. The PgDMM introduces stronger physics-based bias by embedding a parameter-dependent ROM within the transition and emission models, while retaining the amortized inference architecture of the DMM .
Figure 2: Schematic of the shear frame and link mechanism. Blue crosses denote sensor locations.
Parameter:
we
α
wa
β
γ
A,m
wk
Range:
[0.74,1.32]
0.2
U(0.0,1.0)
3.0
2.0
1.0
U(0.1,1.0)
Table 1: Modeling assumptions for the shear-frame. The uniform distribution in (⋆,∙) is U(⋆,∙) . The range of we results from the temperature law given in the text.
Method
run
observations
latent state
window
split
forecast warm-up
PgDMM
131
10 accelerations
16 modal displacements + 16 velocities
200 steps (2 s)
5/1/1
20 steps (0.2 s)
Neural EKF
131
10 accelerations
10 nodal displacements + 10 velocities
200 steps (2 s)
5/1/1
20 steps (0.2 s)
Table 2: Configurations of the dynamic state estimation experiments. The split reports the number of non-overlapping train/validation/test windows.
Open-loop forecast (warm-up prefix only, then free-running dynamics)
Norm. RMSE x over 0.2 s horizon (17 origins)
0.0675
0.0488
Norm. RMSE x over 0.5 s horizon (14 origins)
0.0694
0.0539
Norm. RMSE x over 1.0 s horizon (9 origins)
0.0789
0.0657
Table 3: Dynamic state estimation results on the held-out test window, for the setup of Table 2 . Response errors are RMSE of the ten accelerations, normalized by the training statistics of each channel, in which units the measurement noise has a standard deviation of 0.042 . The three shorter forecast horizons pool the number of rollout origins given in parentheses. State recovery is the normalized RMSE between the projected latent state and the hidden simulation state, averaged over the ten monitored DOFs ; both projections are at raw scale, with nothing fitted to the reference. The first two state-recovery rows score detrended signals, with all displacement content below 1 Hz, which includes the linear trend an acceleration record cannot determine, and the velocity mean removed from the recovered and the reference signal alike; the next two score the projection as it stands. The noise floor is the error of an estimate that matched the noise-free state exactly.
Figure 3: Posterior reconstruction and open-loop forecasting on the held-out test window, for the dynamic state estimation PgDMM (top) and Neural EKF (bottom). The shaded interval is the 0.2 -s measurement warm-up. Left: normalized acceleration at monitored channel 3, which under the numbering of section 4 is the longitudinal ( x ) channel of the second sensor in 2(a) ; right: channel 4, its transverse ( y ) channel.
Figure 4: Horizon dependence of the open-loop forecast and per-channel hidden-state recovery, both on the held-out test window. The dashed line in 4(a) is the standard deviation of the measurement noise. State recovery in 4(b) is scored as in Table 3 , with displacement and velocity detrended, and the black segments mark the noise floor of the reference state. The alternating pattern reflects the two measurement directions: the odd- and even-indexed channels form two groups that are mutually correlated within a group and essentially uncorrelated across groups.
Figure 5: Recovery of unmeasured structural displacement (left) and velocity (right) from the inferred latent states in the dynamic state estimation setting. Results are shown for channel 9, corresponding to the longitudinal ( x ) direction of the fifth sensor in 2(a) , selected because its estimation errors are closest to the average errors of each framework. Neither displacement nor velocity is directly observed by either model. The inferred states are projected at their original scale and processed consistently with the reference signals. For displacement, frequencies below 1 Hz are removed, eliminating the integration-related linear trend that cannot be determined from acceleration measurements and accounting for no more than approximately 3% of the reference displacement energy. For velocity, the mean is removed.
Figure 6: Training convergence of the two frameworks in the dynamic state estimation setting, assessed through the open-loop forecast RMSE on the validation (solid lines) and test (dashed lines) windows. The test data are used exclusively for independent performance assessment and do not inform model selection. The dotted vertical lines indicate the checkpoints selected according to each framework’s validation criterion.
Table 4: Default configuration for the joint dynamic state and parameter estimation experiments. All frameworks observe the same ten acceleration channels and three system parameters, sampled at 1 kHz and downsampled by a factor of ten. The latent state is reported with its composition given in parentheses.
Figure 7: Time-history response reconstruction on two representative test sequences, showing the reconstructed (dashed) against the true (solid) normalized acceleration u¨ on a monitored channel over the 1 – 5 s window. The two columns correspond to two different held-out runs, each reporting the transverse y -direction response at the top right monitored node of the second floor in 2(a) .
Parameter estimation — MAE over the test windows, ×10−3
Bouc-Wen stiffness weighting wk
1.71
1.44
2.17
Table 5: Joint dynamic state and parameter estimation on the held-out test windows and test records. The response entries refer to the monitored acceleration u¨ in normalized RMSE . Every channel is standardized by its own mean and standard deviation and a value of 0.05 corresponds to 5% error of the (training) standard deviation of that channel.
Figure 8: Distribution of the normalized acceleration response RMSE over the test windows shown as box plots (left) and as kernel density estimates (right). The vertical axis is logarithmic. The horizontal bar marks the median.
Figure 9: Distribution of the normalized acceleration response RMSE over the test windows per monitored channel shown as box plots (left) and as kernel density estimates (right). Channels are labeled by monitored node and direction following the numbering in 2(a) . The vertical axis is logarithmic.
Figure 10: Distribution of the absolute parameter error over the test windows shown as box plots (left) and as kernel density estimates (right). The vertical axis is logarithmic. The horizontal bar marks the median.
Figure 11: Parameter estimation on the held-out testing windows. Top : inferred against true values over all samples, with the dashed line marking exact recovery. Bottom : the corresponding residuals, scaled by 103 .
Figure 12: Latent-coordinate variance ranking for the joint dynamic state and parameter estimation setting PgDMM of section 4.3 , on a representative held-out testing run. The bars show the contribution of each of the sixteen latent displacement coordinates of the ROM state to the total latent displacement variance. The cumulative contribution is also shown, with the 90% level indicated.
Figure 13: Connection between the dominant latent modes and the frequency content of the input excitation, shown for the dynamic state estimation PgDMM of section 3.5.5 on the held-out window of run 131. Top: power spectral density of the excitation, normalized to its peak. Bottom: variance of each of the modal displacement coordinates, as a share of their total variance, placed at the dominant frequency of its own latent trajectory and colored by increasing frequency.
Figure 14: Frequency content of the inferred latent space for the joint dynamic state and parameter estimation setting PgDMM , DMM and Neural EKF of section 4.3 , on a representative held-out testing run. Each row shows the periodogram of a latent coordinate, normalized to its own peak and displayed on a logarithmic color scale. The horizontal axis is restricted to 15 Hz, encompassing the principal excited frequency band. For the PgDMM , the dashed line separates the displacement (bottom) and velocity (top) coordinates of its ROM state. The corresponding coordinates of the other architectures are not organized according to the same prescribed modal partition.
Deep state-space models (DSSMs) enable temporal predictions by learning the underlying dynamics of observed sequence data. They are often trained by maximising the evidence lower bound. However, as we show, this does not ensure the model actually learns the underlying dynamics. We therefore propose a constrained optimisation framework as a general approach for training DSSMs. Building upon this, we introduce the extended Kalman VAE (EKVAE), which combines amortised variational inference with classic Bayesian filtering/smoothing to model dynamics more accurately than RNN-based DSSMs. Our results show that the constrained optimisation framework significantly improves system identification and prediction accuracy on the example of established state-of-the-art DSSMs. The EKVAE outperforms previous models w.r.t. prediction accuracy, achieves remarkable results in identifying dynamical systems, and can furthermore successfully learn state-space representations where static and dynamic features are disentangled.
Alexej Klushyn, Richard Kurle, Maximilian Soelch +2
Technical University of Munich · AWS AI Labs · Machine Learning Research Lab, Volkswagen Group, Munich +1
Sensor-rich data-driven applications increasingly use Bayesian approaches to infer latent states of dynamic systems from noisy sensor measurements and physical models. Yet the computation of the likelihood remains an essential bottleneck for accurate posteriors and performant inference. This paper presents a Bayesian filtering technique that uses processor-native uncertainty tracking for both uncertainty propagation and inference. The technique implements deterministic hierarchical importance restructuring through a native operation, giving deterministic latency and bounded memory use for arbitrary models written as program code. Benchmarks across three nonlinear state-space systems compare the approach against particle filters and Monte-Carlo-based likelihood estimators. The technique enables deterministic approximate filtering with as high as 805× average speedup against direct Monte Carlo work at matched result quality for model evaluation, and Pareto-dominant accuracy-latency trade-offs for posterior inference while remaining competitive in RMSE with baseline particle filters.
Deep learning has proven highly effective for nonlinear system identification, but heavily parameterized neural networks are prone to overfitting in low-data regimes and lack reliable uncertainty quantification. The recently developed manifold meta-learning framework addresses the data efficiency problem by restricting the model parameters to a meta-learned low-dimensional manifold. However, that method is purely deterministic. We propose a fully probabilistic extension of the manifold meta-learning framework, based on amortized Variational Inference, where a generative prior over the low-dimensional parameter manifold is learned. During task-specific adaptation, we combine Maximum A Posteriori estimation with the Laplace approximation to yield a mathematically grounded posterior approximation. Evaluated on a static regression task and the Bouc--Wen dynamical system benchmark, the proposed approach achieves predictive accuracy comparable to its deterministic counterpart while successfully providing calibrated uncertainty bounds in severely low-data regimes.
Matteo Rufolo, Dario Piga, Marco Forgione
SUPSI-DTI-IDSIA, Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland.