Deep Bayesian Approaches for Interpretable Nonlinear Structural System Identification

Wei Liu; Konstantinos Vlachas; Eleni Chatzi

cs.LGOct 27, 2026

Deep Bayesian Approaches for Interpretable Nonlinear Structural System Identification

Authors: Wei Liu, Konstantinos Vlachas, Eleni Chatzi

Organizations: National University of Singapore Singapore · ETH Zurich Zurich, Switzerland

Abstract

System identification, namely the inference of mathematical representations of dynamical systems from measured data, is fundamental to engineering diagnostics, prognostics, and decision-making. Bayesian filtering offers a principled framework for probabilistic state estimation but traditionally relies on prescribed transition and observation models. Recent advances in deep Bayesian filtering address this limitation by embedding learnable representations within probabilistic state-space formulations, enabling data-driven identification of unknown dynamics while retaining physical structure and uncertainty-aware inference. This tutorial provides a systematic introduction to deep Bayesian filtering for nonlinear structural system identification, focusing on the interplay between learned dynamics, physical inductive biases, and posterior inference. The Deep Markov Model serves as a data-driven reference, while the Physics-guided Deep Markov Model (PgDMM) and Neural Extended Kalman Filter (Neural EKF) illustrate complementary approaches based on amortized variational inference and differentiable Kalman filtering, respectively. Their capabilities are examined through dynamic state estimation and joint dynamic state and parameter estimation on a three-dimensional shear-frame benchmark with hysteretic Bouc-Wen connections and sparse acceleration measurements. The results highlight important distinctions between measurement-conditioned reconstruction and open-loop forecasting, demonstrate how physical structure can improve latent-state interpretability without necessarily ensuring predictive fidelity, and reveal differences in the robustness of response and parameter estimation. Practical guidelines for model construction, training, evaluation, and deployment are provided, together with an outlook on remaining challenges in uncertainty calibration, identifiability, and real-world validation.

Figures & tables

Panel of Figure 1 — Figure 1: Common probabilistic state-space skeleton and method-specific realizations of the three deep Bayesian filtering frameworks considered in this work. The top panel shows the latent state $\bm{z}_{k}$ evolving under the transition model $f_{\nu}$ and generating observations $\bm{x}_{k}$ through the emission model $h_{\phi}$ , with the dynamics driven by inputs $\bm{u}_{k}$ . Inference recovers the latent state from the observation sequence through an approximate posterior $q$ . The bottom panels arrange the three frameworks according to increasing physics-based inductive bias. The DMM constitutes the fully data-driven baseline, with learned transition, emission, and amortized inference models and no prescribed physical organization of the latent state. The Neural EKF introduces an intermediate level of structure by replacing the amortized encoder with the EKF predict-update recursion and, in its structural specialization, by organizing the latent state into displacement- and velocity-like coordinates, while retaining learnable transition and emission models. The PgDMM introduces stronger physics-based bias by embedding a parameter-dependent ROM within the transition and emission models, while retaining the amortized inference architecture of the DMM .

Panel of Figure 2 — Figure 2: Schematic of the shear frame and link mechanism. Blue crosses denote sensor locations.

Parameter:	$w_{e}$	$\alpha$	$w_{a}$	$\beta$	$\gamma$	$A,m$	$w_{k}$
Range:	$[0.74,1.32]$	$0.2$	$\mathcal{U}(0.0,1.0)$	$3.0$	$2.0$	$1.0$	$\mathcal{U}(0.1,1.0)$

Table 1: Modeling assumptions for the shear-frame. The uniform distribution in

(\star,\bullet)

\mathcal{U}(\star,\bullet)

. The range of

w_{e}

results from the temperature law given in the text.

Method	run	observations	latent state	window	split	forecast warm-up
PgDMM	131	10 accelerations	16 modal displacements + 16 velocities	200 steps (2 s)	5/1/1	20 steps (0.2 s)
Neural EKF	131	10 accelerations	10 nodal displacements + 10 velocities	200 steps (2 s)	5/1/1	20 steps (0.2 s)

Table 2: Configurations of the dynamic state estimation experiments. The split reports the number of non-overlapping train/validation/test windows.

Posterior reconstruction (complete observation window used)
	PgDMM	Neural EKF
Normalized RMSE x	0.0203	0.0386
Open-loop forecast (warm-up prefix only, then free-running dynamics)
Norm. RMSE x over $0.2$ s horizon (17 origins)	0.0675	0.0488
Norm. RMSE x over $0.5$ s horizon (14 origins)	0.0694	0.0539
Norm. RMSE x over $1.0$ s horizon (9 origins)	0.0789	0.0657

Table 3: Dynamic state estimation results on the held-out test window, for the setup of Table 2 . Response errors are RMSE of the ten accelerations, normalized by the training statistics of each channel, in which units the measurement noise has a standard deviation of

0.042

. The three shorter forecast horizons pool the number of rollout origins given in parentheses. State recovery is the normalized RMSE between the projected latent state and the hidden simulation state, averaged over the ten monitored DOFs ; both projections are at raw scale, with nothing fitted to the reference. The first two state-recovery rows score detrended signals, with all displacement content below

1

Hz, which includes the linear trend an acceleration record cannot determine, and the velocity mean removed from the recovered and the reference signal alike; the next two score the projection as it stands. The noise floor is the error of an estimate that matched the noise-free state exactly.

Figure 3: Posterior reconstruction and open-loop forecasting on the held-out test window, for the dynamic state estimation PgDMM (top) and Neural EKF (bottom). The shaded interval is the

0.2

-s measurement warm-up. Left: normalized acceleration at monitored channel 3, which under the numbering of section 4 is the longitudinal (

x

) channel of the second sensor in 2(a) ; right: channel 4, its transverse (

y

) channel.

Panel of Figure 4 — Figure 4: Horizon dependence of the open-loop forecast and per-channel hidden-state recovery, both on the held-out test window. The dashed line in 4(a) is the standard deviation of the measurement noise. State recovery in 4(b) is scored as in Table 3 , with displacement and velocity detrended, and the black segments mark the noise floor of the reference state. The alternating pattern reflects the two measurement directions: the odd- and even-indexed channels form two groups that are mutually correlated within a group and essentially uncorrelated across groups.

Figure 5: Recovery of unmeasured structural displacement (left) and velocity (right) from the inferred latent states in the dynamic state estimation setting. Results are shown for channel 9, corresponding to the longitudinal (

x

) direction of the fifth sensor in 2(a) , selected because its estimation errors are closest to the average errors of each framework. Neither displacement nor velocity is directly observed by either model. The inferred states are projected at their original scale and processed consistently with the reference signals. For displacement, frequencies below

1

Hz are removed, eliminating the integration-related linear trend that cannot be determined from acceleration measurements and accounting for no more than approximately

3\%

of the reference displacement energy. For velocity, the mean is removed.

Figure 6: Training convergence of the two frameworks in the dynamic state estimation setting, assessed through the open-loop forecast RMSE on the validation (solid lines) and test (dashed lines) windows. The test data are used exclusively for independent performance assessment and do not inform model selection. The dotted vertical lines indicate the checkpoints selected according to each framework’s validation criterion.

Method	observations	latent state	window	split (runs)
DMM	10 accelerations + 3 parameters	35 (32 unstructured + 3 parameters)	200 steps (2 s)	133/28/29
PgDMM		35 (16 reduced displacements + 16 velocities + 3 parameters)
Neural EKF		35 (32 unstructured + 3 parameters)

Table 4: Default configuration for the joint dynamic state and parameter estimation experiments. All frameworks observe the same ten acceleration channels and three system parameters, sampled at

1

kHz and downsampled by a factor of ten. The latent state is reported with its composition given in parentheses.

Figure 7: Time-history response reconstruction on two representative test sequences, showing the reconstructed (dashed) against the true (solid) normalized acceleration

\ddot{u}

on a monitored channel over the

1

–

5

s window. The two columns correspond to two different held-out runs, each reporting the transverse

y

-direction response at the top right monitored node of the second floor in 2(a) .

Response reconstruction — normalized acceleration $\ddot{\bm{u}}$
	DMM	PgDMM	Neural EKF
RMSE over the test windows	0.0504	0.0336	0.0167
MAE over the test windows	0.0141	0.0111	0.0103
RMSE , median over the entire test runs	0.0262	0.0162	0.0142
Parameter estimation — MAE over the test windows, $\times 10^{-3}$
Bouc-Wen stiffness weighting $w_{k}$	1.71	1.44	2.17

Table 5: Joint dynamic state and parameter estimation on the held-out test windows and test records. The response entries refer to the monitored acceleration

\ddot{\bm{u}}

in normalized RMSE . Every channel is standardized by its own mean and standard deviation and a value of

0.05

corresponds to

5\%

error of the (training) standard deviation of that channel.

Figure 8: Distribution of the normalized acceleration response RMSE over the test windows shown as box plots (left) and as kernel density estimates (right). The vertical axis is logarithmic. The horizontal bar marks the median.

Figure 9: Distribution of the normalized acceleration response RMSE over the test windows per monitored channel shown as box plots (left) and as kernel density estimates (right). Channels are labeled by monitored node and direction following the numbering in 2(a) . The vertical axis is logarithmic.

Figure 10: Distribution of the absolute parameter error over the test windows shown as box plots (left) and as kernel density estimates (right). The vertical axis is logarithmic. The horizontal bar marks the median.

Figure 11: Parameter estimation on the held-out testing windows. Top : inferred against true values over all samples, with the dashed line marking exact recovery. Bottom : the corresponding residuals, scaled by

10^{3}

Figure 12: Latent-coordinate variance ranking for the joint dynamic state and parameter estimation setting PgDMM of section 4.3 , on a representative held-out testing run. The bars show the contribution of each of the sixteen latent displacement coordinates of the ROM state to the total latent displacement variance. The cumulative contribution is also shown, with the

90\%

level indicated.

Figure 13: Connection between the dominant latent modes and the frequency content of the input excitation, shown for the dynamic state estimation PgDMM of section 3.5.5 on the held-out window of run 131. Top: power spectral density of the excitation, normalized to its peak. Bottom: variance of each of the modal displacement coordinates, as a share of their total variance, placed at the dominant frequency of its own latent trajectory and colored by increasing frequency.

Figure 14: Frequency content of the inferred latent space for the joint dynamic state and parameter estimation setting PgDMM , DMM and Neural EKF of section 4.3 , on a representative held-out testing run. Each row shows the periodogram of a latent coordinate, normalized to its own peak and displayed on a logarithmic color scale. The horizontal axis is restricted to $15$ Hz, encompassing the principal excited frequency band. For the PgDMM , the dashed line separates the displacement (bottom) and velocity (top) coordinates of its ROM state. The corresponding coordinates of the other architectures are not organized according to the same prescribed modal partition.

System Identification

System identification infers mathematical models and parameters of dynamical systems from input-output, trajectory, sensor, or video data.

See all 158 papers

Latent Matters: Learning Deep State-Space Models
Feb 26, 2026Alexej Klushyn, Richard Kurle, Maximilian Soelch +2State Space Models Latent Dynamics Modeling
Technical University of Munich · AWS AI Labs · Machine Learning Research Lab, Volkswagen Group, Munich +1
Dynamic Online Processor-Native Inference for State Estimation
Jul 13, 2026Orestis KaparounakisParticle Filtering Bayesian Filtering
Variational meta-learning inference for low dimensional neural system identification
Jul 21, 2026Matteo Rufolo, Dario Piga, Marco ForgioneUncertainty Quantification Meta-Learning
SUPSI-DTI-IDSIA, Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland.

Feb 26, 2026J/K move · Enter open · S save

Latent Matters: Learning Deep State-Space Models

Alexej Klushyn, Richard Kurle, Maximilian Soelch, Botond Cseke+1

Technical University of Munich · AWS AI Labs · Machine Learning Research Lab, Volkswagen Group, Munich+1

State Space Models Latent Dynamics Modeling Variational Autoencoders System Identification State Representation Learning

Open paper AskPDF arXiv

Deep state-space models (DSSMs) enable temporal predictions by learning the underlying dynamics of observed sequence data. They are often trained by maximising the evidence lower bound. However, as we show, this does not ensure the model actually learns the underlying dynamics. We therefore propose a constrained optimisation framework as a general approach for training DSSMs. Building upon this, we introduce the extended Kalman VAE (EKVAE), which combines amortised variational inference with classic Bayesian filtering/smoothing to model dynamics more accurately than RNN-based DSSMs. Our results show that the constrained optimisation framework significantly improves system identification and prediction accuracy on the example of established state-of-the-art DSSMs. The EKVAE outperforms previous models w.r.t. prediction accuracy, achieves remarkable results in identifying dynamical systems, and can furthermore successfully learn state-space representations where static and dynamic features are disentangled.

Abstract

System Identification

Explore similar work

Latent Matters: Learning Deep State-Space Models

Dynamic Online Processor-Native Inference for State Estimation

Variational meta-learning inference for low dimensional neural system identification