Variational LSTM with Augmented Inputs: Nonlinear Response History Metamodeling with Aleatoric and Epistemic Uncertainty
Authors: Manisha Sapkota, Min Li, Bowei Li
Organizations: Department of Civil, Environmental, and Construction Engineering, Texas Tech University, 2500 Broadway St, Lubbock 79409, TX, USA · Department of Civil and Environmental Engineering, Rensselaer Polytechnic Institute, 110 8th Street, Troy 12180, NY, USA · National Wind Institute, Texas Tech University, 1009 Canton Ave, Lubbock 79409, TX, USA
Uncertainty propagation in high-dimensional nonlinear dynamic structural systems is pivotal in state-of-the-art performance-based design and risk assessment, where uncertainties from both excitations and structures, i.e., the aleatoric uncertainty, must be considered. This poses a significant challenge due to heavy computational demands. Machine learning techniques are thus introduced as metamodels to alleviate this burden. However, the "black box" nature of Machine learning models underscores the necessity of avoiding overly confident predictions, particularly when data and training efforts are insufficient. This creates a need, in addition to considering the aleatoric uncertainty, of estimating the uncertainty related to the prediction confidence, i.e., epistemic uncertainty, for machine learning-based metamodels. We developed a probabilistic metamodeling technique based on a variational long short-term memory (LSTM) with augmented inputs to simultaneously capture aleatoric and epistemic uncertainties. Key random system parameters are treated as augmented inputs alongside excitation series carrying record-to-record variability to capture the full range of aleatoric uncertainty. Meanwhile, epistemic uncertainty is effectively approximated via the Monte Carlo dropout scheme. Unlike computationally expensive full Bayesian approaches, this method incurs negligible additional training costs while enabling nearly cost-free uncertainty simulation. The proposed technique is demonstrated through multiple case studies involving stochastic seismic or wind excitations. Results show that the calibrated metamodels accurately reproduce nonlinear response time histories and provide confidence bounds indicating the associated epistemic uncertainty.
Figures & tables
Figure 1: The developed framework
Figure 2: (a) The SDOF Bouc-Wen system subjected to ground motion in Case 1; (b) A realization of the hysteresis curve; (c) A realization of ground motion; (d) Mean spectrum and spectra samples.
Parameters
Distribution
Mean
Support
ω
Uniform
5.97
[5.373, 6.567]
α
Uniform
50
[45, 55]
Table 1: Random Parameters in Bouc-Wen System
Figure 3: Training and validation loss of metamodel in Case 1
Figure 4: (a) Comparison of the mean and 95% confidence interval of the displacement response by the metamodel against the high-fidelity reference; (b) Error time history of the mean response by the metamodel; (c) Comparison between peak displacements by the metamodel and by the high-fidelity reference.
Figure 5: (a) The shear building model in Case 2; (b) A representative inter-story hysteresis curve; (c) The target spectrum, mean and sample spectra of the generated ground motion realizations; (d) A representative ground motion realization.
Parameters
Distribution
Mean
Support
Story Weight (w)
Uniform
100 kips (444.82 kN)
[50,150] kips ([222.41, 667.23]kN)
Post-Yield Stiffness ratio ( α )
Uniform
0.1
[0.05,0.15]
Table 2: Random Parameters in Nonlinear Shear Building
Figure 6: Training and validation loss of metamodel in Case 2
Figure 7: (a)(b) Comparison of displacement response time histories between high-fidelity direct integration and the metamodel, including 95% confidence intervals (light blue shaded areas), at the 3rd and 6th DoFs; (c)(d) Comparison between the peak displacements by the high-fidelity direct integration and by the metamodel, at the 3rd and 6th DoFs; (e) The hysteresis curves by the high-fidelity direct integration and by the metamodel, as well as 100 realizations of the hysteresis curve under epistemic uncertainty (light blue).
Floor
Beams
Columns (cm)
1−20
W24×192
502×2.5
21−30
W24×103
402×2.0
31−37
W24×103
352×1.8
Table 3: Dimensions of the structural members in Case 3
Figure 8: (a) The 37-Story moment-resisting frame considered in Case 3; Comparison between the (b) time histories and (c) power spectral densities (PSD) of synthetic wind load and the wind tunnel data at the top floor.
Figure 9: Training and validation loss of metamodel in Case 3
Figure 10: Comparisons of displacement time histories by the metamodel with 95% of confidence intervals against those by the high-fidelity direct integration at 20th (a) and 37th floor (b); Peak displacements by the metamodel compared to those by the high-fidelity direct integration at 20th (c) and 37th floor (d); (e) Comparisons of strain time histories by the metamodel with 95% of confidence intervals against those by the high-fidelity direct integration; (f) Comparison of the hysteresis curve by the metamodel, its 100 realizations under epistemic uncertainty, against those by the high-fidelity direct integration of an exterior fiber in the exterior column at the 1st floor.
Modeling high-dimensional, nonlinear dynamic structural systems under natural hazards presents formidable computational challenges, especially when simultaneously accounting for uncertainties in external loads and structural parameters. Studies have successfully incorporated uncertainties related to external loads from natural hazards, but few have simultaneously addressed loading and parameter uncertainties within structural systems while accounting for prediction uncertainty of neural networks. To address these gaps, three metamodeling frameworks were formulated, each coupling a feature-extraction module implemented through a multi-layer perceptron (MLP), a message-passing neural network (MPNN), or an autoencoder (AE) with a long short-term memory (LSTM) network using Monte Carlo dropout and a negative log-likelihood loss. The resulting architectures (MLP-LSTM, MPNN-LSTM, and AE-LSTM) were validated on two case studies: a multi-degree-of-freedom Bouc-Wen system and a 37-story fiber-discretized nonlinear steel moment-resisting frame, both subjected to stochastic seismic excitation and structural parameter uncertainty. All three approaches achieved low prediction errors: the MLP-LSTM yielded the most accurate results for the lower-dimensional Bouc-Wen system, whereas the MPNN-LSTM and AE-LSTM provided superior performance on the more complex steel-frame model. Moreover, a consistent correlation between predictive variance and actual error confirms the suitability of these frameworks for active-learning strategies and for assessing model confidence in structural response predictions.
Haimiti Atila, Seymour M. J. Spence
Department of Civil and Environmental Engineering, University of Michigan, Ann Arbor, MI 48109, USA
The Linear Parameter-Varying (LPV) framework enables the construction of surrogate models of complex nonlinear and high-dimensional systems, facilitating efficient stability and performance analysis together with controller design. Despite significant advances in data-driven LPV modelling, existing approaches do not quantify the uncertainty of the obtained LPV models. Consequently, assessing model reliability for analysis and control or detecting operation outside the training regime requires extensive validation and user expertise. This paper proposes a Bayesian approach for the joint estimation of LPV state-space models, including their scheduling map, together with characterization of the model uncertainty and confidence bounds on the predicted model response directly from input-output data. Both aleatoric uncertainty due to measurement noise and epistemic uncertainty arising from limited training data and structural bias are considered. The resulting model preserves the LPV structure required for controller synthesis while enabling computationally efficient simulation and uncertainty propagation. The approach is demonstrated on the surrogate modelling of a two-dimensional nonlinear interconnection of mass-spring-damper systems.
E. Javier Olucha, Amritam Das, Roland Tóth
Control Systems Group, Eindhoven University of Technology, The Netherlands · Systems and Control Laboratory, HUN-REN Institute for Computer Science and Control, Hungary
Deep learning has proven highly effective for nonlinear system identification, but heavily parameterized neural networks are prone to overfitting in low-data regimes and lack reliable uncertainty quantification. The recently developed manifold meta-learning framework addresses the data efficiency problem by restricting the model parameters to a meta-learned low-dimensional manifold. However, that method is purely deterministic. We propose a fully probabilistic extension of the manifold meta-learning framework, based on amortized Variational Inference, where a generative prior over the low-dimensional parameter manifold is learned. During task-specific adaptation, we combine Maximum A Posteriori estimation with the Laplace approximation to yield a mathematically grounded posterior approximation. Evaluated on a static regression task and the Bouc--Wen dynamical system benchmark, the proposed approach achieves predictive accuracy comparable to its deterministic counterpart while successfully providing calibrated uncertainty bounds in severely low-data regimes.
Matteo Rufolo, Dario Piga, Marco Forgione
SUPSI-DTI-IDSIA, Dalle Molle Institute for Artificial Intelligence, Lugano, Switzerland.