Bayesian Tensor Autoencoder with Physics-informed Predictive Prior for Multi-dimensional Time Series Anomaly Detection

Jianan Liu; Chunguang Li

cs.LGSep 25, 2026

Bayesian Tensor Autoencoder with Physics-informed Predictive Prior for Multi-dimensional Time Series Anomaly Detection

Authors: Jianan Liu, Chunguang Li

Organizations: Zhejiang University, Hangzhou, China

Abstract

Multi-dimensional time series, inherently tensorial, are common in practice. Despite great progress in time series anomaly detection, most existing methods are confined to uni-/multi-variate time series. When handling multi-dimensional time series using these methods, reshaping operations are required, which inevitably break the intrinsic correlations and thus lead to performance degradation. In uni-/multi-variate time series anomaly detection, AutoEncoders (AEs) are widely adopted and generally categorized into reconstruction-based and prediction-based AEs. The reconstruction-based AE utilizes the current observation for reconstruction, while the prediction-based AE utilizes the historical information to predict the current observation. Thus, the two AEs utilize different information. To bridge the gap between reconstruction-based and prediction-based AEs, so as to fully leverage the available information and thus further enhance performance, we propose a predictive prior and incorporate it into the reconstruction-based AE. It may not be very difficult to conceive this idea, but designing the predictive prior so that it can work for tensor anomaly detection is non-trivial. Specifically, to avoid breaking the intrinsic correlations within the multi-dimensional time series, we use the tensor AE as the backbone. To incorporate the predictive prior into the reconstruction-based AE, we propose a Bayesian fusion approach and our analysis reveals that this approach can enhance the modeling capability of the model for normal data. To mitigate the over-generalization problem of AE, we incorporate physical laws, i.e. tensor low-rank decomposition rules, into the neural networks in the predictive prior, leading to the Physics-informed Predictive Prior Tensor AE (PPPTAE) framework. Experimental results on real-world datasets demonstrate the effectiveness of the proposed method.

Figures & tables

Figure 1. A schematic illustration of the distributions

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})\cdot p(\mathcal{F}_{t}\mid\mathcal{F}_{(t-M):(t-1)}^{(i)})

and

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})

, where

\mathcal{X}_{t}^{(i)}\sim p(\mathcal{X}_{t})

is a training sample drawn from the data distribution. Since the mean

\mu_{\text{fused}}^{(i)}

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})\cdot p(\mathcal{F}_{t}\mid\mathcal{F}_{(t-M):(t-1)}^{(i)})

is close to the mean

\langle\mathcal{F}_{t}^{1}\rangle^{(i)}

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})

, the probability densities of the two distributions exhibit an approximately positive correlation. That is, when the probability density of

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})\cdot p(\mathcal{F}_{t}\mid\mathcal{F}_{(t-M):(t-1)}^{(i)})

is large,

p(\mathcal{F}_{t}\mid\mathcal{X}_{t}^{(i)})

is roughly also large, and vice versa. A schematic plot comparing two related probability density functions over feature values, illustrating that their densities are approximately positively correlated.

Figure 2. The graphical model of the Bayesian tensor decomposition model. A probabilistic graphical model showing the dependencies among tensor factors, core tensor, sparse component, noise precision, and observed feature tensor in the Bayesian tensor decomposition model.

Item	Expression and Interpretation
$L_{\mathcal{F}}$	$L_{\mathcal{F}}=\frac{1}{2}\langle\tau\rangle\left\\|\mathcal{F}-\mathrm{TW}[\{\mathcal{G}^{(k)}\}_{k\in[3]};\mathcal{C}]-\mathcal{S}\right\\|^{2}_{F}$
$L_{\mathcal{F}}$	Ensure the consistency between feature $\mathcal{F}$ and the tensor decomposition results, $\mathrm{TW}[\{\mathcal{G}^{(k)}\}_{k\in[3]};\mathcal{C}]$ and $\mathcal{S}$ .
$L_{\langle\mathcal{G}^{(k)}\rangle}$	$L_{\langle\mathcal{G}^{(k)}\rangle}=\frac{1}{2}\sum_{r_{k}=1}^{R_{k}}\sum_{r_{k+1}=1}^{R_{k+1}}\sum_{l_{k}=1}^{L_{k}}\sum_{i_{k}=1}^{I_{k}}\langle\lambda_{r_{k}}^{(k)}\rangle\langle\lambda_{r_{k+1}}^{(k+1)}\rangle\langle\lambda_{l_{k}}^{(k)}\rangle\left\langle\mathcal{G}^{(k)}(r_{k},r_{k+1},l_{k},i_{k})\right\rangle^{2}$
$L_{\langle\mathcal{G}^{(k)}\rangle}$	Adapt to the corresponding elements of the TW rank.
$L_{\sigma_{g^{k}}^{2}}$	$L_{\sigma_{g^{k}}^{2}}=\frac{1}{2}\sum_{r_{k}=1}^{R_{k}}\sum_{r_{k+1}=1}^{R_{k+1}}\sum_{l_{k}=1}^{L_{k}}\sum_{i_{k}=1}^{I_{k}}(\langle\lambda_{r_{k}}^{(k)}\rangle\langle\lambda_{r_{k+1}}^{(k+1)}\rangle\langle\lambda_{l_{k}}^{(k)}\rangle\sigma_{g^{k}_{r_{k},r_{k+1},l_{k},i_{k}}}^{2}-\ln\sigma_{g^{k}_{r_{k},r_{k+1},l_{k},i_{k}}}^{2})$
$L_{\sigma_{g^{k}}^{2}}$	Prevent the variance from shrinking to zero, so as to prevent $q(\mathcal{G}^{(k)})$ from degenerating into a one-point distribution.

Table 1. The expressions and interpretations for items in the VB loss function

L_{\text{VB}}

( 36 )

Figure 3. The architecture of the Bayesian tensor decomposition module. Given a feature $\mathcal{F}$ , we first use networks to estimate the mean $\langle\mathcal{G}_{t}^{(k)}\rangle$ and variance $\sigma_{\mathcal{G}_{t}^{(k)}}^{2}$ of the TW ring factors $\mathcal{G}_{t}^{(k)}$ , $k\in[3]$ , the mean $\langle\mathcal{C}_{t}\rangle$ and variance $\sigma_{\mathcal{C}_{t}}^{2}$ of the TW core factor $\mathcal{C}_{t}$ , and the mean $\langle\mathcal{S}_{t}\rangle$ and variance $\sigma_{\mathcal{S}_{t}}^{2}$ of the sparse component $\mathcal{S}_{t}$ . Then, we apply the reparameterization trick to draw samples from these distributions. Subsequently, we compute the low-rank component $\mathcal{L}$ w.r.t the sampled ring factors $\{\mathcal{G}_{t}^{(k)}\}_{k\in[3]}$ and the sampled core factor $\mathcal{C}_{t}$ via the TW operation ( 18 ). Meanwhile, we can compute $\langle\boldsymbol{\lambda}_{R}^{(k)}\rangle$ , $\langle\boldsymbol{\lambda}_{L}^{(k)}\rangle$ , $\langle\tau\rangle$ and $\langle\beta\rangle$ , in closed-form via ( 70 ), ( 72 ), ( 77 ), and ( 81 ) in Appendix C , respectively. Then, the VB loss function ( 36 ) can be computed, so as to train the neural networks $f_{\mathcal{G}^{(k)}}(\cdot)$ , $f_{\mathcal{C}}(\cdot)$ , and $f_{\mathcal{S}}(\cdot)$ . This training process can be jointly performed with that of the reconstruction-based AE (see Section 3.1 ). A flow diagram of the Bayesian tensor decomposition module, where neural networks estimate tensor factor distributions, samples are drawn by reparameterization, a low-rank component is computed by the tensor wheel operation, and a variational Bayes loss is formed.

Figure 4. The schematic feature distribution of normal and abnormal samples after applying the low-rank constraint. A key function of TW decomposition is its ability to perform dimensionality reduction for each tensor order, which compresses the normal features into a “smaller” space. For illustrative purposes, the figure presents a simplified scenario where the tensor order is 1. In this scenario, 3-dimensional normal features are compressed into a 2-dimensional plane. Abnormal features, which are typically non-low-rank (or sparse), do not conform to this low-rank subspace and thus deviate from the normal features. A schematic three-dimensional feature space in which normal samples lie near a low-rank two-dimensional plane while abnormal samples deviate from that plane.

Figure 5. The architecture of PPPTAE, which consists of an encoder, a decoder, and a physics-informed predictive prior. For an observation $\mathcal{X}_{t}$ at a specific timestamp $t$ , the overall forward process begins with the encoder, which infers the distribution $p(\mathcal{F}_{t}|\mathcal{X}_{t})$ as defined in ( 8 ). Meanwhile, the physics-informed predictive prior predicts the distribution $p(\mathcal{F}_{t}|\mathcal{F}_{(t-M):(t-1)})$ as defined in ( 9 ). These two outputs are then fused using the Bayesian fusion approach detailed in Section 2.2 to produce $p(\mathcal{F}_{t}|\mathcal{X}_{t},\mathcal{F}_{(t-M):(t-1)})$ . Following this, feature samples are approximately drawn from $p(\mathcal{F}_{t}|\mathcal{X}_{t},\mathcal{F}_{(t-M):(t-1)})$ (as presented in Section 3.1 ). These samples are subsequently processed by the Bayesian Tensor Decomposition (TD) module to estimate the TW ring factors $\{\mathcal{G}_{t}^{(k)}\}_{k\in[3]}$ and the TW core factor $\mathcal{C}_{t}$ . Finally, the low-rank component $\mathcal{L}_{t}$ is calculated through the TW operation and then passed to the decoder for reconstruction. A block diagram of PPPTAE showing the encoder, physics-informed predictive prior, Bayesian fusion, feature sampling, Bayesian tensor decomposition, tensor wheel operation, and decoder used to reconstruct the current observation.

Encoder			Decoder
Stage	Layer	Output Size	Stage	Layer	Output Size
Input	-	$C\times N_{1}\times N_{2}$	Input	Sampled $\mathcal{F}$	$256\times N_{1}^{\prime\prime}\times N_{2}^{\prime\prime}$
Stage 1	$\mathcal{F}_{conv}(C,64)$	$64\times N_{1}\times N_{2}$	Stage 1	$\mathcal{F}_{conv}(256,256)$	$256\times N_{1}^{\prime\prime}\times N_{2}^{\prime\prime}$
Stage 1	MaxPool ( $\psi(\cdot)$ )	$64\times N_{1}^{\prime}\times N_{2}^{\prime}$		Bilinear( $N_{1}^{\prime},N_{2}^{\prime}$ )	$256\times N_{1}^{\prime}\times N_{2}^{\prime}$
Stage 2	$\mathcal{F}_{conv}(64,128)$	$128\times N_{1}^{\prime}\times N_{2}^{\prime}$		Conv $3\times 3$ , ReLU	$128\times N_{1}^{\prime}\times N_{2}^{\prime}$
Stage 2	MaxPool ( $\psi(\cdot)$ )	$128\times N_{1}^{\prime\prime}\times N_{2}^{\prime\prime}$	Stage 2	$\mathcal{F}_{conv}(128,128)$	$128\times N_{1}^{\prime}\times N_{2}^{\prime}$

Table 2. The Architecture of the Encoder and Decoder of PPPTAE

$f_{\mathcal{G}^{(1)}}$		$f_{\mathcal{G}^{(2)}}$
Layer	Output Size	Layer	Output Size
Trilinear( $8,8,256$ )	$1\times 8\times 8\times 256$	Trilinear( $8,8,N_{1}^{\prime\prime}$ )	$1\times 8\times 8\times N_{1}^{\prime\prime}$
Conv $3\times 3\times 3$ , BN, LReLU	$4\times 8\times 8\times 256$	Conv $3\times 3\times 3$ , BN, LReLU	$4\times 8\times 8\times N_{1}^{\prime\prime}$
Conv $3\times 3\times 3$ , BN	$16\times 8\times 8\times 256$	Conv $3\times 3\times 3$ , BN	$16\times 8\times 8\times N_{1}^{\prime\prime}$
$f_{\mathcal{G}^{(3)}}$		$f_{\mathcal{C}}$
Trilinear( $8,8,N_{2}^{\prime\prime}$ )	$1\times 8\times 8\times N_{2}^{\prime\prime}$	Bilinear( $8,8$ )	$256\times 8\times 8^{\prime}$

Table 3. The Architecture of the Networks in the Bayesian Tensor Decomposition Module

$\text{NN}_{\mathcal{G}^{(1)}}$		$\text{NN}_{\mathcal{G}^{(2)}}$
Layer	Output Size	Layer	Output Size
Conv $3\times 3\times 3$ , BN, LReLU	$16\times 8\times 8\times 256$	Conv $3\times 3\times 3$ , BN, LReLU	$16\times 8\times 8\times N_{1}^{\prime\prime}$
Conv $3\times 3\times 3$ , BN	$16\times 8\times 8\times 256$	Conv $3\times 3\times 3$ , BN	$16\times 8\times 8\times N_{1}^{\prime\prime}$
$\text{NN}_{\mathcal{G}^{(3)}}$		$\text{NN}_{\mathcal{C}}$
Conv $3\times 3\times 3$ , BN, LReLU	$16\times 8\times 8\times N_{2}^{\prime\prime}$	Conv $3\times 3$ , BN, LReLU	$16\times 8\times 8^{\prime}$
Conv $3\times 3\times 3$ , BN	$16\times 8\times 8\times N_{2}^{\prime\prime}$	Conv $3\times 3$ , BN	$16\times 8\times 8^{\prime}$

Table 4. The Architecture of the Networks in the Neural Network (NN) Module

Method	Venue	Freeway	HR-Extreme_S1	HR-Extreme_S2	HR-Extreme_S3	HR-Extreme_Y	Milan
Reformer ( Kitaev et al., 2020 )	ICLR-20	75.24%	69.92%	84.52%	72.79%	78.86%	69.40%
AnoTrans ( Xu et al., 2022a )	ICLR-21	72.71%	71.51%	72.81%	76.46%	71.75%	68.43%
Autoformer ( Wu et al., 2021 )	NeurIPS-21	69.15%	72.66%	70.70%	70.23%	70.53%	83.87%
Informer ( Zhou et al., 2021 )	AAAI-21	75.22%	71.90%	84.00%	78.98%	80.88%	79.01%
TFAD ( Zhang et al., 2022 )	CIKM-22	71.74%	70.19%	70.21%	80.44%	71.48%	80.36%
Fedformer ( Zhou et al., 2022 )	ICML-22	69.44%	78.80%	74.04%	84.79%	78.72%	74.05%

Table 5. AUC of Time Series Anomaly Detection Methods

Index	AE_r	AE_p	BayesLRM	Predictive Prior	Freeway	HR-Extreme_S1	HR-Extreme_S2	HR-Extreme_S3	HR-Extreme_Y	Milan
1	$\checkmark$				75.09%	96.63%	85.58%	87.93%	91.09%	90.12%
2	$\checkmark$		$\checkmark$		88.80%	96.92%	89.58%	90.23%	91.33%	95.12%
3	$\checkmark$			$\checkmark$	86.20%	97.41%	86.35%	91.45%	92.03%	94.28%
4		$\checkmark$			83.37%	86.31%	87.35%	92.87%	90.58%	93.83%
5*	$\checkmark$		$\checkmark$	$\checkmark$	95.39%	98.82%	92.21%	95.28%	94.65%	98.90%

Table 6. Ablation Results of PPPTAE

Figure 6. The MSE loss of AE_r, AE_p, and PPPTAE on the Freeway dataset. A line chart comparing the mean squared error training losses of AE-r, AE-p, and PPPTAE over epochs on the Freeway dataset.

Figure 7. The feature visualization of (a) AE_r and (b) PPPTAE on the training set of the Freeway dataset. Two t-SNE feature visualizations for AE-r and PPPTAE on the Freeway training set, with colors indicating reconstruction errors.

Explore similar work

Jun 18, 2026

PaAno+: Multiscale Encoding and Cross-Variable Attention for Time Series Anomaly Detection

Time-series anomaly detection has significant practical value for industrial and medical monitoring, as well as other critical domains. Current Transformer- and large-model-based detection approaches incur excessive computational overhead, while existing lightweight alternatives are constrained by insufficient feature extraction and inadequate modeling of dependencies across multivariate variables. To mitigate the above drawbacks, this study develops a lightweight, efficient anomaly detection model, dubbed PaAno, within the patch-oriented representation learning paradigm. In the encoder module, a multiscale feature-extraction backbone is constructed using convolutional kernels with differentiated receptive fields to capture hierarchical temporal characteristics; subsequent cross-scale adaptive attention aggregation, combined with residual connection optimization, further stabilizes feature representation learning. A cross-variable fusion attention module is embedded to explicitly characterize inter-variable correlations, empowering the model to identify anomalous patterns amid intricate operational conditions. Moreover, a novel pretext task based on temporal patch-window sorting is customized to uncover intrinsic structural properties of time series, and triplet loss is leveraged to optimize the patch embedding space for enhanced feature discrimination. Extensive experiments on the TSB-AD benchmark demonstrate that the proposed PaAno achieves state-of-the-art detection accuracy on both univariate and multivariate tasks, yielding significant performance gains across evaluation metrics, including VUS-PR, relative to the original PaAno. Leveraging a compact network design, the presented model achieves favorable computational efficiency, enabling deployment on resource-limited terminals for real-time anomaly inference.

Youji Zhu, Hongbing Wang, Wenchao Liu +2

School of Mathematical Sciences, Guizhou Normal University, Guiyang 550025, China. · School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China.

Self-Supervised Time Series Representation Learning Time-Series Anomaly Detection Multiscale Time-Series Modeling

Ask PDF

Oct 11, 2026

Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection

Reconstruction-based detectors for multivariate time series anomaly detection typically learn a single entangled representation of normal behavior and flag anomalies based on reconstruction error. However, real-world anomalies are not only isolated spikes in a single channel but also disruptions in the relationships between channels, where each sensor individually appears normal, but their joint behavior deviates from the norm. Because these models compress both temporal evolution and inter-channel structure into this single space, they lack an explicit mechanism to isolate such inter-channel dependencies from normal temporal dependencies, leading to lower detection rates for cross-channel anomalies. We propose CARMA (Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection), which is a reconstruction-based variational autoencoder with two complementary latent spaces: a temporal latent that captures sequential dynamics, and a correlation latent that is trained to reconstruct the window's pairwise Pearson correlations. Both representations are fused in the signal decoder, and anomalies are detected from the reconstruction error. We evaluate CARMA on the TSB-AD-M benchmark suite (17 datasets, 180 series) against 29 baseline methods, and show that it significantly outperforms all of them across multiple threshold-free and range-aware metrics, achieving a VUS-PR of 0.4929. Ablations show that removing the correlation branch degrades performance, that adding a second latent without supervision recovers only part of the gain, and that supervising the correlation latent with an explicit correlation target accounts for the rest and produces a sharper error response at anomalies.

Md. Johir Raihan, Zahra Zamanzadeh Darban, Yueyang Liu +3

Monash University, Melbourne, Victoria, Australia · University of New South Wales, Sydney, New South Wales, Australia

Time-Series Anomaly Detection Anomaly Detection Reconstruction-Based Anomaly Detection Variational Autoencoders

Ask PDF

Mar 10, 2026

Temporal-Conditioned Normalizing Flows for Multivariate Time Series Anomaly Detection

This paper introduces temporal-conditioned normalizing flows (tcNF), a novel framework that addresses anomaly detection in time series data with accurate modeling of temporal dependencies and uncertainty. By conditioning normalizing flows on previous observations, tcNF effectively captures complex temporal dynamics and generates accurate probability distributions of expected behavior. This autoregressive approach enables robust anomaly detection by identifying low-probability events within the learned distribution. We evaluate tcNF on diverse datasets, demonstrating good accuracy and robustness compared to existing methods. A comprehensive analysis of strengths and limitations and open-source code is provided to facilitate reproducibility and future research.

David Baumgartner, Helge Langseth, Kenth Engø-Monsen +1

Department of Computer Science, Norwegian University of Science and Technology.

Time-Series Anomaly Detection Normalizing Flows

Ask PDF