Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection

Kadir-Kaan Özer; René Ebeling; Markus Enzweiler

cs.LGMar 13, 2026

Surprised by Attention: Predictable Query Dynamics for Time Series Anomaly Detection

Authors: Kadir-Kaan Özer, René Ebeling, Markus Enzweiler

Organizations: Mercedes-Benz AG, Germany · Institute for Intelligent Systems, Esslingen University of Applied Sciences, Germany

Abstract

Multivariate time series anomalies often manifest as shifts in cross-channel dependencies rather than simple amplitude excursions. In autonomous driving, for instance, a steering command might be internally consistent but decouple from the resulting lateral acceleration. Residual-based detectors can miss such anomalies when flexible sequence models still reconstruct signals plausibly despite altered coordination. We introduce AxonAD, an unsupervised detector that treats multi-head attention query evolution as a short horizon predictable process. A gradient-updated reconstruction pathway is coupled with a history-only predictor that forecasts future query vectors from past context. This is trained via a masked predictor-target objective against an exponential moving average (EMA) target encoder. At inference, reconstruction error is combined with a tail-aggregated query mismatch score, which measures cosine deviation between predicted and target queries on recent timesteps. This dual approach provides sensitivity to structural dependency shifts while retaining amplitude-level detection. On proprietary in-vehicle telemetry with interval annotations and on the TSB-AD multi-variate suite (17 datasets, 180 series) with threshold-free and range-aware metrics, AxonAD improves ranking quality and temporal localization over strong baselines. Ablations confirm that query prediction and combined scoring are the primary drivers of the observed gains. Code is available at the URL https://github.com/iis-esslingen/AxonAD.

Figures & tables

Panel of Figure 1 — Figure 1: Query predictability in 2D query space (schematic, single head). Gray : past query trajectory. Blue dashed : predicted query. Red : EMA target query. (a) Nominal: predictor and target agree. (b) Coordination anomaly: the target query diverges from the predicted trajectory, producing large $d_{q}$ even when per-channel amplitudes are within normal bounds.

Figure 2: AxonAD overview. The online reconstruction encoder computes self attention using queries

\mathbf{Q}_{\mathrm{rec}}

. In parallel, a history-only predictor forecasts

\widehat{\mathbf{Q}}_{\mathrm{pred}}

and is trained to match EMA target queries

\mathbf{Q}_{\mathrm{tgt}}

(stop-gradient). Query mismatch on the last valid timesteps yields

d_{q}

, and reconstruction yields

d_{\mathrm{rec}}

. Attention divergence (KL tail) is not included in the default scoring pipeline.

Panel of Figure 3 — Figure 3: Score complementarity (schematic). Nominal windows spread along both axes but cluster near the origin in both scores simultaneously. Near-boundary amplitude and coordination anomalies are moderate on both axes, falling inside both single-axis thresholds (dotted lines) but separated by the additive $S$ (dashed diagonal).

Panel of Figure 4 — Figure 4: Chronological split and anomaly onset for the proprietary telemetry stream.

Model	Proprietary Telemetry (19 channels, 80,000 timesteps)
Model	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	PA-F1	Event-F1	Range-F1	Affiliation-F1
LSTMAD [ 24 ]	0.082 $\pm$ 0.004	0.651 $\pm$ 0.009	0.083 $\pm$ 0.004	0.624 $\pm$ 0.009	0.533 $\pm$ 0.014	0.255 $\pm$ 0.015	0.139 $\pm$ 0.006	0.723 $\pm$ 0.003
SISVAE [ 18 ]	0.128 $\pm$ 0.030	0.586 $\pm$ 0.026	0.070 $\pm$ 0.012	0.504 $\pm$ 0.052	0.270 $\pm$ 0.100	0.231 $\pm$ 0.060	0.225 $\pm$ 0.054	0.699 $\pm$ 0.018
TFTResidual [ 20 ]	0.071 $\pm$ 0.006	0.644 $\pm$ 0.025	0.070 $\pm$ 0.005	0.582 $\pm$ 0.018	0.424 $\pm$ 0.022	0.164 $\pm$ 0.019	0.110 $\pm$ 0.009	0.752 $\pm$ 0.026
VSVAE [ 26 ]	0.100 $\pm$ 0.005	0.617 $\pm$ 0.031	0.065 $\pm$ 0.005	0.535 $\pm$ 0.027	0.214 $\pm$ 0.048	0.188 $\pm$ 0.004	0.262 $\pm$ 0.037	0.730 $\pm$ 0.012
M2N2 [ 1 ]	0.065 $\pm$ 0.001	0.596 $\pm$ 0.001	0.064 $\pm$ 0.001	0.553 $\pm$ 0.004	0.392 $\pm$ 0.022	0.196 $\pm$ 0.020	0.120 $\pm$ 0.009	0.680 $\pm$ 0.000

Table 1: Proprietary telemetry results (TSB-AD evaluation suite). Mean

\pm

std over four seeds. Best per metric in bold .

Model	TSB-AD (multivariate, 17 datasets, 180 time series)
Model	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	PA-F1	Event-F1	Range-F1	Affiliation-F1
VASP [ 31 ]	0.339 $\pm$ 0.319	0.762 $\pm$ 0.195	0.401 $\pm$ 0.338	0.809 $\pm$ 0.185	0.669 $\pm$ 0.318	0.520 $\pm$ 0.361	0.400 $\pm$ 0.260	0.849 $\pm$ 0.123
OmniAnomaly [ 28 ]	0.372 $\pm$ 0.341	0.744 $\pm$ 0.250	0.424 $\pm$ 0.354	0.777 $\pm$ 0.240	0.627 $\pm$ 0.354	0.528 $\pm$ 0.367	0.432 $\pm$ 0.292	0.841 $\pm$ 0.126
WVAE [ 27 ]	0.354 $\pm$ 0.331	0.747 $\pm$ 0.248	0.413 $\pm$ 0.349	0.778 $\pm$ 0.248	0.576 $\pm$ 0.388	0.502 $\pm$ 0.383	0.365 $\pm$ 0.280	0.838 $\pm$ 0.137
USAD [ 3 ]	0.363 $\pm$ 0.339	0.738 $\pm$ 0.256	0.412 $\pm$ 0.350	0.771 $\pm$ 0.244	0.622 $\pm$ 0.355	0.519 $\pm$ 0.364	0.422 $\pm$ 0.288	0.837 $\pm$ 0.131
SISVAE [ 18 ]	0.323 $\pm$ 0.290	0.759 $\pm$ 0.234	0.372 $\pm$ 0.315	0.786 $\pm$ 0.227	0.551 $\pm$ 0.367	0.470 $\pm$ 0.355	0.369 $\pm$ 0.278	0.824 $\pm$ 0.129

Table 2: TSB-AD multivariate benchmark (17 datasets, 180 series). Mean

\pm

std over all series. Best per metric in bold .

Panel of Figure 5 — Figure 5: Paired AUC-PR comparison on TSB-AD multivariate ( $n=180$ ). Left: win/loss counts. Right: median paired difference with lollipop connectors from zero. All paired Wilcoxon tests yield $p<10^{-4}$ with entirely negative 95% bootstrap CIs (full statistics in the Appendix).

Variant	TSB-AD (multivariate, ablation subset)
Variant	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	PA-F1	Event-F1	Range-F1	Affiliation-F1
Base	0.558 $\pm$ 0.285	0.861 $\pm$ 0.137	0.658 $\pm$ 0.301	0.915 $\pm$ 0.102	0.855 $\pm$ 0.248	0.773 $\pm$ 0.262	0.564 $\pm$ 0.263	0.904 $\pm$ 0.123
Recon only	0.511 $\pm$ 0.330	0.820 $\pm$ 0.218	0.603 $\pm$ 0.366	0.858 $\pm$ 0.223	0.728 $\pm$ 0.325	0.656 $\pm$ 0.322	0.541 $\pm$ 0.303	0.856 $\pm$ 0.142
Score MSE	0.513 $\pm$ 0.327	0.828 $\pm$ 0.204	0.604 $\pm$ 0.365	0.868 $\pm$ 0.208	0.730 $\pm$ 0.317	0.652 $\pm$ 0.311	0.534 $\pm$ 0.293	0.852 $\pm$ 0.139
JEPA only, Q	0.413 $\pm$ 0.317	0.764 $\pm$ 0.200	0.533 $\pm$ 0.353	0.846 $\pm$ 0.177	0.822 $\pm$ 0.283	0.683 $\pm$ 0.321	0.396 $\pm$ 0.248	0.892 $\pm$ 0.114
Score MSE+JEPA KL	0.554 $\pm$ 0.285	0.860 $\pm$ 0.137	0.655 $\pm$ 0.300	0.916 $\pm$ 0.100	0.854 $\pm$ 0.248	0.772 $\pm$ 0.262	0.560 $\pm$ 0.266	0.896 $\pm$ 0.127

Table 3: AxonAD ablation on the TSB-AD multivariate tuning subset (20 series). Mean

\pm

std. Best per metric in bold . Rows are grouped to match the discussion order below.

Panel of Figure 6 — Figure 6: Sensitivity to forecast horizon $s$ (left) and tail aggregation length $k$ (right) on the tuning subset.

Panel of Figure 7 — Figure 7: Runtime on the telemetry stream (80,000 samples, stride 1). Left: wall-clock time split into fitting and scoring. Right: per window scoring latency. Classical baselines have no iterative training, so their time is entirely attributed to scoring.

Model	TSB-AD (multivariate, 17 datasets, 180 time series)
Model	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	PA-F1	Event-F1	Range-F1	Affiliation-F1
VASP	0.339 $\pm$ 0.319	0.762 $\pm$ 0.195	0.401 $\pm$ 0.338	0.809 $\pm$ 0.185	0.669 $\pm$ 0.318	0.520 $\pm$ 0.361	0.400 $\pm$ 0.260	0.849 $\pm$ 0.123
OmniAnomaly	0.372 $\pm$ 0.341	0.744 $\pm$ 0.250	0.424 $\pm$ 0.354	0.777 $\pm$ 0.240	0.627 $\pm$ 0.354	0.528 $\pm$ 0.367	0.432 $\pm$ 0.292	0.841 $\pm$ 0.126
WVAE	0.354 $\pm$ 0.331	0.747 $\pm$ 0.248	0.413 $\pm$ 0.349	0.778 $\pm$ 0.248	0.576 $\pm$ 0.388	0.502 $\pm$ 0.383	0.365 $\pm$ 0.280	0.838 $\pm$ 0.137
USAD	0.363 $\pm$ 0.339	0.738 $\pm$ 0.256	0.412 $\pm$ 0.350	0.771 $\pm$ 0.244	0.622 $\pm$ 0.355	0.519 $\pm$ 0.364	0.422 $\pm$ 0.288	0.837 $\pm$ 0.131
SISVAE	0.323 $\pm$ 0.290	0.759 $\pm$ 0.234	0.372 $\pm$ 0.315	0.786 $\pm$ 0.227	0.551 $\pm$ 0.367	0.470 $\pm$ 0.355	0.369 $\pm$ 0.278	0.824 $\pm$ 0.129

Table S4: TSB-AD multivariate benchmark (17 datasets, 180 time series). Mean

\pm

standard deviation over all evaluated series. Best result per metric in bold .

Model	TSB-AD (Proprietary Dataset)
Model	AUC-PR	AUC-ROC	VUS-PR	VUS-ROC	PA-F1	Event-F1	Range-F1	Affiliation-F1
LSTMAD	0.082 $\pm$ 0.004	0.651 $\pm$ 0.009	0.083 $\pm$ 0.004	0.624 $\pm$ 0.009	0.533 $\pm$ 0.014	0.255 $\pm$ 0.015	0.139 $\pm$ 0.006	0.723 $\pm$ 0.003
SISVAE	0.128 $\pm$ 0.030	0.586 $\pm$ 0.026	0.070 $\pm$ 0.012	0.504 $\pm$ 0.052	0.270 $\pm$ 0.100	0.231 $\pm$ 0.060	0.225 $\pm$ 0.054	0.699 $\pm$ 0.018
TFTResidual	0.071 $\pm$ 0.006	0.644 $\pm$ 0.025	0.070 $\pm$ 0.005	0.582 $\pm$ 0.018	0.424 $\pm$ 0.022	0.164 $\pm$ 0.019	0.110 $\pm$ 0.009	0.752 $\pm$ 0.026
RobustPCA	0.070 $\pm$ 0.000	0.634 $\pm$ 0.000	0.066 $\pm$ 0.000	0.570 $\pm$ 0.000	0.312 $\pm$ 0.000	0.162 $\pm$ 0.000	0.100 $\pm$ 0.000	0.680 $\pm$ 0.000
VSVAE	0.100 $\pm$ 0.005	0.617 $\pm$ 0.031	0.065 $\pm$ 0.005	0.535 $\pm$ 0.027	0.214 $\pm$ 0.048	0.188 $\pm$ 0.004	0.262 $\pm$ 0.037	0.730 $\pm$ 0.012

Table S5: TSB-AD multivariate benchmark protocol on proprietary dataset. Mean

\pm

standard deviation over 4 random seeds. Best result per metric in bold .

Model	AUC-PR (vs AxonAD)					AUC-ROC (vs AxonAD)
Model	WR	$\boldsymbol{\Delta}$ mean	$\boldsymbol{\Delta}$ med	$p$	CI 95	WR	$\boldsymbol{\Delta}$ mean	$\boldsymbol{\Delta}$ med	$p$	CI 95
AnomalyTransformer	0.039	-0.3693	-0.3203	$1.24\!\times\!10^{-30}$	[-0.4124, -0.3255]	0.033	-0.3185	-0.3520	$1.14\!\times\!10^{-29}$	[-0.3432, -0.2919]
AutoEncoder	0.194	-0.1427	-0.1462	$1.93\!\times\!10^{-11}$	[-0.1969, -0.0882]	0.172	-0.1554	-0.1776	$2.33\!\times\!10^{-14}$	[-0.1916, -0.1224]
CBLOF	0.178	-0.1738	-0.1506	$2.19\!\times\!10^{-13}$	[-0.2287, -0.1185]	0.139	-0.1610	-0.1601	$6.06\!\times\!10^{-16}$	[-0.1949, -0.1268]
CNN	0.267	-0.0899	-0.0859	$1.36\!\times\!10^{-8}$	[-0.1388, -0.0393]	0.228	-0.0544	-0.0555	$2.24\!\times\!10^{-8}$	[-0.0829, -0.0268]
COPOD	0.194	-0.2324	-0.2315	$2.30\!\times\!10^{-14}$	[-0.2903, -0.1716]	0.133	-0.1730	-0.2094	$2.19\!\times\!10^{-16}$	[-0.2064, -0.1398]

Table S6: Pairwise comparison against AxonAD on TSB-AD multivariate (17 datasets, 180 time series). For each baseline and metric we report: win-rate (wins/180), mean and median performance delta (

\Delta

), Wilcoxon

p

-value, and 95% CI for

\Delta

. Negative

\Delta

indicates the baseline is worse than AxonAD (per your export). Entries with non-significant Wilcoxon test (

p\geq 0.05

) are bold .

Model	VUS-PR (vs AxonAD)					R-based-F1 (vs AxonAD)					Affiliation-F (vs AxonAD)
Model	WR	$\Delta$ mean	$\Delta$ med	$p$	CI 95	WR	$\Delta$ mean	$\Delta$ med	$p$	CI 95	WR	$\Delta$ mean	$\Delta$ med	$p$	CI 95
AnomalyTransformer	0.039	-0.3784	-0.2998	$1.65\!\times\!10^{-30}$	[-0.4217, -0.3367]	0.067	-0.3325	-0.3168	$2.57\!\times\!10^{-29}$	[-0.3678, -0.2964]	0.200	-0.1228	-0.0718	$3.64\!\times\!10^{-18}$	[-0.1478, -0.0993]
AutoEncoder	0.172	-0.1987	-0.1348	$1.88\!\times\!10^{-16}$	[-0.2422, -0.1567]	0.189	-0.1882	-0.2248	$2.51\!\times\!10^{-13}$	[-0.2396, -0.1352]	0.333	-0.0596	-0.0207	$6.96\!\times\!10^{-7}$	[-0.0831, -0.0354]
CBLOF	0.161	-0.2334	-0.1601	$1.49\!\times\!10^{-19}$	[-0.2776, -0.1924]	0.200	-0.1692	-0.2151	$1.54\!\times\!10^{-12}$	[-0.2177, -0.1185]	0.361	-0.0487	-0.0081	$8.71\!\times\!10^{-5}$	[-0.0752, -0.0225]
CNN	0.278	-0.1408	-0.0769	$9.59\!\times\!10^{-12}$	[-0.1805, -0.1039]	0.217	-0.1694	-0.1844	$3.38\!\times\!10^{-13}$	[-0.2190, -0.1180]	0.483	0.0062	-0.0002	0.676	[-0.0127, 0.0264]
COPOD	0.150	-0.2900	-0.2262	$3.03\!\times\!10^{-22}$	[-0.3360, -0.2450]	0.183	-0.2287	-0.2535	$6.43\!\times\!10^{-15}$	[-0.2815, -0.1741]	0.333	-0.0609	-0.0538	$2.21\!\times\!10^{-7}$	[-0.0853, -0.0363]

Table S7: Continuation of Table S6 : VUS-PR, R-based-F1, and Affiliation-F1 vs AxonAD. Missing metrics in your export (VUS-ROC, PA-F1, Event-F1, Range-F1) are not shown.

Explore similar work

Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection
Oct 11, 2026Md. Johir Raihan, Zahra Zamanzadeh Darban, Yueyang Liu +3Time-Series Anomaly Detection Anomaly Detection
Monash University, Melbourne, Victoria, Australia · University of New South Wales, Sydney, New South Wales, Australia
No Scale Left Behind: Multi-Scale Autoencoder with Bi-directional Attention for Time Series Anomaly Detection
Sep 29, 2026Jiaheng Guo, Haochen Zhang, Yu-Chao Huang +3Reconstruction-Based Anomaly Detection Attention Mechanisms
UNITES Lab, University of North Carolina at Chapel Hill
PaAno+: Multiscale Encoding and Cross-Variable Attention for Time Series Anomaly Detection
Jun 18, 2026Youji Zhu, Hongbing Wang, Wenchao Liu +2Self-Supervised Time Series Representation Learning Time-Series Anomaly Detection
School of Mathematical Sciences, Guizhou Normal University, Guiyang 550025, China. · School of Big Data and Computer Science, Guizhou Normal University, Guiyang 550025, China.

Oct 11, 2026J/K move · Enter open · S save

Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection

Md. Johir Raihan, Zahra Zamanzadeh Darban, Yueyang Liu, Flora D. Salim+2

Monash University, Melbourne, Victoria, Australia · University of New South Wales, Sydney, New South Wales, Australia

Time-Series Anomaly Detection Anomaly Detection Reconstruction-Based Anomaly Detection Variational Autoencoders

Open paper AskPDF arXiv

Reconstruction-based detectors for multivariate time series anomaly detection typically learn a single entangled representation of normal behavior and flag anomalies based on reconstruction error. However, real-world anomalies are not only isolated spikes in a single channel but also disruptions in the relationships between channels, where each sensor individually appears normal, but their joint behavior deviates from the norm. Because these models compress both temporal evolution and inter-channel structure into this single space, they lack an explicit mechanism to isolate such inter-channel dependencies from normal temporal dependencies, leading to lower detection rates for cross-channel anomalies. We propose CARMA (Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection), which is a reconstruction-based variational autoencoder with two complementary latent spaces: a temporal latent that captures sequential dynamics, and a correlation latent that is trained to reconstruct the window's pairwise Pearson correlations. Both representations are fused in the signal decoder, and anomalies are detected from the reconstruction error. We evaluate CARMA on the TSB-AD-M benchmark suite (17 datasets, 180 series) against 29 baseline methods, and show that it significantly outperforms all of them across multiple threshold-free and range-aware metrics, achieving a VUS-PR of 0.4929. Ablations show that removing the correlation branch degrades performance, that adding a second latent without supervision recovers only part of the gain, and that supervising the correlation latent with an explicit correlation target accounts for the rest and produces a sharper error response at anomalies.

Abstract

Explore similar work

Correlation-Aware Reconstruction for Multivariate Time Series Anomaly Detection

No Scale Left Behind: Multi-Scale Autoencoder with Bi-directional Attention for Time Series Anomaly Detection

PaAno+: Multiscale Encoding and Cross-Variable Attention for Time Series Anomaly Detection