Benchmarking Deep Learning Models for Anomaly Detection in Real-World Cloud Telemetry

Sumit Dhyani; Sougata Mukherjea

cs.LGOct 11, 2026

Benchmarking Deep Learning Models for Anomaly Detection in Real-World Cloud Telemetry

Authors: Sumit Dhyani, Sougata Mukherjea

Organizations: Electrical Engineering Department, Indian Institute of Technology, Delhi, India

Abstract

Modern cloud platforms generate high-dimensional, noisy telemetry from thousands of microservices. Unlike classical anomaly detection benchmarks, cloud anomalies are short-lived, irregular, and largely non-periodic, making timely detection challenging. We present an empirical benchmark of deep learning models for anomaly detection using a real-world IBM Cloud Console dataset. We evaluate GRU, Transformer, TimesNet, and Autoformer architectures using the Numenta Anomaly Benchmark (NAB), a timing-aware metric that accounts for early detection and false alarms. TimesNet achieves the highest NAB scores due to strong early-detection sensitivity. Transformer models also detect anomalies early but incur higher false-positive rates. An improved GRU autoencoder, incorporating robust scoring and persistence rules, achieves competitive performance with a more stable false-alarm profile. Overall, the modest performance differences suggest that real cloud anomalies are dominated by short-term deviations rather than persistent periodic structure, emphasizing the importance of timing-aware evaluation and robust post-processing.

Figures & tables

Panel of Figure 1 — Fig. 1 : Improved GRU autoencoder results on the IBM Cloud Console dataset (Data from Mar 12–13, 2024).

Panel of Figure 2 — Fig. 2 : Baseline Transformer anomaly detection results (Data from Mar 12–13, 2024).

Panel of Figure 3 — Fig. 3 : TimesNet behavior on non-seasonal cloud telemetry (Data from Mar 12–13, 2024).

Model	NAB (Standard)	NAB (Low-FN)
GRU (Baseline [ 2 ] )	$17.00\pm 0.42$	$18.74\pm 0.45$
GRU (Improved)	$21.27\pm 0.36$	$23.44\pm 0.39$
Anomaly Transformer (Baseline)	$26.52\pm 0.71$	$28.79\pm 0.68$
Anomaly Transformer (Improved)	$26.68\pm 0.63$	$28.89\pm 0.61$
TimesNet	$29.05\pm 0.28$	$30.48\pm 0.24$
Autoformer	$26.56\pm 0.66$	$28.82\pm 0.64$

TABLE I : Mean and standard deviation of NAB scores over multiple runs on the IBM Cloud Console dataset.

Model	TP	FP	FN	TN
GRU (Baseline)	4	121	14	25451
GRU (Improved)	5	66	13	25548
Transformer (Baseline)	6	163	12	25199
Transformer (Improved)	6	155	12	25180
TimesNet	6	110	12	25469
Autoformer	6	151	12	25215

TABLE II : Window-based confusion matrix statistics for anomaly detection.

Model	TP	FP	FN	TN
GRU (Baseline)	8	163	865	25451
GRU (Improved)	5	66	868	25548
Transformer (Baseline)	25	320	844	25199
Transformer (Improved)	26	339	843	25180
TimesNet	8	114	865	25469
Autoformer	26	304	843	25215

TABLE III : Per-timestamp confusion matrix statistics for different models.

Fig. 4 : Normalized NAB score as a function of the anomaly detection threshold. The selected operating threshold lies within a stable region where performance remains high across small threshold variations.

Fig. 5 : False positive event count as a function of the detection threshold. Lower thresholds significantly increase the number of spurious alerts, while the selected operating point maintains a low false positive rate.

Configuration	TP	FP	FN	TN
Baseline (no improvements)	10	250	863	25364
Log-MSE only	10	250	863	25364
Robust likelihood only	125	3318	748	22296
Persistence only	2	47	871	25567
All components enabled	5	66	868	25548

TABLE IV : Per-timestamp confusion matrix statistics for different GRU ablation configurations.

Configuration	NAB (Standard)	NAB (Low-FN)
Baseline (no improvements)	17.03	18.76
Log-MSE only	17.03	18.76
Robust likelihood only	41.24	44.16
Persistence only	8.48	9.36
All components enabled	21.27	23.44

TABLE V : Window-based normalized NAB scores for different GRU ablation configurations.

Explore similar work

ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring
Jul 20, 2026Thu T. H. Doan, Mohammad Saiful Islam, Andriy Miranskyy +3Cloud Computing Time-Series Anomaly Detection
Department of Computer Science, Gran Sasso Science Institute, L’Aquila, Italy · Department of Computer Science, Toronto Metropolitan University, Toronto, Canada · Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences, Bergen, Norway +1
Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability
Oct 1, 2026Roberto Stanzione, Jules Barbe, Magali Parrino +2Interpretable Anomaly Detection Anomaly Localization
Inria, ENS, CNRS, PSL Paris, France · Scality Paris, France · Inria, ENS, CNRS, PSL, EDF Paris, France
CARE: A Cascaded Framework for Efficient and Reliable Time Series Anomaly Detection
Aug 3, 2026Zemin Chao, Qianhui Xu, Jianhe Cen +3Efficient Neural Network Inference Cost-Aware Inference
Massive Data Computing Lab, Harbin Institute of Technology

Jul 20, 2026J/K move · Enter open · S save

ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring

Thu T. H. Doan, Mohammad Saiful Islam, Andriy Miranskyy, Ngoc-Thanh Nguyen+2

Department of Computer Science, Gran Sasso Science Institute, L’Aquila, Italy · Department of Computer Science, Toronto Metropolitan University, Toronto, Canada · Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences, Bergen, Norway+1

Cloud Computing Time-Series Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

With the rapid growth of cloud computing infrastructures in scale and complexity, network monitoring for Large-scale Cloud Systems (LCSs) has become increasingly challenging, requiring automated and reliable anomaly detection to maintain service availability. Modern LCSs continuously generate telemetry logs from distributed cloud services, producing high-dimensional multivariate time series that capture system operations. Detecting anomalies in this context is difficult due to extreme dimensionality, complex dependencies among distributed components, and severe sparsity from intermittently active services. Taking these challenges into account, we first conduct an empirical study on telemetry logs from the IBM Cloud Console platform, and then propose ClouDens, an anomaly detection framework tailored to LCS monitoring that leverages operational-context attributes encoded in the telemetry log schema to improve detection accuracy and early identification of anomalies. ClouDens partitions high-dimensional telemetry logs into domain-guided subsets, constructs a context-aware graph modeling operational service dependencies, and employs Spatio-Temporal Graph Neural Networks for forecasting-based anomaly detection. We evaluate ClouDens on the recently released IBM Cloud Telemetry Dataset and provide practical insights into designing reliable anomaly detection solutions for LCS monitoring. Results show ClouDens achieves higher NAB scores in count-based telemetry features, indicating more accurate, earlier anomaly detection with broader coverage than a GRU-based model. Our study further reveals that telemetry feature subsets, operational-context modeling, scoring strategies, and sparsity imputation all substantially influence detection performance, offering practical guidance for designing and fairly benchmarking anomaly detection approaches for LCS monitoring.

Abstract

Explore similar work

ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring

Detect, Explain, Interpret: An End-to-End Benchmark for Time Series Anomaly Detection, Explainability and Interpretability

CARE: A Cascaded Framework for Efficient and Reliable Time Series Anomaly Detection