LogCDM: Interpretable Log Anomaly Detection via Behavioral Graph Drift and Adaptive Signal Fusion
Organizations: Purdue University Fort Wayne
Abstract
Log-based anomaly detection is increasingly dominated by large pretrained models, from BERT-scale encoders to 8B-parameter language models, which require GPUs at inference time and are difficult to deploy on the monitoring nodes where logs are produced. We present LogCDM, a compact detector that scores each log session using three complementary signals: the next-event surprise of a small causal Transformer, the distance between a graph-attention embedding of the session's own transition graph and that of a normal reference graph, and a transition-drift statistic that we show is exactly a frequency-weighted chi-squared divergence between transition kernels. The divergence view yields an asymptotic null distribution, a length-standardized score, and a per-edge decomposition that explains each alarm. A sigmoid gate fuses the three signals; under a shared-covariance Gaussian model, this gate is the Bayes posterior, and its learned weights indicate which signal each dataset relies on. With 8.1-8.8M parameters and no GPU required at inference, LogCDM reaches F1 scores of 0.984 on HDFS, 0.870 on chronologically split BGL, and 0.850 on Thunderbird, within 0.013-0.12 of an 8.1B-parameter LLM detector evaluated under the same protocol and above all compact baselines on BGL. It scores a session in 5.5-10.5 ms on a CPU. Ablations show that the structural signals add 0.025-0.030 F1 on BGL and Thunderbird but almost nothing on HDFS, consistent with the learned fusion weights.
Figures & tables
| HDFS | BGL | Thunderbird | |
| Sessions / windows | 575,061 | 47,100 | 100,000 |
| Anomaly rate (%) | 2.93 | 9.5 | 1.0 |
| Train / val / test | 402.5k / 57.5k / 115.0k | 33.0k / 4.7k / 9.4k | 70.0k / 10.0k / 20.0k |
| Test anomalies | 3,367 | 800 | – |
| Templates (fit on train) | 39 | 1,300 | 1,076 |
| Test OOV template rate (%) | 0 | 3 | 4 |
| HDFS | BGL | Thunderbird | |||
|---|---|---|---|---|---|
| Metric | Random | Chrono. | Chrono. | Random | Chrono. |
| Precision | 0.985 | 0.935 | 0.875 | 0.965 | 0.825 |
| Recall | 0.992 | 0.955 | 0.915 | 0.975 | 0.900 |
| F1 | 0.984 0.003 | 0.930 0.010 | 0.870 0.018 | 0.965 0.006 | 0.850 0.028 |
| F1 95% CI half-width | 0.003 | 0.008 | 0.030 | 0.008 | 0.075 |
| AUC-ROC | 0.9992 | 0.983 | 0.970 | 0.995 | 0.970 |
| HDFS (random) | BGL (chronological) | Thunderbird (chronological) | |||||||
| Method | P | R | F1 | P | R | F1 | P | R | F1 |
| DeepLog | 0.835 | 0.994 | 0.908 | 0.166 | 0.988 | 0.285 | 0.017 | 0.966 | 0.033 |
| LogBERT | 0.989 | 0.614 | 0.758 | 0.165 | 0.989 | 0.283 | 0.022 | 0.172 | 0.039 |
| LogRobust | 0.961 | 1.000 | 0.980 | 0.696 | 0.968 | 0.810 | 0.318 | 1.000 | 0.482 |
| CNN | 0.966 | 1.000 | 0.982 | 0.698 | 0.965 | 0.810 | 0.870 | 0.690 | 0.769 |
| NeuralLog | 0.971 | 0.988 | 0.979 | 0.792 | 0.884 | 0.835 | 0.794 | 0.931 | 0.857 |
| Method | Parameters | HDFS | BGL | Thunderbird |
|---|---|---|---|---|
| LogLLM † ( Guan et al., 2024 ) | 8.1B | 0.997 | 0.916 | 0.966 |
| BGE-Trans † ( Guan et al., 2024 ) | not reported | 0.981 | 0.823 | 0.867 |
| LogLLaMA † ( Yang and Harris, 2025 ) | not reported | 0.968 | 0.326 | 0.127 |
| LogMamba ‡ ( Hu et al., 2026 ) | not reported | 0.837 | 0.987 | 0.983 |
| LogFormer ‡ ( Guo et al., 2024 ) | 7.2M trainable | 0.98 | 0.97 | 0.99 |
| LogDA ‡ ( Zhang and Fu, 2025 ) | not reported | 0.99 | 0.95 | 0.95 |
| HDFS | BGL | Thunderbird | |
| Total parameters | 8,109,667 | 8,766,648 | 8,649,944 |
| Model file (MiB, fp32) | 30.9 | 33.4 | 33.0 |
| Peak inference RAM (MB) | 275 | 275 | 275 |
| CPU latency p50 / p95, batch 1 (ms) | 5.5 / 7 | 10.5 / 13 | 10.5 / 13 |
| CPU throughput, batch 64 (sessions/s) | 1,050 | 625 | 625 |
| GPU throughput, batch 1,024 (sessions/s) | 40,000 | 27,500 | 27,500 |
| Variant | HDFS | BGL | Thunderbird |
|---|---|---|---|
| Full LogCDM | 0.984 | 0.870 | 0.850 |
| Phase 1 only (unsupervised ) | 0.860 | 0.390 | – |
| Transformer + BCE, no structural signals | 0.983 | 0.845 | 0.820 |
| STDS only (no neural model) | 0.800 | 0.550 | 0.600 |
| No adapter | 0.983 | 0.870 | – |
| No | 0.984 | 0.860 | – |
| HDFS | BGL | Thunderbird | |
|---|---|---|---|
| (sequence ) | 3.25 | 1.50 | 1.70 |
| (session graph ) | 1.00 | 2.75 | 2.25 |
| (standardized STDS ) | 1.00 | 1.75 | 1.75 |
| 2.00 | 1.50 | 2.00 | |
| , | 4.00 | 1.50 | 2.00 |
| , | 1.50 | 2.75 | 2.25 |
| Condition | HDFS | BGL |
|---|---|---|
| Default (15% fusion set, clean labels) | 0.984 | 0.870 |
| 1% label noise in fusion set | 0.955 | 0.825 |
| 5% label noise in fusion set | 0.910 | 0.775 |
| Fusion set = 1% of training data | 0.965 | 0.775 |
| Fusion set = 5% of training data | 0.980 | 0.840 |
| Expected calibration error | 0.020 | 0.055 |
| Rank | Transition | Share of | ||
|---|---|---|---|---|
| 1 | Error template same error template (repeat burst) | 0.55 | 0.005 | 30% |
| 2 | Normal template error template | 0.30 | 0.005 | 17.5% |
| 3 | Error template recovery/info template | 0.20 | 0.03 | 12.5% |
| 4 | Info template info template (suppressed) | 0.125 | 0.45 | 7.5% |
| 5 | Unseen transition ( ) | 0.10 | 6.5% |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Component | Setting |
|---|---|
| Parser | Drain3, similarity threshold 0.5 (HDFS), 0.5 (BGL, Thunderbird), depth 4, max children 100 |
| Template encoder | all-MiniLM-L6-v2 (384-d), PCA or learned projection to |
| Transformer | 6 pre-norm layers, 8 heads, , , dropout 0.1, sinusoidal positions |
| GAT | 2 layers, edge-weighted attention, ELU, mean pooling, normalization |
| Adapter | rank , , initialized to zero |
| STDS | lag , smoothing , standardized as in Eq. equation 5 |
| Seed | HDFS (random) | BGL (chronological) | Thunderbird (chronological) |
|---|---|---|---|
| 42 | 0.986 | 0.884 | 0.881 |
| 123 | 0.981 | 0.851 | 0.812 |
| 7 | 0.988 | 0.892 | 0.874 |
| 2024 | 0.983 | 0.866 | 0.846 |
| 99 | 0.982 | 0.857 | 0.837 |
| Mean std | 0.984 0.003 | 0.870 0.018 | 0.850 0.028 |
Interpretable Anomaly Detection
Interpretable anomaly detection identifies unusual data and explains the features, patterns, or conditions behind each alert.
See all 84 papers