Log-based anomaly detection is increasingly dominated by large pretrained models, from BERT-scale encoders to 8B-parameter language models, which require GPUs at inference time and are difficult to deploy on the monitoring nodes where logs are produced. We present LogCDM, a compact detector that scores each log session using three complementary signals: the next-event surprise of a small causal Transformer, the distance between a graph-attention embedding of the session's own transition graph and that of a normal reference graph, and a transition-drift statistic that we show is exactly a frequency-weighted chi-squared divergence between transition kernels. The divergence view yields an asymptotic null distribution, a length-standardized score, and a per-edge decomposition that explains each alarm. A sigmoid gate fuses the three signals; under a shared-covariance Gaussian model, this gate is the Bayes posterior, and its learned weights indicate which signal each dataset relies on. With 8.1-8.8M parameters and no GPU required at inference, LogCDM reaches F1 scores of 0.984 on HDFS, 0.870 on chronologically split BGL, and 0.850 on Thunderbird, within 0.013-0.12 of an 8.1B-parameter LLM detector evaluated under the same protocol and above all compact baselines on BGL. It scores a session in 5.5-10.5 ms on a CPU. Ablations show that the structural signals add 0.025-0.030 F1 on BGL and Thunderbird but almost nothing on HDFS, consistent with the learned fusion weights.
Figures & tables
Figure 1: LogCDM overview. Grey boxes run once offline; blue boxes run per session on a CPU. The three scores are fused by a sigmoid gate, and the STDS term decomposes over transitions to explain each alarm.
HDFS
BGL
Thunderbird
Sessions / windows
575,061
47,100
100,000
Anomaly rate (%)
2.93
9.5
1.0
Train / val / test
402.5k / 57.5k / 115.0k
33.0k / 4.7k / 9.4k
70.0k / 10.0k / 20.0k
Test anomalies
3,367
800
–
Templates (fit on train)
39
1,300
1,076
Test OOV template rate (%)
0
3
4
Table 1: Dataset statistics after grouping and splitting.
HDFS
BGL
Thunderbird
Metric
Random
Chrono.
Chrono.
Random
Chrono.
Precision
0.985
0.935
0.875
0.965
0.825
Recall
0.992
0.955
0.915
0.975
0.900
F1
0.984 ± 0.003
0.930 ± 0.010
0.870 ± 0.018
0.965 ± 0.006
0.850 ± 0.028
F1 95% CI half-width
0.003
0.008
0.030
0.008
0.075
AUC-ROC
0.9992
0.983
0.970
0.995
0.970
Table 2: LogCDM across datasets and splits. All values are means over 5 seeds; F1 is the mean of the per-seed F1 scores ( ± std over seeds), so it need not equal 2PR/(P+R) of the mean precision and recall.
HDFS (random)
BGL (chronological)
Thunderbird (chronological)
Method
P
R
F1
P
R
F1
P
R
F1
DeepLog
0.835
0.994
0.908
0.166
0.988
0.285
0.017
0.966
0.033
LogBERT
0.989
0.614
0.758
0.165
0.989
0.283
0.022
0.172
0.039
LogRobust
0.961
1.000
0.980
0.696
0.968
0.810
0.318
1.000
0.482
CNN
0.966
1.000
0.982
0.698
0.965
0.810
0.870
0.690
0.769
NeuralLog
0.971
0.988
0.979
0.792
0.884
0.835
0.794
0.931
0.857
Table 3: Comparison under the same split. Baseline values are those reported by Guan et al. (2024) under this protocol; LogCDM values are means over 5 seeds (F1 averaged per seed). Best compact result in bold.
Method
Parameters
HDFS
BGL
Thunderbird
LogLLM † ( Guan et al., 2024 )
∼ 8.1B
0.997
0.916
0.966
BGE-Trans † ( Guan et al., 2024 )
not reported
0.981
0.823
0.867
LogLLaMA † ( Yang and Harris, 2025 )
not reported
0.968
0.326
0.127
LogMamba ‡ ( Hu et al., 2026 )
not reported
0.837
0.987
0.983
LogFormer ‡ ( Guo et al., 2024 )
7.2M trainable
0.98
0.97
0.99
LogDA ‡ ( Zhang and Fu, 2025 )
not reported
0.99
> 0.95
> 0.95
Table 4: F1 against published detectors. † : same protocol as ours; ‡ : different windows or splits, shown for reference only.
Figure 2: F1 under the same protocol. LogCDM (blue) matches compact baselines on HDFS and Thunderbird and leads them on BGL, while LogLLM remains ahead at about 1,000 × the size.
HDFS
BGL
Thunderbird
Total parameters
8,109,667
8,766,648
8,649,944
Model file (MiB, fp32)
30.9
33.4
33.0
Peak inference RAM (MB)
275
275
275
CPU latency p50 / p95, batch 1 (ms)
5.5 / 7
10.5 / 13
10.5 / 13
CPU throughput, batch 64 (sessions/s)
1,050
625
625
GPU throughput, batch 1,024 (sessions/s)
40,000
27,500
27,500
Table 5: Cost of LogCDM. Parameter counts use the vocabulary sizes of Table 1 . CPU: 16-core server CPU; GPU: A100 40GB.
Variant
HDFS
BGL
Thunderbird
Full LogCDM
0.984
0.870
0.850
Phase 1 only (unsupervised pe )
0.860
0.390
–
Transformer + BCE, no structural signals
0.983
0.845
0.820
STDS only (no neural model)
0.800
0.550
0.600
No adapter
0.983
0.870
–
No Lrec
0.984
0.860
–
Table 6: Ablations (F1, mean over 5 seeds). Every variant is retrained from scratch.
Figure 3: F1 change on BGL when removing or replacing one component. Structural signals and semantic embeddings account for the largest drops; the two variants not shown (Phase 1 only, STDS only) fall by more than 0.3.
HDFS
BGL
Thunderbird
wa (sequence pe )
3.25
1.50
1.70
wb (session graph sgraph )
1.00
2.75
2.25
wc (standardized STDS z )
1.00
1.75
1.75
b
− 2.00
− 1.50
− 2.00
Δμ/σ , pe
4.00
1.50
2.00
Δμ/σ , sgraph
1.50
2.75
2.25
Table 7: Learned fusion weights and validation class separation Δμ/σ of each normalized score (means over 5 seeds).
Condition
HDFS
BGL
Default (15% fusion set, clean labels)
0.984
0.870
1% label noise in fusion set
0.955
0.825
5% label noise in fusion set
0.910
0.775
Fusion set = 1% of training data
0.965
0.775
Fusion set = 5% of training data
0.980
0.840
Expected calibration error
0.020
0.055
Table 8: Sensitivity to label noise and labeled-set size (F1), and calibration (means over 5 seeds).
Rank
Transition u→ω
Pi(ω∣u)
P~ref(ω∣u)
Share of si2
1
Error template → same error template (repeat burst)
0.55
0.005
30%
2
Normal template → error template
0.30
0.005
17.5%
3
Error template → recovery/info template
0.20
0.03
12.5%
4
Info template → info template (suppressed)
0.125
0.45
7.5%
5
Unseen transition ( ∅ )
0.10
λ
6.5%
Table 9: Top-5 transitions by STDS contribution ci(u,ω) for one anomalous BGL test window (Theorem 1 e).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Setting
Parser
Drain3, similarity threshold 0.5 (HDFS), ≤ 0.5 (BGL, Thunderbird), depth 4, max children 100
Template encoder
all-MiniLM-L6-v2 (384-d), PCA or learned projection to d=256
2 layers, edge-weighted attention, ELU, mean pooling, ℓ2 normalization
Adapter
rank r=4 , β=0.1 , Q initialized to zero
STDS
lag L∈{2,3} , smoothing λ=10−3 , standardized as in Eq. equation 5
Appendix
Table 10: Hyperparameters, shared across datasets unless stated.
Seed
HDFS (random)
BGL (chronological)
Thunderbird (chronological)
42
0.986
0.884
0.881
123
0.981
0.851
0.812
7
0.988
0.892
0.874
2024
0.983
0.866
0.846
99
0.982
0.857
0.837
Mean ± std
0.984 ± 0.003
0.870 ± 0.018
0.850 ± 0.028
Appendix
Table 11: Test F1 per seed.
Figure 4: Training curves (mean over 5 seeds). Phase 1 loss plateaus by epoch 30–50 and rises slightly afterwards, so we early-stop on normal validation loss. Phase 2 validation F1 peaks at epochs 15–20.
System log anomaly detection is critical for maintaining the reliability of large-scale software systems, yet traditional methods struggle with the heterogeneous and evolving nature of modern log data. Recent advances in Large Language Models (LLMs) offer promising new approaches to log understanding, but a systematic comparison of LLM-based methods against established techniques remains lacking. In this paper, we present a comprehensive benchmark study evaluating both LLM-based and traditional approaches for log anomaly detection across four widely-used public datasets: HDFS, BGL, Thunderbird, and Spirit. We evaluate three categories of methods: (1) classical log parsers (Drain, Spell, AEL) combined with machine learning classifiers, (2) fine-tuned transformer models (BERT, RoBERTa), and (3) prompt-based LLM approaches (GPT-3.5, GPT-4, LLaMA-3) in zero-shot and few-shot settings. Our experiments reveal that while fine-tuned transformers achieve the highest F1-scores (0.96-0.99), prompt-based LLMs demonstrate remarkablezero-shot capabilities (F1: 0.82-0.91) without requiring any labeled training data -- a significant advantage for real-world deployment where labeled anomalies are scarce. We further analyze the cost-accuracy trade-offs, latency characteristics, and failure modes of each approach. Our findings provide actionable guidelines for practitioners choosing log anomaly detection methods based on their specific constraints regarding accuracy, latency, cost, and label availability. All code and experimental configurations are publicly available to facilitate reproducibility.
Disha Patel
Department of Computer Science, California State University, Fullerton
Detecting anomalies in large-scale system logs is critical for the reliability and security of modern computing infrastructure. We present LogNEO, a log anomaly detector built on EleutherAI's GPT-Neo (1.3B parameters) and fine-tuned with a novel partial-credit, exponentially decaying position-aware reward scheme combined with cross-entropy regularisation via Proximal Policy Optimisation (PPO). The position-aware reward explicitly models prediction difficulty: early positions receive higher rewards for correct predictions, while later positions incur stronger penalties for errors. LogNEO attains F1-scores of 0.927, 0.913, and 0.984 on the HDFS, BGL, and Thunderbird benchmarks, improving recall by up to 6 percentage points over the prior state-of-the-art LogGPT while maintaining comparable precision. A production microservice deployment over Apache Kafka, Redis, and TensorRT-accelerated inference demonstrates 45 ms end-to-end latency at 15,000 events per second.
David Eje, Tanmay Sharma, Khush Patel +2
Department of Computer Science, Innopolis University, Innopolis, Russia
The explosive growth of system logs makes streaming compression essential, yet existing log anomaly detection (LAD) methods incur severe pre-processing overhead by requiring full decompression and parsing. We introduce CLAD, the first deep learning framework to perform LAD directly on compressed byte streams. CLAD bypasses these bottlenecks by exploiting a key insight: normal logs compress into regular byte patterns, while anomalies systematically disrupt them. To extract these multi-scale deviations from opaque bytes, we propose a purpose-built architecture integrating a dilated convolutional byte encoder, a hybrid Transformer--mLSTM, and four-way aggregation pooling. This is coupled with a two-stage training strategy of masked pre-training and focal-contrastive fine-tuning to effectively handle severe class imbalance. Evaluated across five datasets, CLAD achieves a state-of-the-art average F1-score of 0.9909 and outperforms the best baseline by 2.72 percentage points. It delivers superior accuracy while completely eliminating decompression and parsing overheads, offering a robust solution that generalizes to structured streaming compressors.