A Novel GPT-Based Framework for Anomaly Detection in System Logs

Zeng Zhang; Wenjie Yin; Xiaoqi Li

cs.CROct 16, 2025

A Novel GPT-Based Framework for Anomaly Detection in System Logs

Authors: Zeng Zhang, Wenjie Yin, Xiaoqi Li

Organizations: Hainan University Haikou, China

Abstract

Identification of anomalous events within system logs constitutes a pivotal element within the frame- work of cybersecurity defense strategies. However, this process faces numerous challenges, including the management of substantial data volumes, the distribution of anomalies, and the precision of con- ventional methods. To address this issue, the present paper puts forward a proposal for an intelligent detection method for system logs based on Genera- tive Pre-trained Transformers (GPT). The efficacy of this approach is attributable to a combination of structured input design and a Focal Loss op- timization strategy, which collectively result in a substantial enhancement of the performance of log anomaly detection. The initial approach involves the conversion of raw logs into event ID sequences through the use of the Drain parser. Subsequently, the Focal Loss loss function is employed to address the issue of class imbalance. The experimental re- sults demonstrate that the optimized GPT-2 model significantly outperforms the unoptimized model in a range of key metrics, including precision, recall, and F1 score. In specific tasks, comparable or superior performance has been demonstrated to that of the GPT-3.5 API.

Figures & tables

Attribute	Value	Description
Data Source	LogHub	Standard HDFS dataset
Dataset Type	System Logs	Large-scale operations
Normal Logs	97.07%	Majority samples
Anomalous Logs	2.93%	Minority samples
Sample Size	3,000 logs	Stratified sampling
Normal Samples	2,912 logs	Original proportion

TABLE I: HDFS Log Dataset Characteristics

Metric	Formula	Purpose
Accuracy	$\frac{TP+TN}{Total}$	Overall correctness
Precision	$\frac{TP}{TP+FP}$	True anomaly rate
Recall	$\frac{TP}{TP+FN}$	Detection completeness
F1-Score	$\frac{2\cdot P\cdot R}{P+R}$	Harmonic mean
AUC	ROC Area	Class separation

TABLE II: Evaluation Metrics

Parameter	Value	Rationale
Learning Rate	$2\times 10^{-5}$	Fine-tuning optimum
Epochs	1	Prevent overfitting
Batch Size	8	Memory efficient
Optimizer	AdamW	Weight decay
Max Length	128 tokens	Context/efficiency
Grad. Accum.	4 steps	Larger batch sim.

TABLE III: Model Training Parameters

Fig. 2: Training Loss Curves for the Original and Optimised GPT-2 Models

Fig. 3: Comparative Analysis of Model Optimization Approaches

Model	Acc.	Prec.	Rec.	F1
GPT-2 (Original)	0.970	0.000	0.000	0.000
GPT-2 (Optimized)	0.990	1.000	0.667	0.800
GPT-3.5-turbo	0.487	0.493	0.900	0.637
GPT-4	0.500	0.500	1.000	0.667

TABLE IV: Model Performance Comparison (Compact)

Focal Loss

See all 8 papers

LogNEO: A GPT-Neo Reinforcement Learning Framework for Accurate Real-Time Log Anomaly Detection
Jun 6, 2026David Eje, Tanmay Sharma, Khush Patel +2RL for Language Models Online Anomaly Detection
Department of Computer Science, Innopolis University, Innopolis, Russia
LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics
Apr 14, 2026Disha PatelAnomaly Detection Online Anomaly Detection
Department of Computer Science, California State University, Fullerton
From Noise to Signal: Improving Security Log Anomaly Detection Using LLMs with Endpoint-Specific Logs
Aug 20, 2026Christopher Henshaw, Gour KarmakarAnomaly Detection LLMs for Cybersecurity
Institute of Innovation, Science and Sustainability, Federation University Australia · Institute of Innovation, Science and Sustainability, and Centre for Smart Analytics, Federation University Australia

Jun 6, 2026J/K move · Enter open · S save

LogNEO: A GPT-Neo Reinforcement Learning Framework for Accurate Real-Time Log Anomaly Detection

David Eje, Tanmay Sharma, Khush Patel, Manuel Mazzara+1

Department of Computer Science, Innopolis University, Innopolis, Russia

RL for Language Models Online Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

Detecting anomalies in large-scale system logs is critical for the reliability and security of modern computing infrastructure. We present LogNEO, a log anomaly detector built on EleutherAI's GPT-Neo (1.3B parameters) and fine-tuned with a novel partial-credit, exponentially decaying position-aware reward scheme combined with cross-entropy regularisation via Proximal Policy Optimisation (PPO). The position-aware reward explicitly models prediction difficulty: early positions receive higher rewards for correct predictions, while later positions incur stronger penalties for errors. LogNEO attains F1-scores of 0.927, 0.913, and 0.984 on the HDFS, BGL, and Thunderbird benchmarks, improving recall by up to 6 percentage points over the prior state-of-the-art LogGPT while maintaining comparable precision. A production microservice deployment over Apache Kafka, Redis, and TensorRT-accelerated inference demonstrates 45 ms end-to-end latency at 15,000 events per second.

Abstract

Focal Loss

Explore similar work

LogNEO: A GPT-Neo Reinforcement Learning Framework for Accurate Real-Time Log Anomaly Detection

LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics

From Noise to Signal: Improving Security Log Anomaly Detection Using LLMs with Endpoint-Specific Logs