SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration

Dongyi Fan; Suqiong Zhang; Lili He; Ming Liu; Yifan Huo

cs.CLMar 28, 2026

SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration

Authors: Dongyi Fan, Suqiong Zhang, Lili He, Ming Liu, Yifan Huo

Organizations: Zhejiang Sci-Tech University Hangzhou, Zhejiang, China

Abstract

Log parsing is a critical step for automated log analysis in complex systems. Traditional heuristic-based methods offer high efficiency but are limited in accuracy due to overlooking semantic context. In contrast, recent LLM-based parsers improve accuracy via se mantic understanding but incur high latency from frequent model calls. To address this, we propose SCOPE, the first self-correcting online log parsing method that integrates the strengths of both heuristic and LLM-based paradigms. SCOPE introduces a novel bi-directional tree structure that enables efficient template match ing from both forward and reverse directions, resulting in a higher overall matching rate. Additionally, it adopts a two-stage syntactic semantic collaboration framework: a lightweight NLP model first utilizes part-of-speech (POS) information for syntax-based match ing, while the LLM is selectively invoked as a fallback to handle semantically complex cases when uncertainty remains. This design significantly reduces LLM API usage while maintaining high ac curacy, achieving a balance between efficiency and effectiveness. Extensive evaluations on diverse benchmark datasets show that SCOPE outperforms state-of-the-art methods in both accuracy and efficiency. The implementation and datasets are publicly released to facilitate further research.

Figures & tables

Figure 2. Example of log parsing in BDPT. ① A log template is constructed in the bi-directional parse tree. ② A new log is matched to the template in the reverse branch but is missing in the forward branch of the parse tree. ③ The log template is updated and the incorrect branch is obsoleted.

Category	Rules
Constant	• Domain-specific term (e.g., IPv4 ). • Modifier in a compound noun representing a fixed label (e.g., Failed ). • Subject in a subject–verb–object structure. • Tokens expressing opposing or discrete semantics (e.g., boot vs. shutdown ). • Singular/plural variants are not treated as equivalent (e.g., user vs. users ).
Variable	• Identifiers, IPs, timestamps, user names, and other data-like tokens. • Key–value patterns: retain the key and abstract the value (e.g., user root → user <*> ). • key:value or key=value forms: keep key, abstract value.

Table 1. Rules for token classification

Method	Metric	HDFS	Hadoop	Spark	Zookeeper	BGL	HPC	Thunderbird	Linux	HealthApp	Apache	Proxifier	OpenSSH	OpenStack	Mac	Avg.
Non-LLM Log Parsers
Drain	GA	0.998	0.948	0.920	0.967	0.963	0.887	0.957	0.422	0.901	1.000	0.765	0.789	0.224	0.814	0.825
	PA	0.999	0.613	0.398	0.799	0.479	0.662	0.180	0.217	0.375	0.978	0.704	0.594	0.105	0.392	0.535
	FGA	0.935	0.785	0.861	0.904	0.624	0.309	0.237	0.778	0.010	1.000	0.206	0.872	0.007	0.229	0.554
	FTA	0.609	0.384	0.412	0.614	0.193	0.152	0.071	0.259	0.004	0.517	0.176	0.487	0.002	0.069	0.282
Brain	GA	0.960	0.563	0.972	0.993	0.940	0.800	0.792	0.790	0.979	0.997	0.521	0.663	1.000	0.834	0.843

Table 2. Accuracy comparison with the state-of-the-art log parsers. The best results are in bold.

Figure 4. Efficiency comparison with baseline parsers. SCOPE achieves the highest efficiency, with BDPT (219.5 s), PTMP (125.5 s), and NLPE (138 s).

Method	Acc.	#Invocations	#Tokens	Cost
LILAC	0.85	279.7	82.9K	$0.052
LogBatcher	0.83	258.2	65.8K	$0.038
LUNAR	0.88	268.5	142.8K	$0.084
SCOPE	0.91	41.7	98.4K	$0.062

Table 3. Statistics of LLM invocations and tokens cost

Comp.	#Log Match	#NLPE Inv.	#Templates Extr.
BDPT	95%	64%	Fwd: 46% Rev: 23%
PTMP	5%	36%	31%

Table 4. SCOPE Log parsing statistics by component

	GA	PA	LLM calls
Full SCOPE	0.968	0.904	41.7
w/o NLPE	0.834 ( $\downarrow$ 15.8%)	0.780 ( $\downarrow$ 15.0%)	0
-w/o LLM	0.901 ( $\downarrow$ 6.7%)	0.833 ( $\downarrow$ 7.1%)	0
-w/o POS	0.968 ( $\downarrow$ 0.0%)	0.903 ( $\downarrow$ 0.1%)	63.8 ( $\uparrow$ 53.1%)
w/o PTMP	0.966 ( $\downarrow$ 2.5%)	0.903 ( $\downarrow$ 1.6%)	35.0 ( $\downarrow$ 16.3%)
w/o BDPT	0.967 ( $\downarrow$ 0.1%)	0.890 ( $\downarrow$ 1.4%)	40.7 ( $\downarrow$ 3.1%)

Table 5. SCOPE Perfermance under different settings

Model	Parameters	GA	PA	FGA	FTA
Qwen3	235B	0.968	0.904	0.935	0.834
	32B	0.966	0.902	0.928	0.828
	8B	0.952	0.882	0.915	0.814
GPT-4o	˜200B	0.968	0.902	0.935	0.830
Claude-3.5	˜175B	0.965	0.896	0.930	0.826

Table 6. Configuration study with different LLM models

Model	Parameters	w/ CoT	w/o CoT
Qwen3	235B	0.91	0.86
	32B	0.90	0.82
	8B	0.89	0.78

Table 7. Accuracy under different prompt strategies

LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics
Apr 14, 2026Disha PatelAnomaly Detection Online Anomaly Detection
Department of Computer Science, California State University, Fullerton
Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems
Apr 6, 2026Ahmad Maroof Karimi, Jong Youl Choi, Charles Qing Cao +1Document Parsing LLM Information Extraction
Oak Ridge National Laboratory, Oak Ridge, USA · University of Tennessee, Knoxville, USA
SpikeLogBERT: Energy-Efficient Log Parsing Using Spiking Transformer Networks
Jun 30, 2026Thuan Bui, Duong Do, Tung Vu +2Efficient Neural Network Inference Spiking Transformers
Swinburne Vietnam, FPT University Hanoi, Vietnam · Posts and Telecommunications Institute of Technology Hanoi, Vietnam · Academy of Cryptography Techniques Hanoi, Vietnam +1

Apr 14, 2026J/K move · Enter open · S save

LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics

Disha Patel

Department of Computer Science, California State University, Fullerton

Anomaly Detection Online Anomaly Detection Few-Shot Anomaly Detection LLM Evaluation Zero-Shot Anomaly Detection

Open paper AskPDF arXiv

System log anomaly detection is critical for maintaining the reliability of large-scale software systems, yet traditional methods struggle with the heterogeneous and evolving nature of modern log data. Recent advances in Large Language Models (LLMs) offer promising new approaches to log understanding, but a systematic comparison of LLM-based methods against established techniques remains lacking. In this paper, we present a comprehensive benchmark study evaluating both LLM-based and traditional approaches for log anomaly detection across four widely-used public datasets: HDFS, BGL, Thunderbird, and Spirit. We evaluate three categories of methods: (1) classical log parsers (Drain, Spell, AEL) combined with machine learning classifiers, (2) fine-tuned transformer models (BERT, RoBERTa), and (3) prompt-based LLM approaches (GPT-3.5, GPT-4, LLaMA-3) in zero-shot and few-shot settings. Our experiments reveal that while fine-tuned transformers achieve the highest F1-scores (0.96-0.99), prompt-based LLMs demonstrate remarkablezero-shot capabilities (F1: 0.82-0.91) without requiring any labeled training data -- a significant advantage for real-world deployment where labeled anomalies are scarce. We further analyze the cost-accuracy trade-offs, latency characteristics, and failure modes of each approach. Our findings provide actionable guidelines for practitioners choosing log anomaly detection methods based on their specific constraints regarding accuracy, latency, cost, and label availability. All code and experimental configurations are publicly available to facilitate reproducibility.

Abstract

Explore similar work

LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnostics

Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems

SpikeLogBERT: Energy-Efficient Log Parsing Using Spiking Transformer Networks