cs.CRMar 4, 2026

CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts

Authors: Max Landauer, Wolfgang Hotwagner, Thorina Boenke, Florian Skopik, Markus Wurzenberger

Organizations: Austrian Institute of Technology, Center for Digital Safety & Security, Vienna, 1210, Austria

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Log data are essential for intrusion detection and forensic investigations. However, manual log analysis is tedious due to high data volumes, heterogeneous event formats, and unstructured messages. Even though many automated methods for log analysis exist, they usually still rely on domain-specific configurations such as expert-defined detection rules, handcrafted log parsers, or manual feature-engineering. Crucially, the level of automation of conventional methods is limited due to their inability to semantically understand logs and explain their underlying causes. In contrast, Large Language Models enable domain- and format-agnostic interpretation of system logs and security alerts. Unfortunately, research on this topic remains challenging, because publicly available and labeled data sets covering a broad range of attack techniques are scarce. To address this gap, we introduce the Cyber Attack Manifestation Log Data Set (CAM-LDS), comprising seven attack scenarios that cover 81 distinct techniques across 13 tactics and collected from 18 distinct sources within a fully open-source and reproducible test environment. We extract log events that directly result from attack executions to facilitate analysis of manifestations concerning command observability, event frequencies, performance metrics, and intrusion detection alerts. We further present an illustrative case study utilizing an LLM to process the CAM-LDS. The results indicate that correct attack techniques are predicted perfectly for approximately one third of attack steps and adequately for another third, highlighting the potential of LLM-based log interpretation and utility of our data set.

Figures & tables

Figure 1: Number of MITRE ATT&CK tactics and techniques covered by the reviewed data sets. Asterisks indicate that tactics and technique counts are not explicitly stated in the paper and thus estimated based on attack descriptions.
Figure 2: Overview of our methodology. Dashed boxes indicate automated steps that require minimal manual intervention.
Figure 3: General network topology and scenario-specific deployments of components.
IDNameDescription#Variants#Sim.#Steps
1Video Server ExploitIn addition to some reconnaissance and scanning attacks, this scenario revolves around compromising the video server. A key element in this attack chain is the exploitation of the vulnerable video server software. Subsequently, the attack graph involves five variants that allow the attacker to escalate to root privileges as well as three variants that provide the attacker with persistent access; we simulate all combinations of these variants.6 (PVE) 3 (PST)1856
2Linux MalwareThe malware considered in this scenario are an implant that establishes reverse-shell connection as well as a rootkit that targets dynamic linking. The remainder of the scenario broadly covers discovery techniques and exfiltrates some data from the compromised machine.2 (IA)229
3Lateral MovementAfter logging into the firewall through brute-force password guessing, the attacker moves laterally to the file share in the LAN segment. This is achieved by injecting malicious code either into a shared service file, a software management tool, or a manipulated package. The scenario ends with several destructive activities on the target host, including data encryption and deletion. The entire scenario is implemented for both SSH and VNC connections.2 (IA) 3 (LAT)660
4Network AttackThis scenario aims to compromise the firewall of the network. A port knocking sequence is used to download and run implants, which in turn enable the attacker to modify firewall rules and reach even more hosts in the network.-122
5Network SniffingIn this short scenario, the attacker captures authentication tokens from network traffic and re-uses them to gain access to a system.-13
6Attack on ClientThis scenario considers attack cases where a human user is deceived into compromising their own computer, either by downloading and opening a malicious document, installing a fake browser plugin, or providing the attacker access via screen sharing and remote control software. Once inside the system, the attacker installs a backdoor for permanent access. The scenario also involves several activities for data collection, including a keylogger, a data exfiltration tool, and extraction of clipboard data.3 (IA) 2 (PST)560
Table 2: Overview of attack scenarios.
Figure 5: Attack paths of Scenario 1: Video Server Exploit.
Figure 6: Attack paths of Scenario 2: Linux Malware.
Figure 7: Attack paths of Scenario 3 Lateral Movement.
Figure 8: Attack paths of Scenario 4: Network Attack.
Figure 9: Attack path of Scenario 5: Network Sniffing.
Figure 10: Attack path of Scenario 6: Attack on Client.
Figure 11: Attack path of Scenario 7: Docker Attack.
Figure 12: Timelines of log event and alert occurrences. Shaded intervals indicate active attack steps.
TacticScenario 1Scenario 2Scenario 3Scenario 4Scenario 5Scenario 6Scenario 7Total (Distinct)MITRE ATT&CK Coverage
Reconnaissance4000004545.5%
Resource Development000000000.0%
Initial Access2021013436.4%
Execution2130033635.3%
Persistence73430441356.5%
Privilege Escalation7333032964.3%
Table 3: Number of attack techniques in scenarios
Figure 13: Observability of attacks and commands executed by the attacker in log data.
Figure 14: Observability of the attack command for credential dumping (T1003.008) across scenarios.
Figure 16: Log event frequencies observed in different log sources. The number in the tags reference the attack step of the respective scenario.
Figure 19: Timelines of system performance metrics collected from the video server in Scenario 1 (Autostart, Local Account).
Figure 22: Severities of intrusion detection alerts across scenarios and attack steps.
Figure 24: Samples of correct LLM-based classification and interpretation of logs from attack steps.
Figure 25: Presence of ground-truth labels among the top-10 predicted attack techniques and estimation of benign versus malicious origin.
Figure 26: Contingency tables of LLM classification accuracies and attack manifestations.
More in this topic

Cybersecurity

Cybersecurity develops methods to protect digital and cyber-physical systems from attacks, vulnerabilities, data loss, and misuse.

See all 354 papers

Explore similar work

CardsList
Oct 22, 2026

SecLog-Bench: A Scalable Benchmark for Cybersecurity Log Reasoning with Scaffolded Evidence

Large language models (LLMs) have shown strong potential for cybersecurity, leading to growing interest in LLM-based automated cybersecurity log analysis. However, current datasets (1) are limited in scale; (2) rely mainly on single-source evidence; and (3) have only binary or multi-class attack labels, without dense annotations of the reasoning process. To address these limitations, SecLog-Bench has three key features. First, we build an extensible simulation platform that covers 29 attack scenarios, supports repeated execution and the integration of new scenarios, and enables scalable generation of cybersecurity logs with attack-stage labels. Second, we use multi-source logs, including audit logs, application logs, and network traffic, to jointly analyze cyber attacks. Third, we annotate each sample with the essential context for cybersecurity log reasoning and a structured step-by-step evidence chain. We evaluate eight mainstream LLMs on this benchmark for post-detection attack-progress reasoning. Results show that, when provided with scaffolded evidence, current LLMs exhibit modest capability in attack-progress reasoning, with the best model achieving only 44.44% accuracy and 45.07% Macro-F1. Their reasoning remains unreliable; i.e., the highest Stage Reasoning F1 (SRF1) score is only 54.87%. Moreover, even correct predictions may be accompanied by flawed reasoning, and reasoning quality is often misaligned with prediction accuracy.

Yifan Wang, Dongliang Xu, Xiaoyong Li +3

School of Airspace Science and Engineering, Shandong University, Shandong 264209, China. · School of Computer Science, University of Technology Sydney, Ultimo, NSW 2007, Australia. · an independent researcher in Braddon, ACT 2612, Australia.

CybersecurityLLMs for CybersecurityLanguage Model Reasoning Evaluation
AskPDF
Jun 9, 2026

Benchmarking and Exploring the Capabilities of LLMs for Attack Investigations

This paper presents AuditBench, a new benchmark dataset for evaluating the capabilities of LLMs at investigating security-related system audit logs. We design and use this benchmark to explore the performance of LLMs on four log-investigation tasks that incident response teams commonly perform, ranging from triaging alerts generated by detectors to identifying persistence mechanisms on compromised systems. AuditBench consists of system audit logs collected from Linux and Windows machines, and spans over 50 different security investigation scenarios, including both malicious and benign activity. Using our benchmark, we evaluate and analyze the performance of five frontier LLMs at analyzing audit logs for attack investigations. Our analysis illuminates how LLM performance and error profiles vary according to different design choices, such as differences in model size, data representation, prompt construction, and specific investigation tasks. Additionally, we characterize the quality of the explanations produced by LLMs and the types of errors that models make across our benchmark. Collectively, our work provides a foundation for assessing the capabilities of LLMs for investigating security logs, novel insights for practitioners using LLMs in security operations, and important directions for future research.

Aniket Anand, Yiwei Hou, Daniel Fields +4

University of Chicago · USA · University of California, Berkeley +1

LLM EvaluationCybersecurityDigital ForensicsLLMs for Cybersecurity
AskPDF
Mar 18, 2026

Retrieval-Augmented LLMs for Security Incident Analysis

Investigating cybersecurity incidents requires collecting and analyzing evidence from multiple log sources, including intrusion detection alerts, network traffic records, and authentication events. This process is labor-intensive: analysts must sift through large volumes of data to identify relevant indicators and piece together what happened. We present a RAG-based system that performs security incident analysis through targeted query-based filtering and LLM semantic reasoning. The system uses a query library with associated MITRE ATT&CK techniques to extract indicators from raw logs, then retrieves relevant context to answer forensic questions and reconstruct attack sequences. We evaluate the system with eight LLM configurations on malware traffic incidents and a multi-stage Active Directory attack. We find that LLMs have different performance and tradeoffs, with Claude Sonnet 4 achieving 94% and DeepSeek V3 achieving 89% average recall across 17 malware scenarios, while DeepSeek costs 15×\times less than Claude per analysis, and locally-deployed Llama 3.1:70b achieves 81% recall at zero per-query cost. Attack step detection on the Active Directory scenario reaches 100% precision and up to 96% recall with an enumeration prompt. These results demonstrate that combining targeted query-based filtering with RAG-based retrieval -- confirmed essential by ablation studies -- enables accurate, cost-effective security analysis within LLM context limits.

Xavier Cadet, Aditya Vikram Singh, Harsh Mamania +6

Dartmouth College Hanover, NH, USA · Northeastern University Boston, MA, USA · PUNCH Cyber Analytics Reston, VA, USA

CybersecurityDigital ForensicsRetrieval-Augmented Generation
AskPDF