Showing 1–8 of 8 results for author: Landauer, M

System Log Parsing with Large Language Models: A Review

Authors: Viktor Beck, Max Landauer, Markus Wurzenberger, Florian Skopik, Andreas Rauber

Abstract: Log data provides crucial insights for tasks like monitoring, root cause analysis, and anomaly detection. Due to the vast volume of logs, automated log parsing is essential to transform semi-structured log messages into structured representations. Recent advances in large language models (LLMs) have introduced the new research field of LLM-based log parsing. Despite promising results, there is no… ▽ More Log data provides crucial insights for tasks like monitoring, root cause analysis, and anomaly detection. Due to the vast volume of logs, automated log parsing is essential to transform semi-structured log messages into structured representations. Recent advances in large language models (LLMs) have introduced the new research field of LLM-based log parsing. Despite promising results, there is no structured overview of the approaches in this relatively new research field with the earliest advances published in late 2023. This work systematically reviews 29 LLM-based log parsing methods. We benchmark seven of them on public datasets and critically assess their comparability and the reproducibility of their reported results. Our findings summarize the advances of this new research field, with insights on how to report results, which data sets, metrics and which terminology to use, and which inconsistencies to avoid, with code and results made publicly available for transparency. △ Less

Submitted 15 May, 2025; v1 submitted 7 April, 2025; originally announced April 2025.

Comments: 36 pages, 11 figures

ACM Class: I.2; I.5

Trace of the Times: Rootkit Detection through Temporal Anomalies in Kernel Activity

Authors: Max Landauer, Leonhard Alton, Martina Lindorfer, Florian Skopik, Markus Wurzenberger, Wolfgang Hotwagner

Abstract: Kernel rootkits provide adversaries with permanent high-privileged access to compromised systems and are often a key element of sophisticated attack chains. At the same time, they enable stealthy operation and are thus difficult to detect. Thereby, they inject code into kernel functions to appear invisible to users, for example, by manipulating file enumerations. Existing detection approaches are… ▽ More Kernel rootkits provide adversaries with permanent high-privileged access to compromised systems and are often a key element of sophisticated attack chains. At the same time, they enable stealthy operation and are thus difficult to detect. Thereby, they inject code into kernel functions to appear invisible to users, for example, by manipulating file enumerations. Existing detection approaches are insufficient, because they rely on signatures that are unable to detect novel rootkits or require domain knowledge about the rootkits to be detected. To overcome this challenge, our approach leverages the fact that runtimes of kernel functions targeted by rootkits increase when additional code is executed. The framework outlined in this paper injects probes into the kernel to measure time stamps of functions within relevant system calls, computes distributions of function execution times, and uses statistical tests to detect time shifts. The evaluation of our open-source implementation on publicly available data sets indicates high detection accuracy with an F1 score of 98.7\% across five scenarios with varying system states. △ Less

Submitted 4 March, 2025; originally announced March 2025.

Towards Improving IDS Using CTF Events

Authors: Manuel Kern, Florian Skopik, Max Landauer, Edgar Weippl

Abstract: In cybersecurity, Intrusion Detection Systems (IDS) serve as a vital defensive layer against adversarial threats. Accurate benchmarking is critical to evaluate and improve IDS effectiveness, yet traditional methodologies face limitations due to their reliance on previously known attack signatures and lack of creativity of automated tests. This paper introduces a novel approach to evaluating IDS th… ▽ More In cybersecurity, Intrusion Detection Systems (IDS) serve as a vital defensive layer against adversarial threats. Accurate benchmarking is critical to evaluate and improve IDS effectiveness, yet traditional methodologies face limitations due to their reliance on previously known attack signatures and lack of creativity of automated tests. This paper introduces a novel approach to evaluating IDS through Capture the Flag (CTF) events, specifically designed to uncover weaknesses within IDS. CTFs, known for engaging a diverse community in tackling complex security challenges, offer a dynamic platform for this purpose. Our research investigates the effectiveness of using tailored CTF challenges to identify weaknesses in IDS by integrating them into live CTF competitions. This approach leverages the creativity and technical skills of the CTF community, enhancing both the benchmarking process and the participants' practical security skills. We present a methodology that supports the development of IDS-specific challenges, a scoring system that fosters learning and engagement, and the insights of running such a challenge in a real Jeopardy-style CTF event. Our findings highlight the potential of CTFs as a tool for IDS evaluation, demonstrating the ability to effectively expose vulnerabilities while also providing insights into necessary improvements for future implementations. △ Less

Submitted 20 January, 2025; originally announced January 2025.

Red Team Redemption: A Structured Comparison of Open-Source Tools for Adversary Emulation

Authors: Max Landauer, Klaus Mayer, Florian Skopik, Markus Wurzenberger, Manuel Kern

Abstract: Red teams simulate adversaries and conduct sophisticated attacks against defenders without informing them about used tactics in advance. These interactive cyber exercises are highly beneficial to assess and improve the security posture of organizations, detect vulnerabilities, and train employees. Unfortunately, they are also time-consuming and expensive, which often limits their scale or prevents… ▽ More Red teams simulate adversaries and conduct sophisticated attacks against defenders without informing them about used tactics in advance. These interactive cyber exercises are highly beneficial to assess and improve the security posture of organizations, detect vulnerabilities, and train employees. Unfortunately, they are also time-consuming and expensive, which often limits their scale or prevents them entirely. To address this situation, adversary emulation tools partially automate attacker behavior and enable fast, continuous, and repeatable security testing even when involved personnel lacks red teaming experience. Currently, a wide range of tools designed for specific use-cases and requirements exist. To obtain an overview of these solutions, we conduct a review and structured comparison of nine open-source adversary emulation tools. To this end, we assemble a questionnaire with 80 questions addressing relevant aspects, including setup, support, documentation, usability, and technical features. In addition, we conduct a user study with domain experts to investigate the importance of these aspects for distinct user roles. Based on the evaluation and user feedback, we rank the tools and find MITRE Caldera, Metasploit, and Atomic Red Team on top. △ Less

Submitted 28 August, 2024; originally announced August 2024.

A Critical Review of Common Log Data Sets Used for Evaluation of Sequence-based Anomaly Detection Techniques

Authors: Max Landauer, Florian Skopik, Markus Wurzenberger

Abstract: Log data store event execution patterns that correspond to underlying workflows of systems or applications. While most logs are informative, log data also include artifacts that indicate failures or incidents. Accordingly, log data are often used to evaluate anomaly detection techniques that aim to automatically disclose unexpected or otherwise relevant system behavior patterns. Recently, detectio… ▽ More Log data store event execution patterns that correspond to underlying workflows of systems or applications. While most logs are informative, log data also include artifacts that indicate failures or incidents. Accordingly, log data are often used to evaluate anomaly detection techniques that aim to automatically disclose unexpected or otherwise relevant system behavior patterns. Recently, detection approaches leveraging deep learning have increasingly focused on anomalies that manifest as changes of sequential patterns within otherwise normal event traces. Several publicly available data sets, such as HDFS, BGL, Thunderbird, OpenStack, and Hadoop, have since become standards for evaluating these anomaly detection techniques, however, the appropriateness of these data sets has not been closely investigated in the past. In this paper we therefore analyze six publicly available log data sets with focus on the manifestations of anomalies and simple techniques for their detection. Our findings suggest that most anomalies are not directly related to sequential manifestations and that advanced detection techniques are not required to achieve high detection rates on these data sets. △ Less

Submitted 6 September, 2023; originally announced September 2023.

Journal ref: Proceedings of the ACM on Software Engineering (FSE 2024)

Introducing a New Alert Data Set for Multi-Step Attack Analysis

Authors: Max Landauer, Florian Skopik, Markus Wurzenberger

Abstract: Intrusion detection systems (IDS) reinforce cyber defense by autonomously monitoring various data sources for traces of attacks. However, IDSs are also infamous for frequently raising false positives and alerts that are difficult to interpret without context. This results in high workloads on security operators who need to manually verify all reported alerts, often leading to fatigue and incorrect… ▽ More Intrusion detection systems (IDS) reinforce cyber defense by autonomously monitoring various data sources for traces of attacks. However, IDSs are also infamous for frequently raising false positives and alerts that are difficult to interpret without context. This results in high workloads on security operators who need to manually verify all reported alerts, often leading to fatigue and incorrect decisions. To generate more meaningful alerts and alleviate these issues, the research domain focused on multi-step attack analysis proposes approaches for filtering, clustering, and correlating IDS alerts, as well as generation of attack graphs. Unfortunately, existing data sets are outdated, unreliable, narrowly focused, or only suitable for IDS evaluation. Since hardly any suitable benchmark data sets are publicly available, researchers often resort to private data sets that prevent reproducibility of evaluations. We therefore generate a new alert data set that we publish alongside this paper. The data set contains alerts from three distinct IDSs monitoring eight executions of a multi-step attack as well as simulations of normal user behavior. To illustrate the potential of our data set, we experiment with alert prioritization as well as two open-source tools for meta-alert generation and attack graph extraction. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Journal ref: CSET 2024: Proceedings of the 17th Cyber Security Experimentation and Test Workshop

Deep Learning for Anomaly Detection in Log Data: A Survey

Authors: Max Landauer, Sebastian Onder, Florian Skopik, Markus Wurzenberger

Abstract: Automatic log file analysis enables early detection of relevant incidents such as system failures. In particular, self-learning anomaly detection techniques capture patterns in log data and subsequently report unexpected log event occurrences to system operators without the need to provide or manually model anomalous scenarios in advance. Recently, an increasing number of approaches leveraging dee… ▽ More Automatic log file analysis enables early detection of relevant incidents such as system failures. In particular, self-learning anomaly detection techniques capture patterns in log data and subsequently report unexpected log event occurrences to system operators without the need to provide or manually model anomalous scenarios in advance. Recently, an increasing number of approaches leveraging deep learning neural networks for this purpose have been presented. These approaches have demonstrated superior detection performance in comparison to conventional machine learning techniques and simultaneously resolve issues with unstable data formats. However, there exist many different architectures for deep learning and it is non-trivial to encode raw and unstructured log data to be analyzed by neural networks. We therefore carry out a systematic literature review that provides an overview of deployed models, data pre-processing mechanisms, anomaly detection techniques, and evaluations. The survey does not quantitatively compare existing approaches but instead aims to help readers understand relevant aspects of different model architectures and emphasizes open issues for future work. △ Less

Submitted 15 May, 2023; v1 submitted 8 July, 2022; originally announced July 2022.

Journal ref: Machine Learning with Applications, vol. 12 (2023)

Maintainable Log Datasets for Evaluation of Intrusion Detection Systems

Authors: Max Landauer, Florian Skopik, Maximilian Frank, Wolfgang Hotwagner, Markus Wurzenberger, Andreas Rauber

Abstract: Intrusion detection systems (IDS) monitor system logs and network traffic to recognize malicious activities in computer networks. Evaluating and comparing IDSs with respect to their detection accuracies is thereby essential for their selection in specific use-cases. Despite a great need, hardly any labeled intrusion detection datasets are publicly available. As a consequence, evaluations are often… ▽ More Intrusion detection systems (IDS) monitor system logs and network traffic to recognize malicious activities in computer networks. Evaluating and comparing IDSs with respect to their detection accuracies is thereby essential for their selection in specific use-cases. Despite a great need, hardly any labeled intrusion detection datasets are publicly available. As a consequence, evaluations are often carried out on datasets from real infrastructures, where analysts cannot control system parameters or generate a reliable ground truth, or private datasets that prevent reproducibility of results. As a solution, we present a collection of maintainable log datasets collected in a testbed representing a small enterprise. Thereby, we employ extensive state machines to simulate normal user behavior and inject a multi-step attack. For scalable testbed deployment, we use concepts from model-driven engineering that enable automatic generation and labeling of an arbitrary number of datasets that comprise repetitions of attack executions with variations of parameters. In total, we provide 8 datasets containing 20 distinct types of log files, of which we label 8 files for 10 unique attack steps. We publish the labeled log datasets and code for testbed setup and simulation online as open-source to enable others to reproduce and extend our results. △ Less

Submitted 16 March, 2022; originally announced March 2022.

Journal ref: IEEE Transactions on Dependable and Secure Computing (2022)