-
Generative AI for Autonomous Driving: A Review
Authors:
Katharina Winter,
Abhishek Vivekanandan,
Rupert Polley,
Yinzhe Shen,
Christian Schlauch,
Mohamed-Khalil Bouzidi,
Bojan Derajic,
Natalie Grabowsky,
Annajoyce Mariani,
Dennis Rochau,
Giovanni Lucente,
Harsh Yadav,
Firas Mualla,
Adam Molin,
Sebastian Bernhard,
Christian Wirth,
Ömer Şahin Taş,
Nadja Klein,
Fabian B. Flohr,
Hanno Gottschalk
Abstract:
Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle motion planning. By examining multiple generative approaches ranging from Variatio…
▽ More
Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle motion planning. By examining multiple generative approaches ranging from Variational Autoencoder (VAEs) over Generative Adversarial Networks (GANs) and Invertible Neural Networks (INNs) to Generative Transformers (GTs) and Diffusion Models (DMs), we highlight and compare their capabilities and limitations for AD-specific applications. Additionally, we discuss hybrid methods integrating conventional techniques with generative approaches, and emphasize their improved adaptability and robustness. We also identify relevant datasets and outline open research questions to guide future developments in GenAI. Finally, we discuss three core challenges: safety, interpretability, and realtime capabilities, and present recommendations for image generation, dynamic scenario generation, and planning.
△ Less
Submitted 21 May, 2025;
originally announced May 2025.
-
BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Driving
Authors:
Katharina Winter,
Mark Azer,
Fabian B. Flohr
Abstract:
Autonomous driving has the potential to set the stage for more efficient future mobility, requiring the research domain to establish trust through safe, reliable and transparent driving. Large Language Models (LLMs) possess reasoning capabilities and natural language understanding, presenting the potential to serve as generalized decision-makers for ego-motion planning that can interact with human…
▽ More
Autonomous driving has the potential to set the stage for more efficient future mobility, requiring the research domain to establish trust through safe, reliable and transparent driving. Large Language Models (LLMs) possess reasoning capabilities and natural language understanding, presenting the potential to serve as generalized decision-makers for ego-motion planning that can interact with humans and navigate environments designed for human drivers. While this research avenue is promising, current autonomous driving approaches are challenged by combining 3D spatial grounding and the reasoning and language capabilities of LLMs. We introduce BEVDriver, an LLM-based model for end-to-end closed-loop driving in CARLA that utilizes latent BEV features as perception input. BEVDriver includes a BEV encoder to efficiently process multi-view images and 3D LiDAR point clouds. Within a common latent space, the BEV features are propagated through a Q-Former to align with natural language instructions and passed to the LLM that predicts and plans precise future trajectories while considering navigation instructions and critical scenarios. On the LangAuto benchmark, our model reaches up to 18.9% higher performance on the Driving Score compared to SoTA methods.
△ Less
Submitted 4 March, 2025;
originally announced March 2025.
-
Fast, Secure, Adaptable: LionsOS Design, Implementation and Performance
Authors:
Gernot Heiser,
Ivan Velickovic,
Peter Chubb,
Alwin Joshy,
Anuraag Ganesh,
Bill Nguyen,
Cheng Li,
Courtney Darville,
Guangtao Zhu,
James Archer,
Jingyao Zhou,
Krishnan Winter,
Lucy Parker,
Szymon Duchniewicz,
Tianyi Bai
Abstract:
We present LionsOS, an operating system for security- and safety-critical embedded systems. LionsOS is based on the formally verified seL4 microkernel and designed with verification in mind. It uses a static architecture and features a highly modular design driven by strict separa- tion of concerns and a focus on simplicity. We demonstrate that LionsOS achieves excellent performance on system-call…
▽ More
We present LionsOS, an operating system for security- and safety-critical embedded systems. LionsOS is based on the formally verified seL4 microkernel and designed with verification in mind. It uses a static architecture and features a highly modular design driven by strict separa- tion of concerns and a focus on simplicity. We demonstrate that LionsOS achieves excellent performance on system-call intensive workloads.
△ Less
Submitted 27 May, 2025; v1 submitted 8 January, 2025;
originally announced January 2025.
-
MEDIATE: Mutually Endorsed Distributed Incentive Acknowledgment Token Exchange
Authors:
Philipp Altmann,
Katharina Winter,
Michael Kölle,
Maximilian Zorn,
Thomy Phan,
Claudia Linnhoff-Popien
Abstract:
Recent advances in multi-agent systems (MAS) have shown that incorporating peer incentivization (PI) mechanisms vastly improves cooperation. Especially in social dilemmas, communication between the agents helps to overcome sub-optimal Nash equilibria. However, incentivization tokens need to be carefully selected. Furthermore, real-world applications might yield increased privacy requirements and l…
▽ More
Recent advances in multi-agent systems (MAS) have shown that incorporating peer incentivization (PI) mechanisms vastly improves cooperation. Especially in social dilemmas, communication between the agents helps to overcome sub-optimal Nash equilibria. However, incentivization tokens need to be carefully selected. Furthermore, real-world applications might yield increased privacy requirements and limited exchange. Therefore, we extend the PI protocol for mutual acknowledgment token exchange (MATE) and provide additional analysis on the impact of the chosen tokens. Building upon those insights, we propose mutually endorsed distributed incentive acknowledgment token exchange (MEDIATE), an extended PI architecture employing automatic token derivation via decentralized consensus. Empirical results show the stable agreement on appropriate tokens yielding superior performance compared to static tokens and state-of-the-art approaches in different social dilemma environments with various reward distributions.
△ Less
Submitted 4 April, 2024;
originally announced April 2024.
-
Bias correction of wind power forecasts with SCADA data and continuous learning
Authors:
Stefan Jonas,
Kevin Winter,
Bernhard Brodbeck,
Angela Meyer
Abstract:
Wind energy plays a critical role in the transition towards renewable energy sources. However, the uncertainty and variability of wind can impede its full potential and the necessary growth of wind power capacity. To mitigate these challenges, wind power forecasting methods are employed for applications in power management, energy trading, or maintenance scheduling. In this work, we present, evalu…
▽ More
Wind energy plays a critical role in the transition towards renewable energy sources. However, the uncertainty and variability of wind can impede its full potential and the necessary growth of wind power capacity. To mitigate these challenges, wind power forecasting methods are employed for applications in power management, energy trading, or maintenance scheduling. In this work, we present, evaluate, and compare four machine learning-based wind power forecasting models. Our models correct and improve 48-hour forecasts extracted from a numerical weather prediction (NWP) model. The models are evaluated on datasets from a wind park comprising 65 wind turbines. The best improvement in forecasting error and mean bias was achieved by a convolutional neural network, reducing the average NRMSE down to 22%, coupled with a significant reduction in mean bias, compared to a NRMSE of 35% from the strongly biased baseline model using uncorrected NWP forecasts. Our findings further indicate that changes to neural network architectures play a minor role in affecting the forecasting performance, and that future research should rather investigate changes in the model pipeline. Moreover, we introduce a continuous learning strategy, which is shown to achieve the highest forecasting performance improvements when new data is made available.
△ Less
Submitted 21 February, 2024;
originally announced February 2024.
-
Bridging MDE and AI: A Systematic Review of Domain-Specific Languages and Model-Driven Practices in AI Software Systems Engineering
Authors:
Simon Raedler,
Luca Berardinelli,
Karolin Winter,
Abbas Rahimi,
Stefanie Rinderle-Ma
Abstract:
Background:Technical systems are growing in complexity with more components and functions across various disciplines. Model-Driven Engineering (MDE) helps manage this complexity by using models as key artifacts. Domain-Specific Languages (DSL) supported by MDE facilitate modeling. As data generation in product development increases, there's a growing demand for AI algorithms, which can be challeng…
▽ More
Background:Technical systems are growing in complexity with more components and functions across various disciplines. Model-Driven Engineering (MDE) helps manage this complexity by using models as key artifacts. Domain-Specific Languages (DSL) supported by MDE facilitate modeling. As data generation in product development increases, there's a growing demand for AI algorithms, which can be challenging to implement. Integrating AI algorithms with DSL and MDE can streamline this process. Objective:This study aims to investigate the existing model-driven approaches relying on DSL in support of the engineering of AI software systems to sharpen future research further and define the current state of the art. Method:We conducted a Systemic Literature Review (SLR), collecting papers from five major databases resulting in 1335 candidate studies, eventually retaining 18 primary studies. Each primary study will be evaluated and discussed with respect to the adoption of MDE principles and practices and the phases of AI development support aligned with the stages of the CRISP-DM methodology. Results:The study's findings show that language workbenches are of paramount importance in dealing with all aspects of modeling language development and are leveraged to define DSL explicitly addressing AI concerns. The most prominent AI-related concerns are training and modeling of the AI algorithm, while minor emphasis is given to the time-consuming preparation of the data. Early project phases that support interdisciplinary communication of requirements, e.g., CRISP-DM Business Understanding phase, are rarely reflected. Conclusion:The study found that the use of MDE for AI is still in its early stages, and there is no single tool or method that is widely used. Additionally, current approaches tend to focus on specific stages of development rather than providing support for the entire development process.
△ Less
Submitted 6 May, 2024; v1 submitted 10 July, 2023;
originally announced July 2023.
-
Predictive Compliance Monitoring in Process-Aware Information Systems: State of the Art, Functionalities, Research Directions
Authors:
Stefanie Rinderle-Ma,
Karolin Winter,
Janik-Vasily Benzin
Abstract:
Business process compliance is a key area of business process management and aims at ensuring that processes obey to compliance constraints such as regulatory constraints or business rules imposed on them. Process compliance can be checked during process design time based on verification of process models and at runtime based on monitoring the compliance states of running process instances. For ex…
▽ More
Business process compliance is a key area of business process management and aims at ensuring that processes obey to compliance constraints such as regulatory constraints or business rules imposed on them. Process compliance can be checked during process design time based on verification of process models and at runtime based on monitoring the compliance states of running process instances. For existing compliance monitoring approaches it remains unclear whether and how compliance violations can be predicted, although predictions are crucial in order to prepare and take countermeasures in time. This work, hence, analyzes existing literature from compliance monitoring as well as predictive process monitoring and provides an updated framework of compliance monitoring functionalities. Moreover, it raises the vision of a comprehensive predictive compliance monitoring system that integrates existing predicate prediction approaches with the idea of employing PPM with different prediction goals such as next activity or remaining time for prediction and subsequent mapping of the prediction results onto the given set of compliance constraints (PCM). For each compliance monitoring functionality we elicit PCM system requirements and assess their coverage by existing approaches. Based on the assessment, open challenges and research directions realizing a comprehensive PCM system are elaborated.
△ Less
Submitted 2 March, 2023; v1 submitted 10 May, 2022;
originally announced May 2022.
-
Detecting Production Phases Based on Sensor Values using 1D-CNNs
Authors:
Burkhard Hoppenstedt,
Manfred Reichert,
Ghada El-Khawaga,
Klaus Kammerer,
Karl-Michael Winter,
Rüdiger Pryss
Abstract:
In the context of Industry 4.0, the knowledge extraction from sensor information plays an important role. Often, information gathered from sensor values reveals meaningful insights for production levels, such as anomalies or machine states. In our use case, we identify production phases through the inspection of sensor values with the help of convolutional neural networks. The data set stems from…
▽ More
In the context of Industry 4.0, the knowledge extraction from sensor information plays an important role. Often, information gathered from sensor values reveals meaningful insights for production levels, such as anomalies or machine states. In our use case, we identify production phases through the inspection of sensor values with the help of convolutional neural networks. The data set stems from a tempering furnace used for metal heat treating. Our supervised learning approach unveils a promising accuracy for the chosen neural network that was used for the detection of production phases. We consider solutions like shown in this work as salient pillars in the field of predictive maintenance.
△ Less
Submitted 23 April, 2020;
originally announced April 2020.
-
An abstract semantics of speculative execution for reasoning about security vulnerabilities
Authors:
Robert J. Colvin,
Kirsten Winter
Abstract:
Reasoning about correctness and security of software is increasingly difficult due to the complexity of modern microarchitectural features such as out-of-order execution. A class of security vulnerabilities termed Spectre that exploits side effects of speculative, out-of-order execution was announced in 2018 and has since drawn much attention. In this paper we formalise speculative execution and i…
▽ More
Reasoning about correctness and security of software is increasingly difficult due to the complexity of modern microarchitectural features such as out-of-order execution. A class of security vulnerabilities termed Spectre that exploits side effects of speculative, out-of-order execution was announced in 2018 and has since drawn much attention. In this paper we formalise speculative execution and its side effects with the intention of allowing speculation to be reasoned about abstractly at the program level, limiting the exposure to processor-specific or low-level semantics. To this end we encode and expose speculative execution explicitly in the programming language, rather than solely in the operational semantics; as a result the effects of speculative execution are captured by redefining the meaning of a conditional statement, and introducing novel language constructs that model transient execution of an alternative branch. We add an abstract cache to the global state of the system, and derive some general refinement rules that expose cache side effects due to speculative loads. Underlying this extension is a semantic model that is based on instruction-level parallelism. The rules are encoded in a simulation tool, which we use to analyse an abstract specification of a Spectre attack and vulnerable code fragments.
△ Less
Submitted 9 March, 2020;
originally announced April 2020.
-
How do Quantifiers Affect the Quality of Requirements?
Authors:
Katharina Winter,
Henning Femmer,
Andreas Vogelsang
Abstract:
Context: Requirements quality can have a substantial impact on the effectiveness and efficiency of using requirements artifacts in a development process. Quantifiers such as "at least", "all", or "exactly" are common language constructs used to express requirements. Quantifiers can be formulated by affirmative phrases ("At least") or negative phrases ("Not less than"). Problem: It is long assumed…
▽ More
Context: Requirements quality can have a substantial impact on the effectiveness and efficiency of using requirements artifacts in a development process. Quantifiers such as "at least", "all", or "exactly" are common language constructs used to express requirements. Quantifiers can be formulated by affirmative phrases ("At least") or negative phrases ("Not less than"). Problem: It is long assumed that negation in quantification negatively affects the readability of requirements, however, empirical research on these topics remains sparse. Principal Idea: In a web-based experiment with 51 participants, we compare the impact of negations and quantifiers on readability in terms of reading effort, reading error rate and perceived reading difficulty of requirements. Results: For 5 out of 9 quantifiers, our participants performed better on the affirmative phrase compared to the negative phrase. Only for one quantifier, the negative phrase was more effective. Contribution: This research focuses on creating an empirical understanding of the effect of language in Requirements Engineering. It furthermore provides concrete advice on how to phrase requirements.
△ Less
Submitted 7 February, 2020;
originally announced February 2020.
-
Untangling the GDPR Using ConRelMiner
Authors:
Karolin Winter,
Stefanie Rinderle-Ma
Abstract:
The General Data Protection Regulation (GDPR) poses enormous challenges on companies and organizations with respect to understanding, implementing, and maintaining the contained constraints. We report on how the ConRelMiner method can be used for untangling the GDPR. For this, the GDPR is filtered and grouped along the roles mentioned by the GDPR and the reduction of sentences to be read by analys…
▽ More
The General Data Protection Regulation (GDPR) poses enormous challenges on companies and organizations with respect to understanding, implementing, and maintaining the contained constraints. We report on how the ConRelMiner method can be used for untangling the GDPR. For this, the GDPR is filtered and grouped along the roles mentioned by the GDPR and the reduction of sentences to be read by analysts is shown. Moreover, the output of the ConRelMiner - a cluster graph with relations between the sentences - is displayed and interpreted. Overall the goal is to illustrate how the effort for implementing the GDPR can be reduced and a structured and meaningful representation of the relevant GDPR sentences can be found.
△ Less
Submitted 8 November, 2018;
originally announced November 2018.
-
Correctness of Concurrent Objects under Weak Memory Models
Authors:
Graeme Smith,
Kirsten Winter,
Robert J. Colvin
Abstract:
In this paper we develop a theory for correctness of concurrent objects under weak memory models. Central to our definitions is the concept of observations which determine when effects of operations become visible, and hence determine the semantics of objects, under a given memory model. The resulting notion of correctness, called object refinement, is generic as it is parameterised by the memory…
▽ More
In this paper we develop a theory for correctness of concurrent objects under weak memory models. Central to our definitions is the concept of observations which determine when effects of operations become visible, and hence determine the semantics of objects, under a given memory model. The resulting notion of correctness, called object refinement, is generic as it is parameterised by the memory model under consideration. Our theory enforces the minimal constraints on the placing of observations and on the semantics of objects that underlie object refinement. Object refinement is suitable as a reference for correctness when proving new proof methods for objects under weak memory models to be sound and complete.
△ Less
Submitted 22 October, 2018;
originally announced October 2018.
-
A sound and complete definition of linearizability on weak memory models
Authors:
Graeme Smith,
Kirsten Winter,
Robert J. Colvin
Abstract:
Linearizability is a widely accepted notion of correctness for concurrent objects. Recent research has investigated redefining linearizability for particular hardware weak memory models, in particular for TSO. In this paper, we provide an overview of this research and show that such redefinitions of linearizability are not required: under an interpretation of specification behaviour which abstract…
▽ More
Linearizability is a widely accepted notion of correctness for concurrent objects. Recent research has investigated redefining linearizability for particular hardware weak memory models, in particular for TSO. In this paper, we provide an overview of this research and show that such redefinitions of linearizability are not required: under an interpretation of specification behaviour which abstracts from weak memory effects, the standard definition of linearizability is sound and complete on all hardware weak memory models. We prove our result with respect to a definition of object refinement which takes a weak memory model as a parameter. The main consequence of our findings is that we can leverage the range of existing techniques and tools for standard linearizability when verifying concurrent objects running on hardware weak memory models.
△ Less
Submitted 1 July, 2019; v1 submitted 13 February, 2018;
originally announced February 2018.
-
A synchronous program algebra: a basis for reasoning about shared-memory and event-based concurrency
Authors:
Ian J. Hayes,
Larissa A. Meinicke,
Kirsten Winter,
Robert J. Colvin
Abstract:
This research started with an algebra for reasoning about rely/guarantee concurrency for a shared memory model. The approach taken led to a more abstract algebra of atomic steps, in which atomic steps synchronise (rather than interleave) when composed in parallel. The algebra of rely/guarantee concurrency then becomes an instantiation of the more abstract algebra. Many of the core properties neede…
▽ More
This research started with an algebra for reasoning about rely/guarantee concurrency for a shared memory model. The approach taken led to a more abstract algebra of atomic steps, in which atomic steps synchronise (rather than interleave) when composed in parallel. The algebra of rely/guarantee concurrency then becomes an instantiation of the more abstract algebra. Many of the core properties needed for rely/guarantee reasoning can be shown to hold in the abstract algebra where their proofs are simpler and hence allow a higher degree of automation. The algebra has been encoded in Isabelle/HOL to provide a basis for tool support for program verification.
In rely/guarantee concurrency, programs are specified to guarantee certain behaviours until assumptions about the behaviour of their environment are violated. When assumptions are violated, program behaviour is unconstrained (aborting), and guarantees need no longer hold. To support these guarantees a second synchronous operator, weak conjunction, was introduced: both processes in a weak conjunction must agree to take each atomic step, unless one aborts in which case the whole aborts. In developing the laws for parallel and weak conjunction we found many properties were shared by the operators and that the proofs of many laws were essentially the same. This insight led to the idea of generalising synchronisation to an abstract operator with only the axioms that are shared by the parallel and weak conjunction operator, so that those two operators can be viewed as instantiations of the abstract synchronisation operator. The main differences between parallel and weak conjunction are how they combine individual atomic steps; that is left open in the axioms for the abstract operator.
△ Less
Submitted 9 October, 2017;
originally announced October 2017.
-
An algebra of synchronous atomic steps
Authors:
Ian J. Hayes,
Robert Colvin,
Larissa Meinicke,
Kirsten Winter,
Andrius Velykis
Abstract:
This research started with an algebra for reasoning about rely/guarantee concurrency for a shared memory model. The approach taken led to a more abstract algebra of atomic steps, in which atomic steps synchronise (rather than interleave) when composed in parallel. The algebra of rely/guarantee concurrency then becomes an interpretation of the more abstract algebra. Many of the core properties need…
▽ More
This research started with an algebra for reasoning about rely/guarantee concurrency for a shared memory model. The approach taken led to a more abstract algebra of atomic steps, in which atomic steps synchronise (rather than interleave) when composed in parallel. The algebra of rely/guarantee concurrency then becomes an interpretation of the more abstract algebra. Many of the core properties needed for rely/guarantee reasoning can be shown to hold in the abstract algebra where their proofs are simpler and hence allow a higher degree of automation. Moreover, the realisation that the synchronisation mechanisms of standard process algebras, such as CSP and CCS/SCCS, can be interpreted in our abstract algebra gives evidence of its unifying power. The algebra has been encoded in Isabelle/HOL to provide a basis for tool support.
△ Less
Submitted 17 January, 2022; v1 submitted 1 September, 2016;
originally announced September 2016.
