-
Deep Learning for Micro-Scale Crack Detection on Imbalanced Datasets Using Key Point Localization
Authors:
Fatahlla Moreh,
Yusuf Hasan,
Bilal Zahid Hussain,
Mohammad Ammar,
Sven Tomforde
Abstract:
Internal crack detection has been a subject of focus in structural health monitoring. By focusing on crack detection in structural datasets, it is demonstrated that deep learning (DL) methods can effectively analyze seismic wave fields interacting with micro-scale cracks, which are beyond the resolution of conventional visual inspection. This work explores a novel application of DL-based key point…
▽ More
Internal crack detection has been a subject of focus in structural health monitoring. By focusing on crack detection in structural datasets, it is demonstrated that deep learning (DL) methods can effectively analyze seismic wave fields interacting with micro-scale cracks, which are beyond the resolution of conventional visual inspection. This work explores a novel application of DL-based key point detection technique, where cracks are localized by predicting the coordinates of four key points that define a bounding region of the crack. The study not only opens new research directions for non-visual applications but also effectively mitigates the impact of imbalanced data which poses a challenge for previous DL models, as it can be biased toward predicting the majority class (non-crack regions). Popular DL techniques, such as the Inception blocks, are used and investigated. The model shows an overall reduction in loss when applied to micro-scale crack detection and is reflected in the lower average deviation between the location of actual and predicted cracks, with an average Intersection over Union (IoU) being 0.511 for all micro cracks (greater than 0.00 micrometers) and 0.631 for larger micro cracks (greater than 4 micrometers).
△ Less
Submitted 15 November, 2024;
originally announced November 2024.
-
MicroCrackAttentionNeXt: Advancing Microcrack Detection in Wave Field Analysis Using Deep Neural Networks through Feature Visualization
Authors:
Fatahlla Moreh,
Yusuf Hasan,
Bilal Zahid Hussain,
Mohammad Ammar,
Sven Tomforde
Abstract:
Micro Crack detection using deep neural networks (DNNs) through an automated pipeline using wave fields interacting with the damaged areas is highly sought after. These high-dimensional spatio-temporal crack data are limited, and these datasets have large dimensions in the temporal domain. The dataset presents a substantial class imbalance, with crack pixels constituting an average of only 5% of t…
▽ More
Micro Crack detection using deep neural networks (DNNs) through an automated pipeline using wave fields interacting with the damaged areas is highly sought after. These high-dimensional spatio-temporal crack data are limited, and these datasets have large dimensions in the temporal domain. The dataset presents a substantial class imbalance, with crack pixels constituting an average of only 5% of the total pixels per sample. This extreme class imbalance poses a challenge for deep learning models with the different micro-scale cracks, as the network can be biased toward predicting the majority class, generally leading to poor detection accuracy. This study builds upon the previous benchmark SpAsE-Net, an asymmetric encoder-decoder network for micro-crack detection. The impact of various activation and loss functions were examined through feature space visualization using the manifold discovery and analysis (MDA) algorithm. The optimized architecture and training methodology achieved an accuracy of 86.85%.
△ Less
Submitted 15 November, 2024;
originally announced November 2024.
-
Towards adaptive trajectories for mixed autonomous and human-operated ships
Authors:
Danilo Pianini,
Sven Tomforde
Abstract:
We are witnessing the rise of autonomous cars, which will likely revolutionize the way we travel. Arguably, the maritime domain lags behind, as ships operate on many more degrees of freedom (thus, a much larger search space): there is less physical infrastructure, and rules are less consistent and constraining than what is found on roads. The problem is further complicated by the inevitable co-exi…
▽ More
We are witnessing the rise of autonomous cars, which will likely revolutionize the way we travel. Arguably, the maritime domain lags behind, as ships operate on many more degrees of freedom (thus, a much larger search space): there is less physical infrastructure, and rules are less consistent and constraining than what is found on roads. The problem is further complicated by the inevitable co-existence of autonomous and human-operated ships: the latter may take unpredictable decisions, which require adjustments on the autonomous ones. Finally, the problem is inherently decentralised, there is no central authority, and communication means can be very diverse in terms of communication distance and performance, mandating special care on which information is shared and how. In this work, we elaborate on the challenges of trajectory prediction and adaptation for mixed autonomous and human-operated ships, and we propose initial ideas on potential approaches to address them.
△ Less
Submitted 19 September, 2024;
originally announced September 2024.
-
Fault Detection for agents on power grid topology optimization: A Comprehensive analysis
Authors:
Malte Lehna,
Mohamed Hassouna,
Dmitry Degtyar,
Sven Tomforde,
Christoph Scholz
Abstract:
Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic grid scenarios and underlying power flow backends. However, th…
▽ More
Optimizing the topology of transmission networks using Deep Reinforcement Learning (DRL) has increasingly come into focus. Various DRL agents have been proposed, which are mostly benchmarked on the Grid2Op environment from the Learning to Run a Power Network (L2RPN) challenges. The environments have many advantages with their realistic grid scenarios and underlying power flow backends. However, the interpretation of agent survival or failure is not always clear, as there are a variety of potential causes. In this work, we focus on the failures of the power grid simulation to identify patterns and detect them in advance. We collect the failed scenarios of three different agents on the WCCI 2022 L2RPN environment, totaling about 40k data points. By clustering, we are able to detect five distinct clusters, identifying common failure types. Further, we propose a multi-class prediction approach to detect failures beforehand and evaluate five different prediction models. Here, the Light Gradient-Boosting Machine (LightGBM) shows the best failure prediction performance, with an accuracy of 82%. It also accurately classifies whether a the grid survives or fails in 87% of cases. Finally, we provide a detailed feature importance analysis that identifies critical features and regions in the grid.
△ Less
Submitted 17 September, 2024; v1 submitted 24 June, 2024;
originally announced June 2024.
-
HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach
Authors:
Malte Lehna,
Clara Holzhüter,
Sven Tomforde,
Christoph Scholz
Abstract:
With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimizati…
▽ More
With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimization by most existing DRL algorithms. In contrast, we propose a more holistic approach by proposing specific Target Topologies (TTs) as actions. These topologies are selected based on their robustness. As part of this paper, we present a search algorithm to find the TTs and upgrade our previously developed DRL agent CurriculumAgent (CAgent) to a novel topology agent. We compare the upgrade to the previous CAgent and can increase their L2RPN score significantly by 10%. Further, we achieve a 25% better median survival time with our TTs included. Later analysis shows that almost all TTs are close to the base topology, explaining their robustness
△ Less
Submitted 23 May, 2024; v1 submitted 1 May, 2024;
originally announced May 2024.
-
Exploring the Dynamics of Data Transmission in 5G Networks: A Conceptual Analysis
Authors:
Nikita Smirnov,
Sven Tomforde
Abstract:
This conceptual analysis examines the dynamics of data transmission in 5G networks. It addresses various aspects of sending data from cameras and LiDARs installed on a remote-controlled ferry to a land-based control center. The range of topics includes all stages of video and LiDAR data processing from acquisition and encoding to final decoding, all aspects of their transmission and reception via…
▽ More
This conceptual analysis examines the dynamics of data transmission in 5G networks. It addresses various aspects of sending data from cameras and LiDARs installed on a remote-controlled ferry to a land-based control center. The range of topics includes all stages of video and LiDAR data processing from acquisition and encoding to final decoding, all aspects of their transmission and reception via the WebRTC protocol, and all possible types of network problems such as handovers or congestion that could affect the quality of experience for end-users. A series of experiments were conducted to evaluate the key aspects of the data transmission. These include simulation-based reproducible runs and real-world experiments conducted using open-source solutions we developed: "Gymir5G" - an OMNeT++-based 5G simulation and "GstWebRTCApp" - a GStreamer-based application for adaptive control of media streams over the WebRTC protocol. One of the goals of this study is to formulate the bandwidth and latency requirements for reliable real-time communication and to estimate their approximate values. This goal was achieved through simulation-based experiments involving docking maneuvers in the Bay of Kiel, Germany. The final latency for the entire data processing pipeline was also estimated during the real tests. In addition, a series of simulation-based experiments showed the impact of key WebRTC features and demonstrated the effectiveness of the WebRTC protocol, while the conducted video codec comparison showed that the hardware-accelerated H.264 codec is the best. Finally, the research addresses the topic of adaptive communication, where the traditional congestion avoidance and deep reinforcement learning approaches were analyzed. The comparison in a sandbox scenario shows that the AI-based solution outperforms the WebRTC baseline GCC algorithm in terms of data rates, latency, and packet loss.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics
Authors:
Lukas Rauch,
Raphael Schwinger,
Moritz Wirth,
René Heinrich,
Denis Huseljic,
Marek Herde,
Jonas Lange,
Stefan Kahl,
Bernhard Sick,
Sven Tomforde,
Christoph Scholz
Abstract:
Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet is a pivotal step to bridge this gap as a universal-domain dataset, its restricted accessibility and limited range of evaluation use cases challenge its role as the sole resource. Therefore, we introdu…
▽ More
Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet is a pivotal step to bridge this gap as a universal-domain dataset, its restricted accessibility and limited range of evaluation use cases challenge its role as the sole resource. Therefore, we introduce BirdSet, a large-scale benchmark dataset for audio classification focusing on avian bioacoustics. BirdSet surpasses AudioSet with over 6,800 recording hours ($\uparrow\!17\%$) from nearly 10,000 classes ($\uparrow\!18\times$) for training and more than 400 hours ($\uparrow\!7\times$) across eight strongly labeled evaluation datasets. It serves as a versatile resource for use cases such as multi-label classification, covariate shift or self-supervised learning. We benchmark six well-known DL models in multi-label classification across three distinct training scenarios and outline further evaluation use cases in audio classification. We host our dataset on Hugging Face for easy accessibility and offer an extensive codebase to reproduce our results.
△ Less
Submitted 18 May, 2025; v1 submitted 15 March, 2024;
originally announced March 2024.
-
Active Bird2Vec: Towards End-to-End Bird Sound Monitoring with Transformers
Authors:
Lukas Rauch,
Raphael Schwinger,
Moritz Wirth,
Bernhard Sick,
Sven Tomforde,
Christoph Scholz
Abstract:
We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ActiveBird2Vec is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environm…
▽ More
We propose a shift towards end-to-end learning in bird sound monitoring by combining self-supervised (SSL) and deep active learning (DAL). Leveraging transformer models, we aim to bypass traditional spectrogram conversions, enabling direct raw audio processing. ActiveBird2Vec is set to generate high-quality bird sound representations through SSL, potentially accelerating the assessment of environmental changes and decision-making processes for wind farms. Additionally, we seek to utilize the wide variety of bird vocalizations through DAL, reducing the reliance on extensively labeled datasets by human experts. We plan to curate a comprehensive set of tasks through Huggingface Datasets, enhancing future comparability and reproducibility of bioacoustic research. A comparative analysis between various transformer models will be conducted to evaluate their proficiency in bird sound recognition tasks. We aim to accelerate the progression of avian bioacoustic research and contribute to more effective conservation strategies.
△ Less
Submitted 21 November, 2023; v1 submitted 14 August, 2023;
originally announced August 2023.
-
Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents
Authors:
Malte Lehna,
Jan Viebahn,
Christoph Scholz,
Antoine Marot,
Sven Tomforde
Abstract:
The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with con…
▽ More
The operation of electricity grids has become increasingly complex due to the current upheaval and the increase in renewable energy production. As a consequence, active grid management is reaching its limits with conventional approaches. In the context of the Learning to Run a Power Network challenge, it has been shown that Reinforcement Learning (RL) is an efficient and reliable approach with considerable potential for automatic grid operation. In this article, we analyse the submitted agent from Binbinchen and provide novel strategies to improve the agent, both for the RL and the rule-based approach. The main improvement is a N-1 strategy, where we consider topology actions that keep the grid stable, even if one line is disconnected. More, we also propose a topology reversion to the original grid, which proved to be beneficial. The improvements are tested against reference approaches on the challenge test sets and are able to increase the performance of the rule-based agent by 27%. In direct comparison between rule-based and RL agent we find similar performance. However, the RL agent has a clear computational advantage. We also analyse the behaviour in an exemplary case in more detail to provide additional insights. Here, we observe that through the N-1 strategy, the actions of the agents become more diversified.
△ Less
Submitted 17 April, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
A Quantification Approach for Transferability in Lifelike Computing Systems
Authors:
Martin Goller,
Sven Tomforde
Abstract:
The basic idea of lifelike computing systems is the transfer of concepts in living systems to technical use that goes even beyond existing concepts of self-adaptation and self-organisation (SASO). As a result, these systems become even more autonomous and changeable - up to a runtime transfer of the actual target function. Maintaining controllability requires a complete and dynamic (self-)quantifi…
▽ More
The basic idea of lifelike computing systems is the transfer of concepts in living systems to technical use that goes even beyond existing concepts of self-adaptation and self-organisation (SASO). As a result, these systems become even more autonomous and changeable - up to a runtime transfer of the actual target function. Maintaining controllability requires a complete and dynamic (self-)quantification of the system behaviour with regard to aspects of SASO but also, in particular, lifelike properties. In this article, we discuss possible approaches for such metrics and establish a first metric for transferability. We analyse the behaviour of the metric using example applications and show that it is suitable for describing the system's behaviour at runtime.
△ Less
Submitted 30 January, 2023;
originally announced January 2023.
-
Active Reinforcement Learning -- A Roadmap Towards Curious Classifier Systems for Self-Adaptation
Authors:
Simon Reichhuber,
Sven Tomforde
Abstract:
Intelligent systems have the ability to improve their behaviour over time taking observations, experiences or explicit feedback into account. Traditional approaches separate the learning problem and make isolated use of techniques from different field of machine learning such as reinforcement learning, active learning, anomaly detection or transfer learning, for instance. In this context, the fund…
▽ More
Intelligent systems have the ability to improve their behaviour over time taking observations, experiences or explicit feedback into account. Traditional approaches separate the learning problem and make isolated use of techniques from different field of machine learning such as reinforcement learning, active learning, anomaly detection or transfer learning, for instance. In this context, the fundamental reinforcement learning approaches come with several drawbacks that hinder their application to real-world systems: trial-and-error, purely reactive behaviour or isolated problem handling. The idea of this article is to present a concept for alleviating these drawbacks by setting up a research agenda towards what we call "active reinforcement learning" in intelligent systems.
△ Less
Submitted 11 January, 2022;
originally announced January 2022.
-
Towards Organic Distribution Systems -- The Vision of Self-Configuring, Self-Organising, Self-Healing, and Self-Optimising Power Distribution Management
Authors:
Inga Loeser,
Martin Braun,
Christian Gruhl,
Jan-Hendrik Menke,
Bernhard Sick,
Sven Tomforde
Abstract:
Due to the decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the basic critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system's operational limits…
▽ More
Due to the decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the basic critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system's operational limits are not violated. On the grid level, distribution management systems (DMS) try to keep the power system in the normal operational state. On the prosumer level, distributed HEMS optimise the internal power flows by using batteries, photovoltaic generators, or flexible loads optimally. The vision of the ODiS (Organic Distribution System) initiative is to develop an architecture to operate a distribution grid reliably, with high resiliency, and fully autonomously by developing "organic" HEMS and DMS which possess multiple self-* capabilities. Thus, ODiS seeks answers to the following question: How can we create the most appropriate models, techniques, and algorithms to develop novel kinds of self-configuring, self-organising, self-healing, and self-optimising DMS that are integrally coupled with the distributed HEMS? In this article, the vision of ODiS is presented in detail based on a thorough review of the state of the art.
△ Less
Submitted 14 December, 2021;
originally announced December 2021.
-
Learning to Learn in Collective Adaptive Systems: Mining Design Patterns for Data-driven Reasoning
Authors:
Mirko D'Angelo,
Sona Ghahremani,
Simos Gerasimou,
Johannes Grohmann,
Ingrid Nunes,
Sven Tomforde,
Evangelos Pournaras
Abstract:
Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multi-dimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patt…
▽ More
Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multi-dimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patterns of learning-based CAS. Using data from a systematic literature review, reasoning is performed with a novel application of data-driven methodologies such as clustering, multiple correspondence analysis and decision trees. The reasoning based on past experience as well as supporting novel and innovative design choices are demonstrated.
△ Less
Submitted 10 August, 2020;
originally announced August 2020.
-
Collaborative Interactive Learning -- A clarification of terms and a differentiation from other research fields
Authors:
Tom Hanika,
Marek Herde,
Jochen Kuhn,
Jan Marco Leimeister,
Paul Lukowicz,
Sarah Oeste-Reiß,
Albrecht Schmidt,
Bernhard Sick,
Gerd Stumme,
Sven Tomforde,
Katharina Anna Zweig
Abstract:
The field of collaborative interactive learning (CIL) aims at developing and investigating the technological foundations for a new generation of smart systems that support humans in their everyday life. While the concept of CIL has already been carved out in detail (including the fields of dedicated CIL and opportunistic CIL) and many research objectives have been stated, there is still the need t…
▽ More
The field of collaborative interactive learning (CIL) aims at developing and investigating the technological foundations for a new generation of smart systems that support humans in their everyday life. While the concept of CIL has already been carved out in detail (including the fields of dedicated CIL and opportunistic CIL) and many research objectives have been stated, there is still the need to clarify some terms such as information, knowledge, and experience in the context of CIL and to differentiate CIL from recent and ongoing research in related fields such as active learning, collaborative learning, and others. Both aspects are addressed in this paper.
△ Less
Submitted 16 May, 2019;
originally announced May 2019.
-
On the Detection of Mutual Influences and Their Consideration in Reinforcement Learning Processes
Authors:
Stefan Rudolph,
Sven Tomforde,
Jörg Hähner
Abstract:
Self-adaptation has been proposed as a mechanism to counter complexity in control problems of technical systems. A major driver behind self-adaptation is the idea to transfer traditional design-time decisions to runtime and into the responsibility of systems themselves. In order to deal with unforeseen events and conditions, systems need creativity -- typically realized by means of machine learnin…
▽ More
Self-adaptation has been proposed as a mechanism to counter complexity in control problems of technical systems. A major driver behind self-adaptation is the idea to transfer traditional design-time decisions to runtime and into the responsibility of systems themselves. In order to deal with unforeseen events and conditions, systems need creativity -- typically realized by means of machine learning capabilities. Such learning mechanisms are based on different sources of knowledge. Feedback from the environment used for reinforcement purposes is probably the most prominent one within the self-adapting and self-organizing (SASO) systems community. However, the impact of other (sub-)systems on the success of the individual system's learning performance has mostly been neglected in this context. In this article, we propose a novel methodology to identify effects of actions performed by other systems in a shared environment on the utility achievement of an autonomous system. Consider smart cameras (SC) as illustrating example: For goals such as 3D reconstruction of objects, the most promising configuration of one SC in terms of pan/tilt/zoom parameters depends largely on the configuration of other SCs in the vicinity. Since such mutual influences cannot be pre-defined for dynamic systems, they have to be learned at runtime. Furthermore, they have to be taken into consideration when self-improving the own configuration decisions based on a feedback loop concept, e.g., known from the SASO domain or the Autonomic and Organic Computing initiatives. We define a methodology to detect such influences at runtime, present an approach to consider this information in a reinforcement learning technique, and analyze the behavior in artificial as well as real-world SASO system settings.
△ Less
Submitted 10 May, 2019;
originally announced May 2019.
-
Self-Adaptation of Activity Recognition Systems to New Sensors
Authors:
David Bannach,
Martin Jänicke,
Vitor F. Rey,
Sven Tomforde,
Bernhard Sick,
Paul Lukowicz
Abstract:
Traditional activity recognition systems work on the basis of training, taking a fixed set of sensors into account. In this article, we focus on the question how pattern recognition can leverage new information sources without any, or with minimal user input. Thus, we present an approach for opportunistic activity recognition, where ubiquitous sensors lead to dynamically changing input spaces. Our…
▽ More
Traditional activity recognition systems work on the basis of training, taking a fixed set of sensors into account. In this article, we focus on the question how pattern recognition can leverage new information sources without any, or with minimal user input. Thus, we present an approach for opportunistic activity recognition, where ubiquitous sensors lead to dynamically changing input spaces. Our method is a variation of well-established principles of machine learning, relying on unsupervised clustering to discover structure in data and inferring cluster labels from a small number of labeled dates in a semi-supervised manner. Elaborating the challenges, evaluations of over 3000 sensor combinations from three multi-user experiments are presented in detail and show the potential benefit of our approach.
△ Less
Submitted 30 January, 2017;
originally announced January 2017.
-
Organic Computing in the Spotlight
Authors:
Sven Tomforde,
Bernhard Sick,
Christian Müller-Schloer
Abstract:
Organic Computing is an initiative in the field of systems engineering that proposed to make use of concepts such as self-adaptation and self-organisation to increase the robustness of technical systems. Based on the observation that traditional design and operation concepts reach their limits, transferring more autonomy to the systems themselves should result in a reduction of complexity for user…
▽ More
Organic Computing is an initiative in the field of systems engineering that proposed to make use of concepts such as self-adaptation and self-organisation to increase the robustness of technical systems. Based on the observation that traditional design and operation concepts reach their limits, transferring more autonomy to the systems themselves should result in a reduction of complexity for users, administrators, and developers. However, there seems to be a need for an updated definition of the term "Organic Computing", of desired properties of technical, organic systems, and the objectives of the Organic Computing initiative. With this article, we will address these points.
△ Less
Submitted 27 January, 2017;
originally announced January 2017.