-
Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs
Authors:
Syed Qutub,
Florian Geissler,
Yang Peng,
Ralf Grafe,
Michael Paulitsch,
Gereon Hinz,
Alois Knoll
Abstract:
Object detection neural network models need to perform reliably in highly dynamic and safety-critical environments like automated driving or robotics. Therefore, it is paramount to verify the robustness of the detection under unexpected hardware faults like soft errors that can impact a systems perception module. Standard metrics based on average precision produce model vulnerability estimates at…
▽ More
Object detection neural network models need to perform reliably in highly dynamic and safety-critical environments like automated driving or robotics. Therefore, it is paramount to verify the robustness of the detection under unexpected hardware faults like soft errors that can impact a systems perception module. Standard metrics based on average precision produce model vulnerability estimates at the object level rather than at an image level. As we show in this paper, this does not provide an intuitive or representative indicator of the safety-related impact of silent data corruption caused by bit flips in the underlying memory but can lead to an over- or underestimation of typical fault-induced hazards. With an eye towards safety-related real-time applications, we propose a new metric IVMOD (Image-wise Vulnerability Metric for Object Detection) to quantify vulnerability based on an incorrect image-wise object detection due to false positive (FPs) or false negative (FNs) objects, combined with a severity analysis. The evaluation of several representative object detection models shows that even a single bit flip can lead to a severe silent data corruption event with potentially critical safety implications, with e.g., up to (much greater than) 100 FPs generated, or up to approx. 90% of true positives (TPs) are lost in an image. Furthermore, with a single stuck-at-1 fault, an entire sequence of images can be affected, causing temporally persistent ghost detections that can be mistaken for actual objects (covering up to approx. 83% of the image). Furthermore, actual objects in the scene are continuously missed (up to approx. 64% of TPs are lost). Our work establishes a detailed understanding of the safety-related vulnerability of such critical workloads against hardware faults.
△ Less
Submitted 7 September, 2022;
originally announced September 2022.
-
Cooperative RADAR Sensors for the Digital Test Field A9 (KoRA9): Algorithmic Recap and Lessons Learned
Authors:
Sören Kohnert,
Julian Stähler,
Reinhard Stolle,
Florian Geissler
Abstract:
Infrastructure sensing systems in combination with Infrastructure-to-Vehicle communication can be used to enhance sensor data obtained from the perspective of a vehicle, only. This paper presents a system consisting of a radar sensor network installed at the side of the street, together with an Edge Processing Unit to fuse the data of different sensors. Measurements taken by the demonstrator are s…
▽ More
Infrastructure sensing systems in combination with Infrastructure-to-Vehicle communication can be used to enhance sensor data obtained from the perspective of a vehicle, only. This paper presents a system consisting of a radar sensor network installed at the side of the street, together with an Edge Processing Unit to fuse the data of different sensors. Measurements taken by the demonstrator are shown, the system architecture is discussed, and some lessons learned are presented.
△ Less
Submitted 4 January, 2022;
originally announced January 2022.
-
A Plausibility-based Fault Detection Method for High-level Fusion Perception Systems
Authors:
Florian Geissler,
Alex Unnervik,
Michael Paulitsch
Abstract:
Trustworthy environment perception is the fundamental basis for the safe deployment of automated agents such as self-driving vehicles or intelligent robots. The problem remains that such trust is notoriously difficult to guarantee in the presence of systematic faults, e.g. non-traceable errors caused by machine learning functions. One way to tackle this issue without making rather specific assumpt…
▽ More
Trustworthy environment perception is the fundamental basis for the safe deployment of automated agents such as self-driving vehicles or intelligent robots. The problem remains that such trust is notoriously difficult to guarantee in the presence of systematic faults, e.g. non-traceable errors caused by machine learning functions. One way to tackle this issue without making rather specific assumptions about the perception process is plausibility checking. Similar to the reasoning of human intuition, the final outcome of a complex black-box procedure is verified against given expectations of an object's behavior. In this article, we apply and evaluate collaborative, sensor-generic plausibility checking as a mean to detect empirical perception faults from their statistical fingerprints. Our real use case is next-generation automated driving that uses a roadside sensor infrastructure for perception augmentation, represented here by test scenarios at a German highway and a city intersection. The plausibilization analysis is integrated naturally in the object fusion process, and helps to diagnose known and possibly yet unknown faults in distributed sensing systems.
△ Less
Submitted 30 September, 2020;
originally announced September 2020.
-
Designing a Roadside Sensor Infrastructure to Support Automated Driving
Authors:
Florian Geissler,
Sören Kohnert,
Reinhard Stolle
Abstract:
Automation of complex traffic scenarios is expected to rely on input from a roadside infrastructure to complement the vehicles' environment perception. We here explore design requirements for a prototypical setup of virtual vision or RADAR sensors along one roadside. Explicitly, we analyze the road coverage and the probability of vehicle occlusions, with the objective of evaluating the completenes…
▽ More
Automation of complex traffic scenarios is expected to rely on input from a roadside infrastructure to complement the vehicles' environment perception. We here explore design requirements for a prototypical setup of virtual vision or RADAR sensors along one roadside. Explicitly, we analyze the road coverage and the probability of vehicle occlusions, with the objective of evaluating the completeness of information that is captured by the sensor field. Simulation case studies are performed based on real traffic data acquired at the German Autobahn 9 near Munich. Our findings indicate how the sensor network should be designed in terms of sensor range, orientation and opening angle, in order to enable effective traffic detection. The achieved degree of completeness suggests that such a setup could be used to support automated vehicles to a substantial extent.
△ Less
Submitted 15 February, 2019;
originally announced February 2019.
