Showing 1–2 of 2 results for author: Vaicenavicius, J

Self-driving car safety quantification via component-level analysis

Authors: Juozas Vaicenavicius, Tilo Wiklund, Austė Grigaitė, Antanas Kalkauskas, Ignas Vysniauskas, Steven Keen

Abstract: In this paper, we present a rigorous modular statistical approach for arguing safety or its insufficiency of an autonomous vehicle through a concrete illustrative example. The methodology relies on making appropriate quantitative studies of the performance of constituent components. We explain the importance of sufficient and necessary conditions at the component level for the overall safety of th… ▽ More In this paper, we present a rigorous modular statistical approach for arguing safety or its insufficiency of an autonomous vehicle through a concrete illustrative example. The methodology relies on making appropriate quantitative studies of the performance of constituent components. We explain the importance of sufficient and necessary conditions at the component level for the overall safety of the vehicle as well as the cost-saving benefits of the approach. A simple concrete automated braking example studied illustrates how separate perception system and operational design domain statistical analyses can be used to prove or disprove safety at the vehicle level. △ Less

Submitted 12 April, 2021; v1 submitted 2 September, 2020; originally announced September 2020.

Comments: Various minor linguistic, typographical, and notational improvements. To appear in the SAE International Journal of Connected and Automated Vehicles, 4(1):2021, doi:10.4271/12-04-01-0004

Journal ref: SAE Intl. J CAV 4(1):35-45, 2021

Evaluating model calibration in classification

Authors: Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, Thomas B. Schön

Abstract: Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction. Besides providing a clear separation of prediction and decision making, the main advantage of probabilistic models is their ability to represent uncertainty about predictions. In safety-critical applications, it is pivotal for a model to possess an adequate sense of uncertainty, which… ▽ More Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction. Besides providing a clear separation of prediction and decision making, the main advantage of probabilistic models is their ability to represent uncertainty about predictions. In safety-critical applications, it is pivotal for a model to possess an adequate sense of uncertainty, which for probabilistic classifiers translates into outputting probability distributions that are consistent with the empirical frequencies observed from realized outcomes. A classifier with such a property is called calibrated. In this work, we develop a general theoretical calibration evaluation framework grounded in probability theory, and point out subtleties present in model calibration evaluation that lead to refined interpretations of existing evaluation techniques. Lastly, we propose new ways to quantify and visualize miscalibration in probabilistic classification, including novel multidimensional reliability diagrams. △ Less

Submitted 19 February, 2019; originally announced February 2019.