-
Concrete Safety for ML Problems: System Safety for ML Development and Assessment
Authors:
Edgar W. Jatho,
Logan O. Mailloux,
Eugene D. Williams,
Patrick McClure,
Joshua A. Kroll
Abstract:
Many stakeholders struggle to make reliances on ML-driven systems due to the risk of harm these systems may cause. Concerns of trustworthiness, unintended social harms, and unacceptable social and ethical violations undermine the promise of ML advancements. Moreover, such risks in complex ML-driven systems present a special challenge as they are often difficult to foresee, arising over periods of…
▽ More
Many stakeholders struggle to make reliances on ML-driven systems due to the risk of harm these systems may cause. Concerns of trustworthiness, unintended social harms, and unacceptable social and ethical violations undermine the promise of ML advancements. Moreover, such risks in complex ML-driven systems present a special challenge as they are often difficult to foresee, arising over periods of time, across populations, and at scale. These risks often arise not from poor ML development decisions or low performance directly but rather emerge through the interactions amongst ML development choices, the context of model use, environmental factors, and the effects of a model on its target. Systems safety engineering is an established discipline with a proven track record of identifying and managing risks even in high-complexity sociotechnical systems. In this work, we apply a state-of-the-art systems safety approach to concrete applications of ML with notable social and ethical risks to demonstrate a systematic means for meeting the assurance requirements needed to argue for safe and trustworthy ML in sociotechnical systems.
△ Less
Submitted 6 February, 2023;
originally announced February 2023.
-
System Safety Engineering for Social and Ethical ML Risks: A Case Study
Authors:
Edgar W. Jatho III,
Logan O. Mailloux,
Shalaleh Rismani,
Eugene D. Williams,
Joshua A. Kroll
Abstract:
Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems, with a particular focus on social and ethical risks of ML components in complex sociotechnical systems. However, existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. Systems safety engineering is a well established discipline with a track record of identifyin…
▽ More
Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems, with a particular focus on social and ethical risks of ML components in complex sociotechnical systems. However, existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. Systems safety engineering is a well established discipline with a track record of identifying and managing risks in many complex sociotechnical domains. We adopt the natural hypothesis that tools from this domain could serve to enhance risk analyses of ML in its context of use. To test this hypothesis, we apply a "best of breed" systems safety analysis, Systems Theoretic Process Analysis (STPA), to a specific high-consequence system with an important ML-driven component, namely the Prescription Drug Monitoring Programs (PDMPs) operated by many US States, several of which rely on an ML-derived risk score. We focus in particular on how this analysis can extend to identifying social and ethical risks and developing concrete design-level controls to mitigate them.
△ Less
Submitted 8 November, 2022;
originally announced November 2022.
-
Modeling Quantum Optical Components, Pulses and Fiber Channels Using OMNeT++
Authors:
Ryan D. L. Engle,
Douglas D. Hodson,
Michael R. Grimaila,
Logan O. Mailloux,
Colin V. McLaughlin,
Gerald Baumgartner
Abstract:
Quantum Key Distribution (QKD) is an innovative technology which exploits the laws of quantum mechanics to generate and distribute unconditionally secure cryptographic keys. While QKD offers the promise of unconditionally secure key distribution, real world systems are built from non-ideal components which necessitates the need to model and understand the impact these non-idealities have on system…
▽ More
Quantum Key Distribution (QKD) is an innovative technology which exploits the laws of quantum mechanics to generate and distribute unconditionally secure cryptographic keys. While QKD offers the promise of unconditionally secure key distribution, real world systems are built from non-ideal components which necessitates the need to model and understand the impact these non-idealities have on system performance and security. OMNeT++ has been used as a basis to develop a simulation framework to support this endeavor. This framework, referred to as "qkdX" extends OMNeT++'s module and message abstractions to efficiently model optical components, optical pulses, operating protocols and processes. This paper presents the design of this framework including how OMNeT++'s abstractions have been utilized to model quantum optical components, optical pulses, fiber and free space channels. Furthermore, from our toolbox of created components, we present various notional and real QKD systems, which have been studied and analyzed.
△ Less
Submitted 10 September, 2015;
originally announced September 2015.
