-
The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected World
Authors:
Maurice Chiodo,
Dennis Müller
Abstract:
The increasing deployment of Artificial Intelligence (AI) and other autonomous algorithmic systems presents the world with new systemic risks. While focus often lies on the function of individual algorithms, a critical and underestimated danger arises from their interactions, particularly when algorithmic systems operate without awareness of each other, or when those deploying them are unaware of…
▽ More
The increasing deployment of Artificial Intelligence (AI) and other autonomous algorithmic systems presents the world with new systemic risks. While focus often lies on the function of individual algorithms, a critical and underestimated danger arises from their interactions, particularly when algorithmic systems operate without awareness of each other, or when those deploying them are unaware of the full algorithmic ecosystem deployment is occurring in. These interactions can lead to unforeseen, rapidly escalating negative outcomes - from market crashes and energy supply disruptions to potential physical accidents and erosion of public trust - often exceeding the human capacity for effective monitoring and the legal capacities for proper intervention. Current governance frameworks are inadequate as they lack visibility into this complex ecosystem of interactions. This paper outlines the nature of this challenge and proposes some initial policy suggestions centered on increasing transparency and accountability through phased system registration, a licensing framework for deployment, and enhanced monitoring capabilities.
△ Less
Submitted 26 May, 2025;
originally announced May 2025.
-
Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility
Authors:
Maurice Chiodo,
Dennis Müller,
Paul Siewert,
Jean-Luc Wetherall,
Zoya Yasmine,
John Burden
Abstract:
The legal compliance and safety of different Human-in-the-loop (HITL) setups for AI can vary greatly. This manuscript aims to identify new ways of choosing between such setups, and shows that there is an unavoidable trade-off between the attribution of legal responsibility and the technical explainability of AI. We begin by using the notion of oracle machines from computability theory to formalise…
▽ More
The legal compliance and safety of different Human-in-the-loop (HITL) setups for AI can vary greatly. This manuscript aims to identify new ways of choosing between such setups, and shows that there is an unavoidable trade-off between the attribution of legal responsibility and the technical explainability of AI. We begin by using the notion of oracle machines from computability theory to formalise different HITL setups, distinguishing between trivial human monitoring, single endpoint human action, and highly involved interaction between the human(s) and the AI. These correspond to total functions, many-one reductions, and Turing reductions respectively. A taxonomy categorising HITL failure modes is then presented, highlighting the limitations on what any HITL setup can actually achieve. Our approach then identifies oversights from UK and EU legal frameworks, which focus on certain HITL setups which may not always achieve the desired ethical, legal, and sociotechnical outcomes. We suggest areas where the law should recognise the effectiveness of different HITL setups and assign responsibility in these contexts, avoiding unnecessary and unproductive human "scapegoating". Overall, we show how HITL setups involve many technical design decisions, and can be prone to failures which are often out of the humans' control. This opens up a new analytic perspective on the challenges arising in the creation of HITL setups, helping inform AI developers and lawmakers on designing HITL to better achieve their desired outcomes.
△ Less
Submitted 15 May, 2025;
originally announced May 2025.
-
From Interaction to Collaboration: How Hybrid Intelligence Enhances Chatbot Feedback
Authors:
Janet Rafner,
Ryan Q. Guloy,
Eden W. Wen,
Catherine M. Chiodo,
Jacob Sherson
Abstract:
Generative AI (GenAI) chatbots are becoming increasingly integrated into virtual assistant technologies, yet their success hinges on the ability to gather meaningful user feedback to improve interaction quality, system outcomes, and overall user acceptance. Successful chatbot interactions can enable organizations to build long-term relationships with their customers and users, supporting customer…
▽ More
Generative AI (GenAI) chatbots are becoming increasingly integrated into virtual assistant technologies, yet their success hinges on the ability to gather meaningful user feedback to improve interaction quality, system outcomes, and overall user acceptance. Successful chatbot interactions can enable organizations to build long-term relationships with their customers and users, supporting customer loyalty and furthering the organization's goals. This study explores the impact of two distinct narratives and feedback collection mechanisms on user engagement and feedback behavior: a standard AI-focused interaction versus a hybrid intelligence (HI) framed interaction. Initial findings indicate that while small-scale survey measures allowed for no significant differences in user willingness to leave feedback, use the system, or trust the system, participants exposed to the HI narrative statistically significantly provided more detailed feedback. These initial findings offer insights into designing effective feedback systems for GenAI virtual assistants, balancing user effort with system improvement potential.
△ Less
Submitted 8 March, 2025;
originally announced April 2025.
-
Integrators at War: Mediating in AI-assisted Resort-to-Force Decisions
Authors:
Dennis Müller,
Maurice Chiodo,
Mitja Sienknecht
Abstract:
The integration of AI systems into the military domain is changing the way war-related decisions are made. It binds together three disparate groups of actors - developers, integrators, users - and creates a relationship between these groups and the machine, embedded in the (pre-)existing organisational and system structures. In this article, we focus on the important, but often neglected, group of…
▽ More
The integration of AI systems into the military domain is changing the way war-related decisions are made. It binds together three disparate groups of actors - developers, integrators, users - and creates a relationship between these groups and the machine, embedded in the (pre-)existing organisational and system structures. In this article, we focus on the important, but often neglected, group of integrators within such a sociotechnical system. In complex human-machine configurations, integrators carry responsibility for linking the disparate groups of developers and users in the political and military system. To act as the mediating group requires a deep understanding of the other groups' activities, perspectives and norms. We thus ask which challenges and shortcomings emerge from integrating AI systems into resort-to-force (RTF) decision-making processes, and how to address them. To answer this, we proceed in three steps. First, we conceptualise the relationship between different groups of actors and AI systems as a sociotechnical system. Second, we identify challenges within such systems for human-machine teaming in RTF decisions. We focus on challenges that arise a) from the technology itself, b) from the integrators' role in the sociotechnical system, c) from the human-machine interaction. Third, we provide policy recommendations to address these shortcomings when integrating AI systems into RTF decision-making structures.
△ Less
Submitted 12 January, 2025;
originally announced January 2025.
-
Teaching Resources for Embedding Ethics in Mathematics: Exercises, Projects, and Handouts
Authors:
Maurice Chiodo,
Dennis Müller,
Rehan Shah
Abstract:
The resources compiled in this document provide an approach to embed and teach Ethics in Mathematics at the undergraduate level. We provide mathematical exercises and homework problems that teach students ethical awareness and transferable skills, for many of the standard courses in the first and second years of a university degree in mathematics or related courses with significant mathematical co…
▽ More
The resources compiled in this document provide an approach to embed and teach Ethics in Mathematics at the undergraduate level. We provide mathematical exercises and homework problems that teach students ethical awareness and transferable skills, for many of the standard courses in the first and second years of a university degree in mathematics or related courses with significant mathematical content (e.g., physics, engineering, computer science, economics, etc). In addition to the exercises, this document also contains a list of projects, essay topics, and handouts for use as final projects and in seminars. This is a living document, and additional contributions are welcome.
△ Less
Submitted 11 February, 2025; v1 submitted 12 October, 2023;
originally announced October 2023.
-
Mathematical Artifacts Have Politics: The Journey from Examples to Embedded Ethics
Authors:
Dennis Müller,
Maurice Chiodo
Abstract:
We extend Langdon Winner's idea that artifacts have politics into the realm of mathematics. To do so, we first provide a list of examples showing the existence of mathematical artifacts that have politics. In the second step, we provide an argument that shows that all mathematical artifacts have politics. We conclude by showing the implications for embedding ethics into mathematical curricula. We…
▽ More
We extend Langdon Winner's idea that artifacts have politics into the realm of mathematics. To do so, we first provide a list of examples showing the existence of mathematical artifacts that have politics. In the second step, we provide an argument that shows that all mathematical artifacts have politics. We conclude by showing the implications for embedding ethics into mathematical curricula. We show how acknowledging that mathematical artifacts have politics can help mathematicians design better exercises for their mathematics students.
△ Less
Submitted 9 August, 2023;
originally announced August 2023.
-
Manifesto for the Responsible Development of Mathematical Works -- A Tool for Practitioners and for Management
Authors:
Maurice Chiodo,
Dennis Müller
Abstract:
This manifesto has been written as a practical tool and aid for anyone carrying out, managing or influencing mathematical work. It provides insight into how to undertake and develop mathematically-powered products and services in a safe and responsible way. Rather than give a framework of objectives to achieve, we instead introduce a process that can be integrated into the common ways in which mat…
▽ More
This manifesto has been written as a practical tool and aid for anyone carrying out, managing or influencing mathematical work. It provides insight into how to undertake and develop mathematically-powered products and services in a safe and responsible way. Rather than give a framework of objectives to achieve, we instead introduce a process that can be integrated into the common ways in which mathematical products or services are created, from start to finish. This process helps address the various issues and problems that can arise for the product, the developers, the institution, and for wider society. To do this, we break down the typical procedure of mathematical development into 10 key stages; our "10 pillars for responsible development" which follow a somewhat chronological ordering of the steps, and associated challenges, that frequently occur in mathematical work. Together these 10 pillars cover issues of the entire lifecycle of a mathematical product or service, including the preparatory work required to responsibly start a project, central questions of good technical mathematics and data science, and issues of communication, deployment and follow-up maintenance specifically related to mathematical systems. This manifesto, and the pillars within it, are the culmination of 7 years of work done by us as part of the Cambridge University Ethics in Mathematics Project. These are all tried-and-tested ideas, that we have presented and used in both academic and industrial environments. In our work, we have directly seen that mathematics can be an incredible tool for good in society, but also that without careful consideration it can cause immense harm. We hope that following this manifesto will empower its readers to reduce the risk of undesirable and unwanted consequences of their mathematical work.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
