-
Position: A taxonomy for reporting and describing AI security incidents
Authors:
Lukas Bieringer,
Kevin Paeth,
Jochen Stängler,
Andreas Wespi,
Alexandre Alahi,
Kathrin Grosse
Abstract:
As AI usage becomes more ubiquitous, AI incident reporting is both practiced increasingly in industry and mandated by regulatory requirements. At the same time, it is established that AI systems are exploited in practice by a growing number of security threats. Yet, organizations and practitioners lack necessary guidance in describing AI security incidents. In this position paper, we argue that sp…
▽ More
As AI usage becomes more ubiquitous, AI incident reporting is both practiced increasingly in industry and mandated by regulatory requirements. At the same time, it is established that AI systems are exploited in practice by a growing number of security threats. Yet, organizations and practitioners lack necessary guidance in describing AI security incidents. In this position paper, we argue that specific taxonomies are required to describe and report security incidents of AI systems. In other words, existing frameworks for either non-AI security or generic AI safety incident reporting are insufficient to capture the specific properties of AI security. To demonstrate our position, we offer an AI security incident taxonomy and highlight relevant properties, such as machine readability and integration with existing frameworks. We have derived this proposal from interviews with experts, aiming for standardized reporting of AI security incidents, which meets the requirements of affected stakeholder groups. We hope that this taxonomy sparks discussions and eventually allows the sharing of AI security incidents across organizations, enabling more secure AI.
△ Less
Submitted 26 February, 2025; v1 submitted 19 December, 2024;
originally announced December 2024.
-
Lessons for Editors of AI Incidents from the AI Incident Database
Authors:
Kevin Paeth,
Daniel Atherton,
Nikiforos Pittaras,
Heather Frase,
Sean McGregor
Abstract:
As artificial intelligence (AI) systems become increasingly deployed across the world, they are also increasingly implicated in AI incidents - harm events to individuals and society. As a result, industry, civil society, and governments worldwide are developing best practices and regulations for monitoring and analyzing AI incidents. The AI Incident Database (AIID) is a project that catalogs AI in…
▽ More
As artificial intelligence (AI) systems become increasingly deployed across the world, they are also increasingly implicated in AI incidents - harm events to individuals and society. As a result, industry, civil society, and governments worldwide are developing best practices and regulations for monitoring and analyzing AI incidents. The AI Incident Database (AIID) is a project that catalogs AI incidents and supports further research by providing a platform to classify incidents for different operational and research-oriented goals. This study reviews the AIID's dataset of 750+ AI incidents and two independent taxonomies applied to these incidents to identify common challenges to indexing and analyzing AI incidents. We find that certain patterns of AI incidents present structural ambiguities that challenge incident databasing and explore how epistemic uncertainty in AI incident reporting is unavoidable. We therefore report mitigations to make incident processes more robust to uncertainty related to cause, extent of harm, severity, or technical details of implicated systems. With these findings, we discuss how to develop future AI incident reporting practices.
△ Less
Submitted 24 September, 2024;
originally announced September 2024.
-
Indexing AI Risks with Incidents, Issues, and Variants
Authors:
Sean McGregor,
Kevin Paeth,
Khoa Lam
Abstract:
Two years after publicly launching the AI Incident Database (AIID) as a collection of harms or near harms produced by AI in the world, a backlog of "issues" that do not meet its incident ingestion criteria have accumulated in its review queue. Despite not passing the database's current criteria for incidents, these issues advance human understanding of where AI presents the potential for harm. Sim…
▽ More
Two years after publicly launching the AI Incident Database (AIID) as a collection of harms or near harms produced by AI in the world, a backlog of "issues" that do not meet its incident ingestion criteria have accumulated in its review queue. Despite not passing the database's current criteria for incidents, these issues advance human understanding of where AI presents the potential for harm. Similar to databases in aviation and computer security, the AIID proposes to adopt a two-tiered system for indexing AI incidents (i.e., a harm or near harm event) and issues (i.e., a risk of a harm event). Further, as some machine learning-based systems will sometimes produce a large number of incidents, the notion of an incident "variant" is introduced. These proposed changes mark the transition of the AIID to a new version in response to lessons learned from editing 2,000+ incident reports and additional reports that fall under the new category of "issue."
△ Less
Submitted 18 November, 2022;
originally announced November 2022.
