-
Third-party compliance reviews for frontier AI safety frameworks
Authors:
Aidan Homewood,
Sophie Williams,
Noemi Dreksler,
John Lidiard,
Malcolm Murray,
Lennart Heim,
Marta Ziosi,
Seán Ó hÉigeartaigh,
Michael Chen,
Kevin Wei,
Christoph Winter,
Miles Brundage,
Ben Garfinkel,
Jonas Schuett
Abstract:
Safety frameworks have emerged as a best practice for managing risks from frontier artificial intelligence (AI) systems. However, it may be difficult for stakeholders to know if companies are adhering to their frameworks. This paper explores a potential solution: third-party compliance reviews. During a third-party compliance review, an independent external party assesses whether a frontier AI com…
▽ More
Safety frameworks have emerged as a best practice for managing risks from frontier artificial intelligence (AI) systems. However, it may be difficult for stakeholders to know if companies are adhering to their frameworks. This paper explores a potential solution: third-party compliance reviews. During a third-party compliance review, an independent external party assesses whether a frontier AI company is complying with its safety framework. First, we discuss the main benefits and challenges of such reviews. On the one hand, they can increase compliance with safety frameworks and provide assurance to internal and external stakeholders. On the other hand, they can create information security risks, impose additional cost burdens, and cause reputational damage, but these challenges can be partially mitigated by drawing on best practices from other industries. Next, we answer practical questions about third-party compliance reviews, namely: (1) Who could conduct the review? (2) What information sources could the reviewer consider? (3) How could compliance with the safety framework be assessed? (4) What information about the review could be disclosed externally? (5) How could the findings guide development and deployment actions? (6) When could the reviews be conducted? For each question, we evaluate a set of plausible options. Finally, we suggest "minimalist", "more ambitious", and "comprehensive" approaches for each question that a frontier AI company could adopt.
△ Less
Submitted 2 May, 2025;
originally announced May 2025.
-
On Regulating Downstream AI Developers
Authors:
Sophie Williams,
Jonas Schuett,
Markus Anderljung
Abstract:
Foundation models - models trained on broad data that can be adapted to a wide range of downstream tasks - can pose significant risks, ranging from intimate image abuse, cyberattacks, to bioterrorism. To reduce these risks, policymakers are starting to impose obligations on the developers of these models. However, downstream developers - actors who fine-tune or otherwise modify foundational models…
▽ More
Foundation models - models trained on broad data that can be adapted to a wide range of downstream tasks - can pose significant risks, ranging from intimate image abuse, cyberattacks, to bioterrorism. To reduce these risks, policymakers are starting to impose obligations on the developers of these models. However, downstream developers - actors who fine-tune or otherwise modify foundational models - can create or amplify risks by improving a model's capabilities or compromising its safety features. This can make rules on upstream developers ineffective. One way to address this issue could be to impose direct obligations on downstream developers. However, since downstream developers are numerous, diverse, and rapidly growing in number, such direct regulation may be both practically challenging and stifling to innovation. A different approach would be to require upstream developers to mitigate downstream modification risks (e.g. by restricting what modifications can be made). Another approach would be to use alternative policy tools (e.g. clarifying how existing tort law applies to downstream developers or issuing voluntary guidance to help mitigate downstream modification risks). We expect that regulation on upstream developers to mitigate downstream modification risks will be necessary. Although further work is needed, regulation of downstream developers may also be warranted where they retain the ability to increase risk to an unacceptable level.
△ Less
Submitted 14 March, 2025;
originally announced March 2025.
-
Safety case template for frontier AI: A cyber inability argument
Authors:
Arthur Goemans,
Marie Davidsen Buhl,
Jonas Schuett,
Tomek Korbak,
Jessica Wang,
Benjamin Hilton,
Geoffrey Irving
Abstract:
Frontier artificial intelligence (AI) systems pose increasing risks to society, making it essential for developers to provide assurances about their safety. One approach to offering such assurances is through a safety case: a structured, evidence-based argument aimed at demonstrating why the risk associated with a safety-critical system is acceptable. In this article, we propose a safety case temp…
▽ More
Frontier artificial intelligence (AI) systems pose increasing risks to society, making it essential for developers to provide assurances about their safety. One approach to offering such assurances is through a safety case: a structured, evidence-based argument aimed at demonstrating why the risk associated with a safety-critical system is acceptable. In this article, we propose a safety case template for offensive cyber capabilities. We illustrate how developers could argue that a model does not have capabilities posing unacceptable cyber risks by breaking down the main claim into progressively specific sub-claims, each supported by evidence. In our template, we identify a number of risk models, derive proxy tasks from the risk models, define evaluation settings for the proxy tasks, and connect those with evaluation results. Elements of current frontier safety techniques - such as risk models, proxy tasks, and capability evaluations - use implicit arguments for overall system safety. This safety case template integrates these elements using the Claims Arguments Evidence (CAE) framework in order to make safety arguments coherent and explicit. While uncertainties around the specifics remain, this template serves as a proof of concept, aiming to foster discussion on AI safety cases and advance AI assurance.
△ Less
Submitted 12 November, 2024;
originally announced November 2024.
-
Safety cases for frontier AI
Authors:
Marie Davidsen Buhl,
Gaurav Sett,
Leonie Koessler,
Jonas Schuett,
Markus Anderljung
Abstract:
As frontier artificial intelligence (AI) systems become more capable, it becomes more important that developers can explain why their systems are sufficiently safe. One way to do so is via safety cases: reports that make a structured argument, supported by evidence, that a system is safe enough in a given operational context. Safety cases are already common in other safety-critical industries such…
▽ More
As frontier artificial intelligence (AI) systems become more capable, it becomes more important that developers can explain why their systems are sufficiently safe. One way to do so is via safety cases: reports that make a structured argument, supported by evidence, that a system is safe enough in a given operational context. Safety cases are already common in other safety-critical industries such as aviation and nuclear power. In this paper, we explain why they may also be a useful tool in frontier AI governance, both in industry self-regulation and government regulation. We then discuss the practicalities of safety cases, outlining how to produce a frontier AI safety case and discussing what still needs to happen before safety cases can substantially inform decisions.
△ Less
Submitted 28 October, 2024;
originally announced October 2024.
-
Partial regularity for variational integrals with Morrey-Hölder zero-order terms, and the limit exponent in Massari's regularity theorem
Authors:
Thomas Schmidt,
Jule Helena Schütt
Abstract:
We revisit the partial $\mathrm{C}^{1,α}$ regularity theory for minimizers of non-parametric integrals with emphasis on sharp dependence of the Hölder exponent $α$ on structural assumptions for general zero-order terms. A particular case of our conclusions carries over to the parametric setting of Massari's regularity theorem for prescribed-mean-curvature hypersurfaces and there confirms optimal r…
▽ More
We revisit the partial $\mathrm{C}^{1,α}$ regularity theory for minimizers of non-parametric integrals with emphasis on sharp dependence of the Hölder exponent $α$ on structural assumptions for general zero-order terms. A particular case of our conclusions carries over to the parametric setting of Massari's regularity theorem for prescribed-mean-curvature hypersurfaces and there confirms optimal regularity up to the limit exponent.
△ Less
Submitted 7 March, 2025; v1 submitted 4 October, 2024;
originally announced October 2024.
-
A Grading Rubric for AI Safety Frameworks
Authors:
Jide Alaga,
Jonas Schuett,
Markus Anderljung
Abstract:
Over the past year, artificial intelligence (AI) companies have been increasingly adopting AI safety frameworks. These frameworks outline how companies intend to keep the potential risks associated with developing and deploying frontier AI systems to an acceptable level. Major players like Anthropic, OpenAI, and Google DeepMind have already published their frameworks, while another 13 companies ha…
▽ More
Over the past year, artificial intelligence (AI) companies have been increasingly adopting AI safety frameworks. These frameworks outline how companies intend to keep the potential risks associated with developing and deploying frontier AI systems to an acceptable level. Major players like Anthropic, OpenAI, and Google DeepMind have already published their frameworks, while another 13 companies have signaled their intent to release similar frameworks by February 2025. Given their central role in AI companies' efforts to identify and address unacceptable risks from their systems, AI safety frameworks warrant significant scrutiny. To enable governments, academia, and civil society to pass judgment on these frameworks, this paper proposes a grading rubric. The rubric consists of seven evaluation criteria and 21 indicators that concretize the criteria. Each criterion can be graded on a scale from A (gold standard) to F (substandard). The paper also suggests three methods for applying the rubric: surveys, Delphi studies, and audits. The purpose of the grading rubric is to enable nuanced comparisons between frameworks, identify potential areas of improvement, and promote a race to the top in responsible AI development.
△ Less
Submitted 13 September, 2024;
originally announced September 2024.
-
From Principles to Rules: A Regulatory Approach for Frontier AI
Authors:
Jonas Schuett
Abstract:
Several jurisdictions are starting to regulate frontier artificial intelligence (AI) systems, i.e. general-purpose AI systems that match or exceed the capabilities present in the most advanced systems. To reduce risks from these systems, regulators may require frontier AI developers to adopt safety measures. The requirements could be formulated as high-level principles (e.g. 'AI systems should be…
▽ More
Several jurisdictions are starting to regulate frontier artificial intelligence (AI) systems, i.e. general-purpose AI systems that match or exceed the capabilities present in the most advanced systems. To reduce risks from these systems, regulators may require frontier AI developers to adopt safety measures. The requirements could be formulated as high-level principles (e.g. 'AI systems should be safe and secure') or specific rules (e.g. 'AI systems must be evaluated for dangerous model capabilities following the protocol set forth in...'). These regulatory approaches, known as 'principle-based' and 'rule-based' regulation, have complementary strengths and weaknesses. While specific rules provide more certainty and are easier to enforce, they can quickly become outdated and lead to box-ticking. Conversely, while high-level principles provide less certainty and are more costly to enforce, they are more adaptable and more appropriate in situations where the regulator is unsure exactly what behavior would best advance a given regulatory objective. However, rule-based and principle-based regulation are not binary options. Policymakers must choose a point on the spectrum between them, recognizing that the right level of specificity may vary between requirements and change over time. We recommend that policymakers should initially (1) mandate adherence to high-level principles for safe frontier AI development and deployment, (2) ensure that regulators closely oversee how developers comply with these principles, and (3) urgently build up regulatory capacity. Over time, the approach should likely become more rule-based. Our recommendations are based on a number of assumptions, including (A) risks from frontier AI systems are poorly understood and rapidly evolving, (B) many safety practices are still nascent, and (C) frontier AI developers are best placed to innovate on safety practices.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Risk thresholds for frontier AI
Authors:
Leonie Koessler,
Jonas Schuett,
Markus Anderljung
Abstract:
Frontier artificial intelligence (AI) systems could pose increasing risks to public safety and security. But what level of risk is acceptable? One increasingly popular approach is to define capability thresholds, which describe AI capabilities beyond which an AI system is deemed to pose too much risk. A more direct approach is to define risk thresholds that simply state how much risk would be too…
▽ More
Frontier artificial intelligence (AI) systems could pose increasing risks to public safety and security. But what level of risk is acceptable? One increasingly popular approach is to define capability thresholds, which describe AI capabilities beyond which an AI system is deemed to pose too much risk. A more direct approach is to define risk thresholds that simply state how much risk would be too much. For instance, they might state that the likelihood of cybercriminals using an AI system to cause X amount of economic damage must not increase by more than Y percentage points. The main upside of risk thresholds is that they are more principled than capability thresholds, but the main downside is that they are more difficult to evaluate reliably. For this reason, we currently recommend that companies (1) define risk thresholds to provide a principled foundation for their decision-making, (2) use these risk thresholds to help set capability thresholds, and then (3) primarily rely on capability thresholds to make their decisions. Regulators should also explore the area because, ultimately, they are the most legitimate actors to define risk thresholds. If AI risk estimates become more reliable, risk thresholds should arguably play an increasingly direct role in decision-making.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Authors:
Markus Anderljung,
Everett Thornton Smith,
Joe O'Brien,
Lisa Soder,
Benjamin Bucknall,
Emma Bluemke,
Jonas Schuett,
Robert Trager,
Lacey Strahm,
Rumman Chowdhury
Abstract:
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Inv…
▽ More
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Involving outside actors in the evaluation of these systems - what we term 'external scrutiny' - via red-teaming, auditing, and external researcher access, offers a solution. Though there are encouraging signs of increasing external scrutiny of frontier LLMs, its success is not assured. In this paper, we survey six requirements for effective external scrutiny of frontier AI systems and organize them under the ASPIRE framework: Access, Searching attitude, Proportionality to the risks, Independence, Resources, and Expertise. We then illustrate how external scrutiny might function throughout the AI lifecycle and offer recommendations to policymakers.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives
Authors:
Elizabeth Seger,
Noemi Dreksler,
Richard Moulange,
Emily Dardaman,
Jonas Schuett,
K. Wei,
Christoph Winter,
Mackenzie Arnold,
Seán Ó hÉigeartaigh,
Anton Korinek,
Markus Anderljung,
Ben Bucknall,
Alan Chan,
Eoghan Stafford,
Leonie Koessler,
Aviv Ovadya,
Ben Garfinkel,
Emma Bluemke,
Michael Aird,
Patrick Levermore,
Julian Hazell,
Abhishek Gupta
Abstract:
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling ex…
▽ More
Recent decisions by leading AI labs to either open-source their models or to restrict access to their models has sparked debate about whether, and how, increasingly capable AI models should be shared. Open-sourcing in AI typically refers to making model architecture and weights freely and publicly accessible for anyone to modify, study, build on, and use. This offers advantages such as enabling external oversight, accelerating progress, and decentralizing control over AI development and use. However, it also presents a growing potential for misuse and unintended consequences. This paper offers an examination of the risks and benefits of open-sourcing highly capable foundation models. While open-sourcing has historically provided substantial net benefits for most software and AI development processes, we argue that for some highly capable foundation models likely to be developed in the near future, open-sourcing may pose sufficiently extreme risks to outweigh the benefits. In such a case, highly capable foundation models should not be open-sourced, at least not initially. Alternative strategies, including non-open-source model sharing options, are explored. The paper concludes with recommendations for developers, standard-setting bodies, and governments for establishing safe and responsible model sharing practices and preserving open-source benefits where safe.
△ Less
Submitted 29 September, 2023;
originally announced November 2023.
-
Coordinated pausing: An evaluation-based coordination scheme for frontier AI developers
Authors:
Jide Alaga,
Jonas Schuett
Abstract:
As artificial intelligence (AI) models are scaled up, new capabilities can emerge unintentionally and unpredictably, some of which might be dangerous. In response, dangerous capabilities evaluations have emerged as a new risk assessment tool. But what should frontier AI developers do if sufficiently dangerous capabilities are in fact discovered? This paper focuses on one possible response: coordin…
▽ More
As artificial intelligence (AI) models are scaled up, new capabilities can emerge unintentionally and unpredictably, some of which might be dangerous. In response, dangerous capabilities evaluations have emerged as a new risk assessment tool. But what should frontier AI developers do if sufficiently dangerous capabilities are in fact discovered? This paper focuses on one possible response: coordinated pausing. It proposes an evaluation-based coordination scheme that consists of five main steps: (1) Frontier AI models are evaluated for dangerous capabilities. (2) Whenever, and each time, a model fails a set of evaluations, the developer pauses certain research and development activities. (3) Other developers are notified whenever a model with dangerous capabilities has been discovered. They also pause related research and development activities. (4) The discovered capabilities are analyzed and adequate safety precautions are put in place. (5) Developers only resume their paused activities if certain safety thresholds are reached. The paper also discusses four concrete versions of that scheme. In the first version, pausing is completely voluntary and relies on public pressure on developers. In the second version, participating developers collectively agree to pause under certain conditions. In the third version, a single auditor evaluates models of multiple developers who agree to pause if any model fails a set of evaluations. In the fourth version, developers are legally required to run evaluations and pause if dangerous capabilities are discovered. Finally, the paper discusses the desirability and feasibility of our proposed coordination scheme. It concludes that coordinated pausing is a promising mechanism for tackling emerging risks from frontier AI models. However, a number of practical and legal obstacles need to be overcome, especially how to avoid violations of antitrust law.
△ Less
Submitted 30 September, 2023;
originally announced October 2023.
-
Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industries
Authors:
Leonie Koessler,
Jonas Schuett
Abstract:
Companies like OpenAI, Google DeepMind, and Anthropic have the stated goal of building artificial general intelligence (AGI) - AI systems that perform as well as or better than humans on a wide variety of cognitive tasks. However, there are increasing concerns that AGI would pose catastrophic risks. In light of this, AGI companies need to drastically improve their risk management practices. To sup…
▽ More
Companies like OpenAI, Google DeepMind, and Anthropic have the stated goal of building artificial general intelligence (AGI) - AI systems that perform as well as or better than humans on a wide variety of cognitive tasks. However, there are increasing concerns that AGI would pose catastrophic risks. In light of this, AGI companies need to drastically improve their risk management practices. To support such efforts, this paper reviews popular risk assessment techniques from other safety-critical industries and suggests ways in which AGI companies could use them to assess catastrophic risks from AI. The paper discusses three risk identification techniques (scenario analysis, fishbone method, and risk typologies and taxonomies), five risk analysis techniques (causal mapping, Delphi technique, cross-impact analysis, bow tie analysis, and system-theoretic process analysis), and two risk evaluation techniques (checklists and risk matrices). For each of them, the paper explains how they work, suggests ways in which AGI companies could use them, discusses their benefits and limitations, and makes recommendations. Finally, the paper discusses when to conduct risk assessments, when to use which technique, and how to use any of them. The reviewed techniques will be obvious to risk management professionals in other industries. And they will not be sufficient to assess catastrophic risks from AI. However, AGI companies should not skip the straightforward step of reviewing best practices from other industries.
△ Less
Submitted 17 July, 2023;
originally announced July 2023.
-
Frontier AI Regulation: Managing Emerging Risks to Public Safety
Authors:
Markus Anderljung,
Joslyn Barnhart,
Anton Korinek,
Jade Leung,
Cullen O'Keefe,
Jess Whittlestone,
Shahar Avin,
Miles Brundage,
Justin Bullock,
Duncan Cass-Beggs,
Ben Chang,
Tantum Collins,
Tim Fist,
Gillian Hadfield,
Alan Hayes,
Lewis Ho,
Sara Hooker,
Eric Horvitz,
Noam Kolt,
Jonas Schuett,
Yonadav Shavit,
Divya Siddarth,
Robert Trager,
Kevin Wolf
Abstract:
Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilit…
▽ More
Advanced AI models hold the promise of tremendous benefits for humanity, but society needs to proactively manage the accompanying risks. In this paper, we focus on what we term "frontier AI" models: highly capable foundation models that could possess dangerous capabilities sufficient to pose severe risks to public safety. Frontier AI models pose a distinct regulatory challenge: dangerous capabilities can arise unexpectedly; it is difficult to robustly prevent a deployed model from being misused; and, it is difficult to stop a model's capabilities from proliferating broadly. To address these challenges, at least three building blocks for the regulation of frontier models are needed: (1) standard-setting processes to identify appropriate requirements for frontier AI developers, (2) registration and reporting requirements to provide regulators with visibility into frontier AI development processes, and (3) mechanisms to ensure compliance with safety standards for the development and deployment of frontier AI models. Industry self-regulation is an important first step. However, wider societal discussions and government intervention will be needed to create standards and to ensure compliance with them. We consider several options to this end, including granting enforcement powers to supervisory authorities and licensure regimes for frontier AI models. Finally, we propose an initial set of safety standards. These include conducting pre-deployment risk assessments; external scrutiny of model behavior; using risk assessments to inform deployment decisions; and monitoring and responding to new information about model capabilities and uses post-deployment. We hope this discussion contributes to the broader conversation on how to balance public safety risks and innovation benefits from advances at the frontier of AI development.
△ Less
Submitted 7 November, 2023; v1 submitted 6 July, 2023;
originally announced July 2023.
-
Frontier AI developers need an internal audit function
Authors:
Jonas Schuett
Abstract:
This article argues that frontier artificial intelligence (AI) developers need an internal audit function. First, it describes the role of internal audit in corporate governance: internal audit evaluates the adequacy and effectiveness of a company's risk management, control, and governance processes. It is organizationally independent from senior management and reports directly to the board of dir…
▽ More
This article argues that frontier artificial intelligence (AI) developers need an internal audit function. First, it describes the role of internal audit in corporate governance: internal audit evaluates the adequacy and effectiveness of a company's risk management, control, and governance processes. It is organizationally independent from senior management and reports directly to the board of directors, typically its audit committee. In the IIA's Three Lines Model, internal audit serves as the third line and is responsible for providing assurance to the board, while the Combined Assurance Framework highlights the need to coordinate the activities of internal and external assurance providers. Next, the article provides an overview of key governance challenges in frontier AI development: dangerous capabilities can arise unpredictably and undetected; it is difficult to prevent a deployed model from causing harm; frontier models can proliferate rapidly; it is inherently difficult to assess frontier AI risks; and frontier AI developers do not seem to follow best practices in risk governance. Finally, the article discusses how an internal audit function could address some of these challenges: internal audit could identify ineffective risk management practices; it could ensure that the board of directors has a more accurate understanding of the current level of risk and the adequacy of the developer's risk management practices; and it could serve as a contact point for whistleblowers. In light of rapid progress in AI research and development, frontier AI developers need to strengthen their risk governance. Instead of reinventing the wheel, they should follow existing best practices. While this might not be sufficient, they should not skip this obvious first step.
△ Less
Submitted 5 October, 2024; v1 submitted 26 May, 2023;
originally announced May 2023.
-
Towards best practices in AGI safety and governance: A survey of expert opinion
Authors:
Jonas Schuett,
Noemi Dreksler,
Markus Anderljung,
David McCaffary,
Lennart Heim,
Emma Bluemke,
Ben Garfinkel
Abstract:
A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve or exceed human performance across a wide range of cognitive tasks. In pursuing this goal, they may develop and deploy AI systems that pose particularly significant risks. While they have already taken some measures to…
▽ More
A number of leading AI companies, including OpenAI, Google DeepMind, and Anthropic, have the stated goal of building artificial general intelligence (AGI) - AI systems that achieve or exceed human performance across a wide range of cognitive tasks. In pursuing this goal, they may develop and deploy AI systems that pose particularly significant risks. While they have already taken some measures to mitigate these risks, best practices have not yet emerged. To support the identification of best practices, we sent a survey to 92 leading experts from AGI labs, academia, and civil society and received 51 responses. Participants were asked how much they agreed with 50 statements about what AGI labs should do. Our main finding is that participants, on average, agreed with all of them. Many statements received extremely high levels of agreement. For example, 98% of respondents somewhat or strongly agreed that AGI labs should conduct pre-deployment risk assessments, dangerous capabilities evaluations, third-party model audits, safety restrictions on model usage, and red teaming. Ultimately, our list of statements may serve as a helpful foundation for efforts to develop best practices, standards, and regulations for AGI labs.
△ Less
Submitted 11 May, 2023;
originally announced May 2023.
-
The Optimal Hölder Exponent in Massari's Regularity Theorem
Authors:
Thomas Schmidt,
Jule Helena Schütt
Abstract:
We determine the optimal Hölder exponent in Massari's regularity theorem for sets with variational mean curvature in $\mathrm{L}^p$. In fact, we obtain regularity with improved exponents and at the same time provide sharp counterexamples.
We determine the optimal Hölder exponent in Massari's regularity theorem for sets with variational mean curvature in $\mathrm{L}^p$. In fact, we obtain regularity with improved exponents and at the same time provide sharp counterexamples.
△ Less
Submitted 24 March, 2025; v1 submitted 21 April, 2023;
originally announced April 2023.
-
How to design an AI ethics board
Authors:
Jonas Schuett,
Anka Reuel,
Alexis Carlier
Abstract:
Organizations that develop and deploy artificial intelligence (AI) systems need to take measures to reduce the associated risks. In this paper, we examine how AI companies could design an AI ethics board in a way that reduces risks from AI. We identify five high-level design choices: (1) What responsibilities should the board have? (2) What should its legal structure be? (3) Who should sit on the…
▽ More
Organizations that develop and deploy artificial intelligence (AI) systems need to take measures to reduce the associated risks. In this paper, we examine how AI companies could design an AI ethics board in a way that reduces risks from AI. We identify five high-level design choices: (1) What responsibilities should the board have? (2) What should its legal structure be? (3) Who should sit on the board? (4) How should it make decisions and should its decisions be binding? (5) What resources does it need? We break down each of these questions into more specific sub-questions, list options, and discuss how different design choices affect the board's ability to reduce risks from AI. Several failures have shown that designing an AI ethics board can be challenging. This paper provides a toolbox that can help AI companies to overcome these challenges.
△ Less
Submitted 14 April, 2023;
originally announced April 2023.
-
Auditing large language models: a three-layered approach
Authors:
Jakob Mökander,
Jonas Schuett,
Hannah Rose Kirk,
Luciano Floridi
Abstract:
Large language models (LLMs) represent a major advance in artificial intelligence (AI) research. However, the widespread use of LLMs is also coupled with significant ethical and social challenges. Previous research has pointed towards auditing as a promising governance mechanism to help ensure that AI systems are designed and deployed in ways that are ethical, legal, and technically robust. Howeve…
▽ More
Large language models (LLMs) represent a major advance in artificial intelligence (AI) research. However, the widespread use of LLMs is also coupled with significant ethical and social challenges. Previous research has pointed towards auditing as a promising governance mechanism to help ensure that AI systems are designed and deployed in ways that are ethical, legal, and technically robust. However, existing auditing procedures fail to address the governance challenges posed by LLMs, which display emergent capabilities and are adaptable to a wide range of downstream tasks. In this article, we address that gap by outlining a novel blueprint for how to audit LLMs. Specifically, we propose a three-layered approach, whereby governance audits (of technology providers that design and disseminate LLMs), model audits (of LLMs after pre-training but prior to their release), and application audits (of applications based on LLMs) complement and inform each other. We show how audits, when conducted in a structured and coordinated manner on all three levels, can be a feasible and effective mechanism for identifying and managing some of the ethical and social risks posed by LLMs. However, it is important to remain realistic about what auditing can reasonably be expected to achieve. Therefore, we discuss the limitations not only of our three-layered approach but also of the prospect of auditing LLMs at all. Ultimately, this article seeks to expand the methodological toolkit available to technology providers and policymakers who wish to analyse and evaluate LLMs from technical, ethical, and legal perspectives.
△ Less
Submitted 27 June, 2023; v1 submitted 16 February, 2023;
originally announced February 2023.
-
Three lines of defense against risks from AI
Authors:
Jonas Schuett
Abstract:
Organizations that develop and deploy artificial intelligence (AI) systems need to manage the associated risks - for economic, legal, and ethical reasons. However, it is not always clear who is responsible for AI risk management. The Three Lines of Defense (3LoD) model, which is considered best practice in many industries, might offer a solution. It is a risk management framework that helps organi…
▽ More
Organizations that develop and deploy artificial intelligence (AI) systems need to manage the associated risks - for economic, legal, and ethical reasons. However, it is not always clear who is responsible for AI risk management. The Three Lines of Defense (3LoD) model, which is considered best practice in many industries, might offer a solution. It is a risk management framework that helps organizations to assign and coordinate risk management roles and responsibilities. In this article, I suggest ways in which AI companies could implement the model. I also discuss how the model could help reduce risks from AI: it could identify and close gaps in risk coverage, increase the effectiveness of risk management practices, and enable the board of directors to oversee management more effectively. The article is intended to inform decision-makers at leading AI companies, regulators, and standard-setting bodies.
△ Less
Submitted 16 December, 2022;
originally announced December 2022.
-
Risk management in the Artificial Intelligence Act
Authors:
Jonas Schuett
Abstract:
The proposed EU AI Act is the first comprehensive attempt to regulate AI in a major jurisdiction. This article analyses Article 9, the key risk management provision in the AI Act. It gives an overview of the regulatory concept behind Article 9, determines its purpose and scope of application, offers a comprehensive interpretation of the specific risk management requirements, and outlines ways in w…
▽ More
The proposed EU AI Act is the first comprehensive attempt to regulate AI in a major jurisdiction. This article analyses Article 9, the key risk management provision in the AI Act. It gives an overview of the regulatory concept behind Article 9, determines its purpose and scope of application, offers a comprehensive interpretation of the specific risk management requirements, and outlines ways in which the requirements can be enforced. This article is written with the aim of helping providers of high-risk systems comply with the requirements set out in Article 9. In addition, it can inform revisions of the current draft of the AI Act and efforts to develop harmonised standards on AI risk management.
△ Less
Submitted 3 December, 2022;
originally announced December 2022.
-
AI Certification: Advancing Ethical Practice by Reducing Information Asymmetries
Authors:
Peter Cihon,
Moritz J. Kleinaltenkamp,
Jonas Schuett,
Seth D. Baum
Abstract:
As artificial intelligence (AI) systems are increasingly deployed, principles for ethical AI are also proliferating. Certification offers a method to both incentivize adoption of these principles and substantiate that they have been implemented in practice. This paper draws from management literature on certification and reviews current AI certification programs and proposals. Successful programs…
▽ More
As artificial intelligence (AI) systems are increasingly deployed, principles for ethical AI are also proliferating. Certification offers a method to both incentivize adoption of these principles and substantiate that they have been implemented in practice. This paper draws from management literature on certification and reviews current AI certification programs and proposals. Successful programs rely on both emerging technical methods and specific design considerations. In order to avoid two common failures of certification, program designs should ensure that the symbol of the certification is substantially implemented in practice and that the program achieves its stated goals. The review indicates that the field currently focuses on self-certification and third-party certification of systems, individuals, and organizations - to the exclusion of process management certifications. Additionally, the paper considers prospects for future AI certification programs. Ongoing changes in AI technology suggest that AI certification regimes should be designed to emphasize governance criteria of enduring value, such as ethics training for AI developers, and to adjust technical criteria as the technology changes. Overall, certification can play a valuable mix in the portfolio of AI governance tools.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
Inverse Solidification Induced by Active Janus Particles
Authors:
Tao Huang,
Vyacheslav R. Misko,
Sophie Gobeil,
Xu Wang,
Franco Nori,
Julian Schütt,
Jürgen Fassbender,
Gianaurelio Cuniberti,
Denys Makarov,
Larysa Baraban
Abstract:
Crystals melt when thermal excitations or the concentration of defects in the lattice is sufficiently high. Upon melting, the crystalline long-range order vanishes, turning the solid to a fluid. In contrast to this classical scenario of solid melting, here we demonstrate a counter-intuitive behavior of the occurrence of crystalline long-range order in an initially disordered matrix. This unusual s…
▽ More
Crystals melt when thermal excitations or the concentration of defects in the lattice is sufficiently high. Upon melting, the crystalline long-range order vanishes, turning the solid to a fluid. In contrast to this classical scenario of solid melting, here we demonstrate a counter-intuitive behavior of the occurrence of crystalline long-range order in an initially disordered matrix. This unusual solidification is demonstrated in a system of passive colloidal particles accommodating chemically active defects -- photocatalytic Janus particles. The observed crystallization occurs when the amount of active-defect-induced fluctuations (which is the measure of the effective temperature) reaches critical value. The driving mechanism behind this unusual behavior is purely internal and resembles a blast-induced solidification. Here the role of "internal micro-blasts" is played by the photochemical activity of defects residing in the colloidal matrix. The defect-induced solidification occurs under non-equilibrium conditions: the resulting solid exists as long as a constant supply of energy in the form of ion flow is provided by the catalytic photochemical reaction at the surface of active Janus particle defects. Our findings could be useful for understanding of the phase transitions of matter under extreme conditions far from thermodynamic equilibrium.
△ Less
Submitted 8 October, 2020;
originally announced October 2020.
-
Defining the scope of AI regulations
Authors:
Jonas Schuett
Abstract:
The paper argues that the material scope of AI regulations should not rely on the term "artificial intelligence (AI)". The argument is developed by proposing a number of requirements for legal definitions, surveying existing AI definitions, and then discussing the extent to which they meet the proposed requirements. It is shown that existing definitions of AI do not meet the most important require…
▽ More
The paper argues that the material scope of AI regulations should not rely on the term "artificial intelligence (AI)". The argument is developed by proposing a number of requirements for legal definitions, surveying existing AI definitions, and then discussing the extent to which they meet the proposed requirements. It is shown that existing definitions of AI do not meet the most important requirements for legal definitions. Next, the paper argues that a risk-based approach would be preferable. Rather than using the term AI, policy makers should focus on the specific risks they want to reduce. It is shown that the requirements for legal definitions can be better met by defining the main sources of relevant risks: certain technical approaches (e.g. reinforcement learning), applications (e.g. facial recognition), and capabilities (e.g. the ability to physically interact with the environment). Finally, the paper discusses the extent to which this approach can also be applied to more advanced AI systems.
△ Less
Submitted 20 November, 2022; v1 submitted 26 August, 2019;
originally announced September 2019.
-
Infinite-Dimensional Supermanifolds via Multilinear Bundles
Authors:
Jakob Schütt
Abstract:
In this paper, we provide an accessible introduction to the theory of locally convex supermanifolds in the categorical approach. In this setting, a supermanifold is a functor $\mathcal{M}\colon\mathbf{Gr}\to\mathbf{Man}$ from the category of Grassmann algebras to the category of locally convex manifolds that has certain local models, forming something akin to an atlas. We give a mostly self-contai…
▽ More
In this paper, we provide an accessible introduction to the theory of locally convex supermanifolds in the categorical approach. In this setting, a supermanifold is a functor $\mathcal{M}\colon\mathbf{Gr}\to\mathbf{Man}$ from the category of Grassmann algebras to the category of locally convex manifolds that has certain local models, forming something akin to an atlas. We give a mostly self-contained, concrete definition of supermanifolds along these lines, closing several gaps in the literature on the way. If $Λ_n\in\mathbf{Gr}$ is the Grassmann algebra with $n$ generators, we show that $\mathcal{M}_{Λ_n}$ has the structure of a so called multilinear bundle over the base manifold $\mathcal{M}_\mathbb{R}$. We use this fact to show that the projective limit $\varprojlim_n\mathcal{M}_{Λ_n}$ exists in the category of manifolds. In fact, this gives us a faithful functor $\varprojlim\colon\mathbf{SMan}\to\mathbf{Man}$ from the category of supermanifolds to the category of manifolds. This functor respects products, commutes with the respective tangent functor and retains the respective Hausdorff property. In this way, supermanifolds can be seen as a particular kind of infinite-dimensional fiber bundles.
△ Less
Submitted 22 January, 2019; v1 submitted 12 October, 2018;
originally announced October 2018.
-
Symmetry Groups of Principal Bundles Over Non-Compact Bases
Authors:
Jakob Schuett
Abstract:
In this work, we describe how to obtain the structure of an infinite-dimensional Lie group on the group of compactly carried bundle automorphisms Autc(P) for a locally convex prinicpal bundle P over a finite-dimensional smooth sigma-compact base M. This is a generalization of previous work by Wockel, where the base M was compact. We first consider the Lie group structure on the group of compactly…
▽ More
In this work, we describe how to obtain the structure of an infinite-dimensional Lie group on the group of compactly carried bundle automorphisms Autc(P) for a locally convex prinicpal bundle P over a finite-dimensional smooth sigma-compact base M. This is a generalization of previous work by Wockel, where the base M was compact. We first consider the Lie group structure on the group of compactly carried vertical bundle morphisms Gauc(P) (in both cases "compactly carried" refers to being compactly carried on the base in a certain sense). We then introduce the Lie group structure on Autc(P) as an extension of a certain open Lie subgroup of the compactly carried diffeomorphisms Diffc(M) by the gauge group Gauc(P). We find an explicit condition on P ensuring that Gauc(P) can be equipped with a Lie group structure enabling the extension just mentioned and show that this condition is satisfied by selected classes of bundles.
△ Less
Submitted 6 November, 2013; v1 submitted 31 October, 2013;
originally announced October 2013.
-
The H1 Forward Proton Spectrometer at HERA
Authors:
P. van Esch,
M. Kapichine,
A. Morozov,
V. Spaskov,
W. Bartel,
B. List,
H. Mahlke-Krueger,
V. Schroeder,
T. Wilksen,
F. W. Buesser,
K. Geske,
O. Karschnik,
F. Niebergall,
H. Riege,
J. Schuett,
R. van Staa,
C. Wittek,
D. Dau,
D. Newton,
S. K. Kotelnikov,
A. Lebedev,
S. Rusakov,
A. Astvatsatourov,
J. Baehr,
U. Harder
, et al. (4 additional authors not shown)
Abstract:
The forward proton spectrometer is part of the H1 detector at the HERA collider. Protons with energies above 500 GeV and polar angles below 1 mrad can be detected by this spectrometer. The main detector components are scintillating fiber detectors read out by position-sensitive photo-multipliers. These detectors are housed in so-called Roman Pots which allow them to be moved close to the circula…
▽ More
The forward proton spectrometer is part of the H1 detector at the HERA collider. Protons with energies above 500 GeV and polar angles below 1 mrad can be detected by this spectrometer. The main detector components are scintillating fiber detectors read out by position-sensitive photo-multipliers. These detectors are housed in so-called Roman Pots which allow them to be moved close to the circulating proton beam. Four Roman Pot stations are located at distances between 60 m and 90 m from the interaction point.
△ Less
Submitted 20 January, 2000;
originally announced January 2000.
