-
Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effects
Authors:
Reva Schwartz,
Rumman Chowdhury,
Akash Kundu,
Heather Frase,
Marzieh Fadaee,
Tom David,
Gabriella Waters,
Afaf Taik,
Morgan Briggs,
Patrick Hall,
Shomik Jain,
Kyra Yee,
Spencer Thomas,
Sundeep Bhandari,
Paul Duncan,
Andrew Thompson,
Maya Carlyle,
Qinghua Lu,
Matthew Holmes,
Theodora Skeadas
Abstract:
Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors. AI capability evaluations can capture detail about first-order effects, such as whether immediate system outputs are accura…
▽ More
Conventional AI evaluation approaches concentrated within the AI stack exhibit systemic limitations for exploring, navigating and resolving the human and societal factors that play out in real world deployment such as in education, finance, healthcare, and employment sectors. AI capability evaluations can capture detail about first-order effects, such as whether immediate system outputs are accurate, or contain toxic, biased or stereotypical content, but AI's second-order effects, i.e. any long-term outcomes and consequences that may result from AI use in the real world, have become a significant area of interest as the technology becomes embedded in our daily lives. These secondary effects can include shifts in user behavior, societal, cultural and economic ramifications, workforce transformations, and long-term downstream impacts that may result from a broad and growing set of risks. This position paper argues that measuring the indirect and secondary effects of AI will require expansion beyond static, single-turn approaches conducted in silico to include testing paradigms that can capture what actually materializes when people use AI technology in context. Specifically, we describe the need for data and methods that can facilitate contextual awareness and enable downstream interpretation and decision making about AI's secondary effects, and recommend requirements for a new ecosystem.
△ Less
Submitted 30 May, 2025; v1 submitted 24 May, 2025;
originally announced May 2025.
-
International AI Safety Report
Authors:
Yoshua Bengio,
Sören Mindermann,
Daniel Privitera,
Tamay Besiroglu,
Rishi Bommasani,
Stephen Casper,
Yejin Choi,
Philip Fox,
Ben Garfinkel,
Danielle Goldfarb,
Hoda Heidari,
Anson Ho,
Sayash Kapoor,
Leila Khalatbari,
Shayne Longpre,
Sam Manning,
Vasilios Mavroudis,
Mantas Mazeika,
Julian Michael,
Jessica Newman,
Kwan Yee Ng,
Chinasa T. Okolo,
Deborah Raji,
Girish Sastry,
Elizabeth Seger
, et al. (71 additional authors not shown)
Abstract:
The first International AI Safety Report comprehensively synthesizes the current evidence on the capabilities, risks, and safety of advanced AI systems. The report was mandated by the nations attending the AI Safety Summit in Bletchley, UK. Thirty nations, the UN, the OECD, and the EU each nominated a representative to the report's Expert Advisory Panel. A total of 100 AI experts contributed, repr…
▽ More
The first International AI Safety Report comprehensively synthesizes the current evidence on the capabilities, risks, and safety of advanced AI systems. The report was mandated by the nations attending the AI Safety Summit in Bletchley, UK. Thirty nations, the UN, the OECD, and the EU each nominated a representative to the report's Expert Advisory Panel. A total of 100 AI experts contributed, representing diverse perspectives and disciplines. Led by the report's Chair, these independent experts collectively had full discretion over the report's content.
△ Less
Submitted 29 January, 2025;
originally announced January 2025.
-
International Scientific Report on the Safety of Advanced AI (Interim Report)
Authors:
Yoshua Bengio,
Sören Mindermann,
Daniel Privitera,
Tamay Besiroglu,
Rishi Bommasani,
Stephen Casper,
Yejin Choi,
Danielle Goldfarb,
Hoda Heidari,
Leila Khalatbari,
Shayne Longpre,
Vasilios Mavroudis,
Mantas Mazeika,
Kwan Yee Ng,
Chinasa T. Okolo,
Deborah Raji,
Theodora Skeadas,
Florian Tramèr,
Bayo Adekanmbi,
Paul Christiano,
David Dalrymple,
Thomas G. Dietterich,
Edward Felten,
Pascale Fung,
Pierre-Olivier Gourinchas
, et al. (19 additional authors not shown)
Abstract:
This is the interim publication of the first International Scientific Report on the Safety of Advanced AI. The report synthesises the scientific understanding of general-purpose AI -- AI that can perform a wide variety of tasks -- with a focus on understanding and managing its risks. A diverse group of 75 AI experts contributed to this report, including an international Expert Advisory Panel nomin…
▽ More
This is the interim publication of the first International Scientific Report on the Safety of Advanced AI. The report synthesises the scientific understanding of general-purpose AI -- AI that can perform a wide variety of tasks -- with a focus on understanding and managing its risks. A diverse group of 75 AI experts contributed to this report, including an international Expert Advisory Panel nominated by 30 countries, the EU, and the UN. Led by the Chair, these independent experts collectively had full discretion over the report's content.
The final report is available at arXiv:2501.17805
△ Less
Submitted 9 April, 2025; v1 submitted 5 November, 2024;
originally announced December 2024.
