-
Anecdoctoring: Automated Red-Teaming Across Language and Place
Authors:
Alejandro Cuevas,
Saloni Dash,
Bharat Kumar Nayak,
Dan Vann,
Madeleine I. G. Daepp
Abstract:
Disinformation is among the top risks of generative artificial intelligence (AI) misuse. Global adoption of generative AI necessitates red-teaming evaluations (i.e., systematic adversarial probing) that are robust across diverse languages and cultures, but red-teaming datasets are commonly US- and English-centric. To address this gap, we propose "anecdoctoring", a novel red-teaming approach that a…
▽ More
Disinformation is among the top risks of generative artificial intelligence (AI) misuse. Global adoption of generative AI necessitates red-teaming evaluations (i.e., systematic adversarial probing) that are robust across diverse languages and cultures, but red-teaming datasets are commonly US- and English-centric. To address this gap, we propose "anecdoctoring", a novel red-teaming approach that automatically generates adversarial prompts across languages and cultures. We collect misinformation claims from fact-checking websites in three languages (English, Spanish, and Hindi) and two geographies (US and India). We then cluster individual claims into broader narratives and characterize the resulting clusters with knowledge graphs, with which we augment an attacker LLM. Our method produces higher attack success rates and offers interpretability benefits relative to few-shot prompting. Results underscore the need for disinformation mitigations that scale globally and are grounded in real-world adversarial misuse.
△ Less
Submitted 23 September, 2025;
originally announced September 2025.
-
Taxonomizing Representational Harms using Speech Act Theory
Authors:
Emily Corvi,
Hannah Washington,
Stefanie Reed,
Chad Atalla,
Alexandra Chouldechova,
P. Alex Dow,
Jean Garcia-Gathright,
Nicholas Pangakis,
Emily Sheng,
Dan Vann,
Matthew Vogel,
Hanna Wallach
Abstract:
Representational harms are widely recognized among fairness-related harms caused by generative language systems. However, their definitions are commonly under-specified. We make a theoretical contribution to the specification of representational harms by introducing a framework, grounded in speech act theory (Austin, 1962), that conceptualizes representational harms caused by generative language s…
▽ More
Representational harms are widely recognized among fairness-related harms caused by generative language systems. However, their definitions are commonly under-specified. We make a theoretical contribution to the specification of representational harms by introducing a framework, grounded in speech act theory (Austin, 1962), that conceptualizes representational harms caused by generative language systems as the perlocutionary effects (i.e., real-world impacts) of particular types of illocutionary acts (i.e., system behaviors). Building on this argument and drawing on relevant literature from linguistic anthropology and sociolinguistics, we provide new definitions of stereotyping, demeaning, and erasure. We then use our framework to develop a granular taxonomy of illocutionary acts that cause representational harms, going beyond the high-level taxonomies presented in previous work. We also discuss the ways that our framework and taxonomy can support the development of valid measurement instruments. Finally, we demonstrate the utility of our framework and taxonomy via a case study that engages with recent conceptual debates about what constitutes a representational harm and how such harms should be measured.
△ Less
Submitted 7 June, 2025; v1 submitted 1 April, 2025;
originally announced April 2025.
-
Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge
Authors:
Hanna Wallach,
Meera Desai,
A. Feder Cooper,
Angelina Wang,
Chad Atalla,
Solon Barocas,
Su Lin Blodgett,
Alexandra Chouldechova,
Emily Corvi,
P. Alex Dow,
Jean Garcia-Gathright,
Alexandra Olteanu,
Nicholas Pangakis,
Stefanie Reed,
Emily Sheng,
Dan Vann,
Jennifer Wortman Vaughan,
Matthew Vogel,
Hannah Washington,
Abigail Z. Jacobs
Abstract:
The measurement tasks involved in evaluating generative AI (GenAI) systems lack sufficient scientific rigor, leading to what has been described as "a tangle of sloppy tests [and] apples-to-oranges comparisons" (Roose, 2024). In this position paper, we argue that the ML community would benefit from learning from and drawing on the social sciences when developing and using measurement instruments fo…
▽ More
The measurement tasks involved in evaluating generative AI (GenAI) systems lack sufficient scientific rigor, leading to what has been described as "a tangle of sloppy tests [and] apples-to-oranges comparisons" (Roose, 2024). In this position paper, we argue that the ML community would benefit from learning from and drawing on the social sciences when developing and using measurement instruments for evaluating GenAI systems. Specifically, our position is that evaluating GenAI systems is a social science measurement challenge. We present a four-level framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, behaviors, and impacts of GenAI systems. This framework has two important implications: First, it can broaden the expertise involved in evaluating GenAI systems by enabling stakeholders with different perspectives to participate in conceptual debates. Second, it brings rigor to both conceptual and operational debates by offering a set of lenses for interrogating validity.
△ Less
Submitted 6 June, 2025; v1 submitted 1 February, 2025;
originally announced February 2025.
-
A Shared Standard for Valid Measurement of Generative AI Systems' Capabilities, Risks, and Impacts
Authors:
Alexandra Chouldechova,
Chad Atalla,
Solon Barocas,
A. Feder Cooper,
Emily Corvi,
P. Alex Dow,
Jean Garcia-Gathright,
Nicholas Pangakis,
Stefanie Reed,
Emily Sheng,
Dan Vann,
Matthew Vogel,
Hannah Washington,
Hanna Wallach
Abstract:
The valid measurement of generative AI (GenAI) systems' capabilities, risks, and impacts forms the bedrock of our ability to evaluate these systems. We introduce a shared standard for valid measurement that helps place many of the disparate-seeming evaluation practices in use today on a common footing. Our framework, grounded in measurement theory from the social sciences, extends the work of Adco…
▽ More
The valid measurement of generative AI (GenAI) systems' capabilities, risks, and impacts forms the bedrock of our ability to evaluate these systems. We introduce a shared standard for valid measurement that helps place many of the disparate-seeming evaluation practices in use today on a common footing. Our framework, grounded in measurement theory from the social sciences, extends the work of Adcock & Collier (2001) in which the authors formalized valid measurement of concepts in political science via three processes: systematizing background concepts, operationalizing systematized concepts via annotation procedures, and applying those procedures to instances. We argue that valid measurement of GenAI systems' capabilities, risks, and impacts, further requires systematizing, operationalizing, and applying not only the entailed concepts, but also the contexts of interest and the metrics used. This involves both descriptive reasoning about particular instances and inferential reasoning about underlying populations, which is the purview of statistics. By placing many disparate-seeming GenAI evaluation practices on a common footing, our framework enables individual evaluations to be better understood, interrogated for reliability and validity, and meaningfully compared. This is an important step in advancing GenAI evaluation practices toward more formalized and theoretically grounded processes -- i.e., toward a science of GenAI evaluations.
△ Less
Submitted 2 December, 2024;
originally announced December 2024.
-
Evaluating Generative AI Systems is a Social Science Measurement Challenge
Authors:
Hanna Wallach,
Meera Desai,
Nicholas Pangakis,
A. Feder Cooper,
Angelina Wang,
Solon Barocas,
Alexandra Chouldechova,
Chad Atalla,
Su Lin Blodgett,
Emily Corvi,
P. Alex Dow,
Jean Garcia-Gathright,
Alexandra Olteanu,
Stefanie Reed,
Emily Sheng,
Dan Vann,
Jennifer Wortman Vaughan,
Matthew Vogel,
Hannah Washington,
Abigail Z. Jacobs
Abstract:
Across academia, industry, and government, there is an increasing awareness that the measurement tasks involved in evaluating generative AI (GenAI) systems are especially difficult. We argue that these measurement tasks are highly reminiscent of measurement tasks found throughout the social sciences. With this in mind, we present a framework, grounded in measurement theory from the social sciences…
▽ More
Across academia, industry, and government, there is an increasing awareness that the measurement tasks involved in evaluating generative AI (GenAI) systems are especially difficult. We argue that these measurement tasks are highly reminiscent of measurement tasks found throughout the social sciences. With this in mind, we present a framework, grounded in measurement theory from the social sciences, for measuring concepts related to the capabilities, impacts, opportunities, and risks of GenAI systems. The framework distinguishes between four levels: the background concept, the systematized concept, the measurement instrument(s), and the instance-level measurements themselves. This four-level approach differs from the way measurement is typically done in ML, where researchers and practitioners appear to jump straight from background concepts to measurement instruments, with little to no explicit systematization in between. As well as surfacing assumptions, thereby making it easier to understand exactly what the resulting measurements do and do not mean, this framework has two important implications for evaluating evaluations: First, it can enable stakeholders from different worlds to participate in conceptual debates, broadening the expertise involved in evaluating GenAI systems. Second, it brings rigor to operational debates by offering a set of lenses for interrogating the validity of measurement instruments and their resulting measurements.
△ Less
Submitted 16 November, 2024;
originally announced November 2024.
-
A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
Authors:
Ahmed Magooda,
Alec Helyar,
Kyle Jackson,
David Sullivan,
Chad Atalla,
Emily Sheng,
Dan Vann,
Richard Edgar,
Hamid Palangi,
Roman Lutz,
Hongliang Kong,
Vincent Yun,
Eslam Kamal,
Federico Zarfati,
Hanna Wallach,
Sarah Bird,
Mei Chen
Abstract:
We present a framework for the automated measurement of responsible AI (RAI) metrics for large language models (LLMs) and associated products and services. Our framework for automatically measuring harms from LLMs builds on existing technical and sociotechnical expertise and leverages the capabilities of state-of-the-art LLMs, such as GPT-4. We use this framework to run through several case studie…
▽ More
We present a framework for the automated measurement of responsible AI (RAI) metrics for large language models (LLMs) and associated products and services. Our framework for automatically measuring harms from LLMs builds on existing technical and sociotechnical expertise and leverages the capabilities of state-of-the-art LLMs, such as GPT-4. We use this framework to run through several case studies investigating how different LLMs may violate a range of RAI-related principles. The framework may be employed alongside domain-specific sociotechnical expertise to create measurements for new harm areas in the future. By implementing this framework, we aim to enable more advanced harm measurement efforts and further the responsible use of LLMs.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
