Bottom-Up and Top-Down Analysis of Values, Agendas, and Observations in Corpora and LLMs
Authors:
Scott E. Friedman,
Noam Benkler,
Drisana Mosaphir,
Jeffrey Rye,
Sonja M. Schmer-Galunder,
Micah Goldwater,
Matthew McLure,
Ruta Wheelock,
Jeremy Gottlieb,
Robert P. Goldman,
Christopher Miller
Abstract:
Large language models (LLMs) generate diverse, situated, persuasive texts from a plurality of potential perspectives, influenced heavily by their prompts and training data. As part of LLM adoption, we seek to characterize - and ideally, manage - the socio-cultural values that they express, for reasons of safety, accuracy, inclusion, and cultural fidelity. We present a validated approach to automat…
▽ More
Large language models (LLMs) generate diverse, situated, persuasive texts from a plurality of potential perspectives, influenced heavily by their prompts and training data. As part of LLM adoption, we seek to characterize - and ideally, manage - the socio-cultural values that they express, for reasons of safety, accuracy, inclusion, and cultural fidelity. We present a validated approach to automatically (1) extracting heterogeneous latent value propositions from texts, (2) assessing resonance and conflict of values with texts, and (3) combining these operations to characterize the pluralistic value alignment of human-sourced and LLM-sourced textual data.
△ Less
Submitted 6 November, 2024;
originally announced November 2024.
Enable people to identify science news based on retracted articles on social media
Authors:
Waheeb Yaqub,
Judy Kay,
Micah Goldwater
Abstract:
For many people, social media is an important way to consume news on important topics like health. Unfortunately, some influential health news is misinformation because it is based on retracted scientific work. Ours is the first work to explore how people can understand this form of misinformation and how an augmented social media interface can enable them to make use of information about retracti…
▽ More
For many people, social media is an important way to consume news on important topics like health. Unfortunately, some influential health news is misinformation because it is based on retracted scientific work. Ours is the first work to explore how people can understand this form of misinformation and how an augmented social media interface can enable them to make use of information about retraction. We report a between subjects think-aloud study with 44 participants, where the experimental group used our augmented interface. Our results indicate that this helped them consider retraction when judging the credibility of news. Our key contributions are foundational insights for tackling the problem, revealing the interplay between people's understanding of scientific retraction, their prior beliefs about a topic, and the way they use a social media interface that provides access to retraction information.
△ Less
Submitted 2 September, 2023;
originally announced September 2023.
