Showing 1–50 of 197 results for author: Ferrara, E

Information Suppression in Large Language Models: Auditing, Quantifying, and Characterizing Censorship in DeepSeek

Authors: Peiran Qiu, Siyi Zhou, Emilio Ferrara

Abstract: This study examines information suppression mechanisms in DeepSeek, an open-source large language model (LLM) developed in China. We propose an auditing framework and use it to analyze the model's responses to 646 politically sensitive prompts by comparing its final output with intermediate chain-of-thought (CoT) reasoning. Our audit unveils evidence of semantic-level information suppression in De… ▽ More This study examines information suppression mechanisms in DeepSeek, an open-source large language model (LLM) developed in China. We propose an auditing framework and use it to analyze the model's responses to 646 politically sensitive prompts by comparing its final output with intermediate chain-of-thought (CoT) reasoning. Our audit unveils evidence of semantic-level information suppression in DeepSeek: sensitive content often appears within the model's internal reasoning but is omitted or rephrased in the final output. Specifically, DeepSeek suppresses references to transparency, government accountability, and civic mobilization, while occasionally amplifying language aligned with state propaganda. This study underscores the need for systematic auditing of alignment, content moderation, information suppression, and censorship practices implemented into widely-adopted AI models, to ensure transparency, accountability, and equitable access to unbiased information obtained by means of these systems. △ Less

Submitted 14 June, 2025; originally announced June 2025.

Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment

Authors: Priyanka Dey, Yugal Khanter, Aayush Bothra, Jieyu Zhao, Emilio Ferrara

Abstract: As LLMs become central to interactive applications, ranging from tutoring to mental health, the ability to express personality in culturally appropriate ways is increasingly important. While recent works have explored personality evaluation of LLMs, they largely overlook the interplay between culture and personality. To address this, we introduce CulturalPersonas, the first large-scale benchmark w… ▽ More As LLMs become central to interactive applications, ranging from tutoring to mental health, the ability to express personality in culturally appropriate ways is increasingly important. While recent works have explored personality evaluation of LLMs, they largely overlook the interplay between culture and personality. To address this, we introduce CulturalPersonas, the first large-scale benchmark with human validation for evaluating LLMs' personality expression in culturally grounded, behaviorally rich contexts. Our dataset spans 3,000 scenario-based questions across six diverse countries, designed to elicit personality through everyday scenarios rooted in local values. We evaluate three LLMs, using both multiple-choice and open-ended response formats. Our results show that CulturalPersonas improves alignment with country-specific human personality distributions (over a 20% reduction in Wasserstein distance across models and countries) and elicits more expressive, culturally coherent outputs compared to existing benchmarks. CulturalPersonas surfaces meaningful modulated trait outputs in response to culturally grounded prompts, offering new directions for aligning LLMs to global norms of behavior. By bridging personality expression and cultural nuance, we envision that CulturalPersonas will pave the way for more socially intelligent and globally adaptive LLMs. △ Less

Submitted 5 June, 2025; originally announced June 2025.

Bridging the Narrative Divide: Cross-Platform Discourse Networks in Fragmented Ecosystems

Authors: Patrick Gerard, Hans W. A. Hanley, Luca Luceri, Emilio Ferrara

Abstract: Political discourse has grown increasingly fragmented across different social platforms, making it challenging to trace how narratives spread and evolve within such a fragmented information ecosystem. Reconstructing social graphs and information diffusion networks is challenging, and available strategies typically depend on platform-specific features and behavioral signals which are often incompat… ▽ More Political discourse has grown increasingly fragmented across different social platforms, making it challenging to trace how narratives spread and evolve within such a fragmented information ecosystem. Reconstructing social graphs and information diffusion networks is challenging, and available strategies typically depend on platform-specific features and behavioral signals which are often incompatible across systems and increasingly restricted. To address these challenges, we present a platform-agnostic framework that allows to accurately and efficiently reconstruct the underlying social graph of users' cross-platform interactions, based on discovering latent narratives and users' participation therein. Our method achieves state-of-the-art performance in key network-based tasks: information operation detection, ideological stance prediction, and cross-platform engagement prediction$\unicode{x2013}$$\unicode{x2013}$while requiring significantly less data than existing alternatives and capturing a broader set of users. When applied to cross-platform information dynamics between Truth Social and X (formerly Twitter), our framework reveals a small, mixed-platform group of $\textit{bridge users}$, comprising just 0.33% of users and 2.14% of posts, who introduce nearly 70% of $\textit{migrating narratives}$ to the receiving platform. These findings offer a structural lens for anticipating how narratives traverse fragmented information ecosystems, with implications for cross-platform governance, content moderation, and policy interventions. △ Less

Submitted 22 May, 2025; originally announced May 2025.

Comments: 22 pages, 5 figures

Community Moderation and the New Epistemology of Fact Checking on Social Media

Authors: Isabelle Augenstein, Michiel Bakker, Tanmoy Chakraborty, David Corney, Emilio Ferrara, Iryna Gurevych, Scott Hale, Eduard Hovy, Heng Ji, Irene Larraz, Filippo Menczer, Preslav Nakov, Paolo Papotti, Dhruv Sahnan, Greta Warren, Giovanni Zagni

Abstract: Social media platforms have traditionally relied on internal moderation teams and partnerships with independent fact-checking organizations to identify and flag misleading content. Recently, however, platforms including X (formerly Twitter) and Meta have shifted towards community-driven content moderation by launching their own versions of crowd-sourced fact-checking -- Community Notes. If effecti… ▽ More Social media platforms have traditionally relied on internal moderation teams and partnerships with independent fact-checking organizations to identify and flag misleading content. Recently, however, platforms including X (formerly Twitter) and Meta have shifted towards community-driven content moderation by launching their own versions of crowd-sourced fact-checking -- Community Notes. If effectively scaled and governed, such crowd-checking initiatives have the potential to combat misinformation with increased scale and speed as successfully as community-driven efforts once did with spam. Nevertheless, general content moderation, especially for misinformation, is inherently more complex. Public perceptions of truth are often shaped by personal biases, political leanings, and cultural contexts, complicating consensus on what constitutes misleading content. This suggests that community efforts, while valuable, cannot replace the indispensable role of professional fact-checkers. Here we systemically examine the current approaches to misinformation detection across major platforms, explore the emerging role of community-driven moderation, and critically evaluate both the promises and challenges of crowd-checking at scale. △ Less

Submitted 26 May, 2025; originally announced May 2025.

Comments: 1 Figure, 2 tables

Coordinated Inauthentic Behavior on TikTok: Challenges and Opportunities for Detection in a Video-First Ecosystem

Authors: Luca Luceri, Tanishq Vijay Salkar, Ashwin Balasubramanian, Gabriela Pinto, Chenning Sun, Emilio Ferrara

Abstract: Detecting coordinated inauthentic behavior (CIB) is central to the study of online influence operations. However, most methods focus on text-centric platforms, leaving video-first ecosystems like TikTok largely unexplored. To address this gap, we develop and evaluate a computational framework for detecting CIB on TikTok, leveraging a network-based approach adapted to the platform's unique content… ▽ More Detecting coordinated inauthentic behavior (CIB) is central to the study of online influence operations. However, most methods focus on text-centric platforms, leaving video-first ecosystems like TikTok largely unexplored. To address this gap, we develop and evaluate a computational framework for detecting CIB on TikTok, leveraging a network-based approach adapted to the platform's unique content and interaction structures. Building on existing approaches, we construct user similarity networks based on shared behaviors, including synchronized posting, repeated use of similar captions, multimedia content reuse, and hashtag sequence overlap, and apply graph pruning techniques to identify dense networks of likely coordinated accounts. Analyzing a dataset of 793K TikTok videos related to the 2024 U.S. Presidential Election, we uncover a range of coordinated activities, from synchronized amplification of political narratives to semi-automated content replication using AI-generated voiceovers and split-screen video formats. Our findings show that while traditional coordination indicators generalize well to TikTok, other signals, such as those based on textual similarity of video transcripts or Duet and Stitch interactions, prove ineffective, highlighting the platform's distinct content norms and interaction mechanics. This work provides the first empirical foundation for studying and detecting CIB on TikTok, paving the way for future research into influence operations in short-form video platforms. △ Less

Submitted 16 May, 2025; originally announced May 2025.

EDTok: A Dataset for Eating Disorder Content on TikTok

Authors: Charles Bickham, Bryan Ramirez-Gonzalez, Minh Duc Chu, Kristina Lerman, Emilio Ferrara

Abstract: Eating disorders, which include anorexia nervosa and bulimia nervosa, have been exacerbated by the COVID-19 pandemic, with increased diagnoses linked to heightened exposure to idealized body images online. TikTok, a platform with over a billion predominantly adolescent users, has become a key space where eating disorder content is shared, raising concerns about its impact on vulnerable populations… ▽ More Eating disorders, which include anorexia nervosa and bulimia nervosa, have been exacerbated by the COVID-19 pandemic, with increased diagnoses linked to heightened exposure to idealized body images online. TikTok, a platform with over a billion predominantly adolescent users, has become a key space where eating disorder content is shared, raising concerns about its impact on vulnerable populations. In response, we present a curated dataset of 43,040 TikTok videos, collected using keywords and hashtags related to eating disorders. Spanning from January 2019 to June 2024, this dataset, offers a comprehensive view of eating disorder-related content on TikTok. Our dataset has the potential to address significant research gaps, enabling analysis of content spread and moderation, user engagement, and the pandemic's influence on eating disorder trends. This work aims to inform strategies for mitigating risks associated with harmful content, contributing valuable insights to the study of digital health and social media's role in shaping mental health. △ Less

Submitted 4 May, 2025; originally announced May 2025.

Comments: 10 pages, 6 figures

Limited Effectiveness of LLM-based Data Augmentation for COVID-19 Misinformation Stance Detection

Authors: Eun Cheol Choi, Ashwin Balasubramanian, Jinhu Qi, Emilio Ferrara

Abstract: Misinformation surrounding emerging outbreaks poses a serious societal threat, making robust countermeasures essential. One promising approach is stance detection (SD), which identifies whether social media posts support or oppose misleading claims. In this work, we finetune classifiers on COVID-19 misinformation SD datasets consisting of claims and corresponding tweets. Specifically, we test cont… ▽ More Misinformation surrounding emerging outbreaks poses a serious societal threat, making robust countermeasures essential. One promising approach is stance detection (SD), which identifies whether social media posts support or oppose misleading claims. In this work, we finetune classifiers on COVID-19 misinformation SD datasets consisting of claims and corresponding tweets. Specifically, we test controllable misinformation generation (CMG) using large language models (LLMs) as a method for data augmentation. While CMG demonstrates the potential for expanding training datasets, our experiments reveal that performance gains over traditional augmentation methods are often minimal and inconsistent, primarily due to built-in safeguards within LLMs. We release our code and datasets to facilitate further research on misinformation detection and generation. △ Less

Submitted 4 March, 2025; originally announced March 2025.

Beyond Interaction Patterns: Assessing Claims of Coordinated Inter-State Information Operations on Twitter/X

Authors: Valeria Pantè, David Axelrod, Alessandro Flammini, Filippo Menczer, Emilio Ferrara, Luca Luceri

Abstract: Social media platforms have become key tools for coordinated influence operations, enabling state actors to manipulate public opinion through strategic, collective actions. While previous research has suggested collaboration between states, such research failed to leverage state-of-the-art coordination indicators or control datasets. In this study, we investigate inter-state coordination by analyz… ▽ More Social media platforms have become key tools for coordinated influence operations, enabling state actors to manipulate public opinion through strategic, collective actions. While previous research has suggested collaboration between states, such research failed to leverage state-of-the-art coordination indicators or control datasets. In this study, we investigate inter-state coordination by analyzing multiple online behavioral traces and using sophisticated coordination detection models. By incorporating a control dataset to differentiate organic user activity from coordinated efforts, our findings reveal no evidence of inter-state coordination. These results challenge earlier claims and underscore the importance of robust methodologies and control datasets in accurately detecting online coordination. △ Less

Submitted 24 February, 2025; originally announced February 2025.

Synthetic Politics: Prevalence, Spreaders, and Emotional Reception of AI-Generated Political Images on X

Authors: Zhiyi Chen, Jinyi Ye, Beverlyn Tsai, Emilio Ferrara, Luca Luceri

Abstract: Despite widespread concerns about the risks of AI-generated content (AIGC) to the integrity of social media discourse, little is known about its scale and scope, the actors responsible for its dissemination online, and the user responses it elicits. In this work, we measure and characterize the prevalence, spreaders, and emotional reception of AI-generated political images. Analyzing a large-scale… ▽ More Despite widespread concerns about the risks of AI-generated content (AIGC) to the integrity of social media discourse, little is known about its scale and scope, the actors responsible for its dissemination online, and the user responses it elicits. In this work, we measure and characterize the prevalence, spreaders, and emotional reception of AI-generated political images. Analyzing a large-scale dataset from Twitter/X related to the 2024 U.S. Presidential Election, we find that approximately 12% of shared images are detected as AI-generated, and around 10% of users are responsible for sharing 80% of AI-generated images. AIGC superspreaders--defined as the users who not only share a high volume of AI-generated images but also receive substantial engagement through retweets--are more likely to be X Premium subscribers, have a right-leaning orientation, and exhibit automated behavior. Their profiles contain a higher proportion of AI-generated images than non-superspreaders, and some engage in extreme levels of AIGC sharing. Moreover, superspreaders' AI image tweets elicit more positive and less toxic responses than their non-AI image tweets. This study serves as one of the first steps toward understanding the role generative AI plays in shaping online socio-political environments and offers implications for platform governance. △ Less

Submitted 13 May, 2025; v1 submitted 16 February, 2025; originally announced February 2025.

Network-informed Prompt Engineering against Organized Astroturf Campaigns under Extreme Class Imbalance

Authors: Nikos Kanakaris, Heng Ping, Xiongye Xiao, Nesreen K. Ahmed, Luca Luceri, Emilio Ferrara, Paul Bogdan

Abstract: Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the te… ▽ More Detecting organized political campaigns is of paramount importance in fighting against disinformation on social media. Existing approaches for the identification of such organized actions employ techniques mostly from network science, graph machine learning and natural language processing. Their ultimate goal is to analyze the relationships and interactions (e.g. re-posting) among users and the textual similarities of their posts. Despite their effectiveness in recognizing astroturf campaigns, these methods face significant challenges, notably the class imbalance in available training datasets. To mitigate this issue, recent methods usually resort to data augmentation or increasing the number of positive samples, which may not always be feasible or sufficient in real-world settings. Following a different path, in this paper, we propose a novel framework for identifying astroturf campaigns based solely on large language models (LLMs), introducing a Balanced Retrieval-Augmented Generation (Balanced RAG) component. Our approach first gives both textual information concerning the posts (in our case tweets) and the user interactions of the social network as input to a language model. Then, through prompt engineering and the proposed Balanced RAG method, it effectively detects coordinated disinformation campaigns on X (Twitter). The proposed framework does not require any training or fine-tuning of the language model. Instead, by strategically harnessing the strengths of prompt engineering and Balanced RAG, it facilitates LLMs to overcome the effects of class imbalance and effectively identify coordinated political campaigns. The experimental results demonstrate that by incorporating the proposed prompt engineering and Balanced RAG methods, our framework outperforms the traditional graph-based baselines, achieving 2x-3x improvements in terms of precision, recall and F1 scores. △ Less

Submitted 17 February, 2025; v1 submitted 20 January, 2025; originally announced January 2025.

Safe Spaces or Toxic Places? Content Moderation and Social Dynamics of Online Eating Disorder Communities

Authors: Kristina Lerman, Minh Duc Chu, Charles Bickham, Luca Luceri, Emilio Ferrara

Abstract: Social media platforms have become critical spaces for discussing mental health concerns, including eating disorders. While these platforms can provide valuable support networks, they may also amplify harmful content that glorifies disordered cognition and self-destructive behaviors. While social media platforms have implemented various content moderation strategies, from stringent to laissez-fair… ▽ More Social media platforms have become critical spaces for discussing mental health concerns, including eating disorders. While these platforms can provide valuable support networks, they may also amplify harmful content that glorifies disordered cognition and self-destructive behaviors. While social media platforms have implemented various content moderation strategies, from stringent to laissez-faire approaches, we lack a comprehensive understanding of how these different moderation practices interact with user engagement in online communities around these sensitive mental health topics. This study addresses this knowledge gap through a comparative analysis of eating disorder discussions across Twitter/X, Reddit, and TikTok. Our findings reveal that while users across all platforms engage similarly in expressing concerns and seeking support, platforms with weaker moderation (like Twitter/X) enable the formation of toxic echo chambers that amplify pro-anorexia rhetoric. These results demonstrate how moderation strategies significantly influence the development and impact of online communities, particularly in contexts involving mental health and self-harm. △ Less

Submitted 20 December, 2024; originally announced December 2024.

Comments: arXiv admin note: text overlap with arXiv:2401.09647

Tracking the 2024 US Presidential Election Chatter on TikTok: A Public Multimodal Dataset

Authors: Gabriela Pinto, Charles Bickham, Tanishq Salkar, Joyston Menezes, Luca Luceri, Emilio Ferrara

Abstract: This paper presents the TikTok 2024 U.S. Presidential Election Dataset, a large-scale, resource designed to advance research into political communication and social media dynamics. The dataset comprises 3.14 million videos published on TikTok between November 1, 2023, and October 16, 2024, encompassing video ids and transcripts. Data collection was conducted using the TikTok Research API with a co… ▽ More This paper presents the TikTok 2024 U.S. Presidential Election Dataset, a large-scale, resource designed to advance research into political communication and social media dynamics. The dataset comprises 3.14 million videos published on TikTok between November 1, 2023, and October 16, 2024, encompassing video ids and transcripts. Data collection was conducted using the TikTok Research API with a comprehensive set of election-related keywords and hashtags, supplemented by third-party tools to address API limitations and expand content coverage, enabling analysis of hashtag co-occurrence networks that reveal politically aligned hashtags based on ideological affiliations, the evolution of top hashtags over time, and summary statistics that highlight the dataset's scale and richness. This dataset offers insights into TikTok's role in shaping electoral discourse by providing a multimodal view of election-related content. It enables researchers to explore critical topics such as coordinated messaging, misinformation spread, audience engagement, and linguistic trends. The TikTok 2024 U.S. Presidential Election Dataset is publicly available and aims to contribute to the broader understanding of social media's impact on democracy and public opinion. △ Less

Submitted 20 December, 2024; originally announced December 2024.

A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science

Authors: Chenxiao Yu, Jinyi Ye, Yuangang Li, Zheng Li, Emilio Ferrara, Xiyang Hu, Yue Zhao

Abstract: While LLMs have demonstrated remarkable capabilities in text generation and reasoning, their ability to simulate human decision-making -- particularly in political contexts -- remains an open question. However, modeling voter behavior presents unique challenges due to limited voter-level data, evolving political landscapes, and the complexity of human reasoning. In this study, we develop a theory-… ▽ More While LLMs have demonstrated remarkable capabilities in text generation and reasoning, their ability to simulate human decision-making -- particularly in political contexts -- remains an open question. However, modeling voter behavior presents unique challenges due to limited voter-level data, evolving political landscapes, and the complexity of human reasoning. In this study, we develop a theory-driven, multi-step reasoning framework that integrates demographic, temporal and ideological factors to simulate voter decision-making at scale. Using synthetic personas calibrated to real-world voter data, we conduct large-scale simulations of recent U.S. presidential elections. Our method significantly improves simulation accuracy while mitigating model biases. We examine its robustness by comparing performance across different LLMs. We further investigate the challenges and constraints that arise from LLM-based political simulations. Our work provides both a scalable framework for modeling political decision-making behavior and insights into the promise and limitations of using LLMs in political science research. △ Less

Submitted 9 April, 2025; v1 submitted 19 December, 2024; originally announced December 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2411.03321 This version adds a new model to our experimental setup, modifies the paper's main discussion, and updates the authorship list

IOHunter: Graph Foundation Model to Uncover Online Information Operations

Authors: Marco Minici, Luca Luceri, Francesco Fabbri, Emilio Ferrara

Abstract: Social media platforms have become vital spaces for public discourse, serving as modern agoràs where a wide range of voices influence societal narratives. However, their open nature also makes them vulnerable to exploitation by malicious actors, including state-sponsored entities, who can conduct information operations (IOs) to manipulate public opinion. The spread of misinformation, false news, a… ▽ More Social media platforms have become vital spaces for public discourse, serving as modern agoràs where a wide range of voices influence societal narratives. However, their open nature also makes them vulnerable to exploitation by malicious actors, including state-sponsored entities, who can conduct information operations (IOs) to manipulate public opinion. The spread of misinformation, false news, and misleading claims threatens democratic processes and societal cohesion, making it crucial to develop methods for the timely detection of inauthentic activity to protect the integrity of online discourse. In this work, we introduce a methodology designed to identify users orchestrating information operations, a.k.a. IO drivers, across various influence campaigns. Our framework, named IOHunter, leverages the combined strengths of Language Models and Graph Neural Networks to improve generalization in supervised, scarcely-supervised, and cross-IO contexts. Our approach achieves state-of-the-art performance across multiple sets of IOs originating from six countries, significantly surpassing existing approaches. This research marks a step toward developing Graph Foundation Models specifically tailored for the task of IO detection on social media platforms. △ Less

Submitted 3 March, 2025; v1 submitted 19 December, 2024; originally announced December 2024.

Comments: Accepted at AAAI 2025

Hybrid Forecasting of Geopolitical Events

Authors: Daniel M. Benjamin, Fred Morstatter, Ali E. Abbas, Andres Abeliuk, Pavel Atanasov, Stephen Bennett, Andreas Beger, Saurabh Birari, David V. Budescu, Michele Catasta, Emilio Ferrara, Lucas Haravitch, Mark Himmelstein, KSM Tozammel Hossain, Yuzhong Huang, Woojeong Jin, Regina Joseph, Jure Leskovec, Akira Matsui, Mehrnoosh Mirtaheri, Xiang Ren, Gleb Satyukov, Rajiv Sethi, Amandeep Singh, Rok Sosic , et al. (4 additional authors not shown)

Abstract: Sound decision-making relies on accurate prediction for tangible outcomes ranging from military conflict to disease outbreaks. To improve crowdsourced forecasting accuracy, we developed SAGE, a hybrid forecasting system that combines human and machine generated forecasts. The system provides a platform where users can interact with machine models and thus anchor their judgments on an objective ben… ▽ More Sound decision-making relies on accurate prediction for tangible outcomes ranging from military conflict to disease outbreaks. To improve crowdsourced forecasting accuracy, we developed SAGE, a hybrid forecasting system that combines human and machine generated forecasts. The system provides a platform where users can interact with machine models and thus anchor their judgments on an objective benchmark. The system also aggregates human and machine forecasts weighting both for propinquity and based on assessed skill while adjusting for overconfidence. We present results from the Hybrid Forecasting Competition (HFC) - larger than comparable forecasting tournaments - including 1085 users forecasting 398 real-world forecasting problems over eight months. Our main result is that the hybrid system generated more accurate forecasts compared to a human-only baseline which had no machine generated predictions. We found that skilled forecasters who had access to machine-generated forecasts outperformed those who only viewed historical data. We also demonstrated the inclusion of machine-generated forecasts in our aggregation algorithms improved performance, both in terms of accuracy and scalability. This suggests that hybrid forecasting systems, which potentially require fewer human resources, can be a viable approach for maintaining a competitive level of accuracy over a larger number of forecasting questions. △ Less

Submitted 14 December, 2024; originally announced December 2024.

Comments: 20 pages, 6 figures, 4 tables

Journal ref: AI Magazine, Volume 44, Issue 1, Pages 112-128, Spring 2023

Political-LLM: Large Language Models in Political Science

Authors: Lincan Li, Jiaqi Li, Catherine Chen, Fred Gui, Hongjia Yang, Chenxiao Yu, Zhengguang Wang, Jianing Cai, Junlong Aaron Zhou, Bolin Shen, Alex Qian, Weixin Chen, Zhongkai Xue, Lichao Sun, Lifang He, Hanjie Chen, Kaize Ding, Zijian Du, Fangzhou Mu, Jiaxin Pei, Jieyu Zhao, Swabha Swayamdipta, Willie Neiswanger, Hua Wei, Xiyang Hu , et al. (22 additional authors not shown)

Abstract: In recent years, large language models (LLMs) have been widely adopted in political science tasks such as election prediction, sentiment analysis, policy impact assessment, and misinformation detection. Meanwhile, the need to systematically understand how LLMs can further revolutionize the field also becomes urgent. In this work, we--a multidisciplinary team of researchers spanning computer scienc… ▽ More In recent years, large language models (LLMs) have been widely adopted in political science tasks such as election prediction, sentiment analysis, policy impact assessment, and misinformation detection. Meanwhile, the need to systematically understand how LLMs can further revolutionize the field also becomes urgent. In this work, we--a multidisciplinary team of researchers spanning computer science and political science--present the first principled framework termed Political-LLM to advance the comprehensive understanding of integrating LLMs into computational political science. Specifically, we first introduce a fundamental taxonomy classifying the existing explorations into two perspectives: political science and computational methodologies. In particular, from the political science perspective, we highlight the role of LLMs in automating predictive and generative tasks, simulating behavior dynamics, and improving causal inference through tools like counterfactual generation; from a computational perspective, we introduce advancements in data preparation, fine-tuning, and evaluation methods for LLMs that are tailored to political contexts. We identify key challenges and future directions, emphasizing the development of domain-specific datasets, addressing issues of bias and fairness, incorporating human expertise, and redefining evaluation criteria to align with the unique requirements of computational political science. Political-LLM seeks to serve as a guidebook for researchers to foster an informed, ethical, and impactful use of Artificial Intelligence in political science. Our online resource is available at: http://political-llm.org/. △ Less

Submitted 9 December, 2024; originally announced December 2024.

Comments: 54 Pages, 9 Figures

What Are The Risks of Living in a GenAI Synthetic Reality? The Generative AI Paradox

Authors: Emilio Ferrara

Abstract: Generative AI (GenAI) technologies possess unprecedented potential to reshape our world and our perception of reality. These technologies can amplify traditionally human-centered capabilities, such as creativity and complex problem-solving in socio-technical contexts. By fostering human-AI collaboration, GenAI could enhance productivity, dismantle communication barriers across abilities and cultur… ▽ More Generative AI (GenAI) technologies possess unprecedented potential to reshape our world and our perception of reality. These technologies can amplify traditionally human-centered capabilities, such as creativity and complex problem-solving in socio-technical contexts. By fostering human-AI collaboration, GenAI could enhance productivity, dismantle communication barriers across abilities and cultures, and drive innovation on a global scale. Yet, experts and the public are deeply divided on the implications of GenAI. Concerns range from issues like copyright infringement and the rights of creators whose work trains these models without explicit consent, to the conditions of those employed to annotate vast datasets. Accordingly, new laws and regulatory frameworks are emerging to address these unique challenges. Others point to broader issues, such as economic disruptions from automation and the potential impact on labor markets. Although history suggests that society can adapt to such technological upheavals, the scale and complexity of GenAI's impact warrant careful scrutiny. This paper, however, highlights a subtler, yet potentially more perilous risk of GenAI: the creation of $\textit{personalized synthetic realities}$. GenAI could enable individuals to experience a reality customized to personal desires or shaped by external influences, effectively creating a "filtered" worldview unique to each person. Such personalized synthetic realities could distort how people perceive and interact with the world, leading to a fragmented understanding of shared truths. This paper seeks to raise awareness about these profound and multifaceted risks, emphasizing the potential of GenAI to fundamentally alter the very fabric of our collective reality. △ Less

Submitted 12 November, 2024; originally announced November 2024.

Comments: HUMANS Lab -- Working Paper No. 2024.2 -- The 2024 Election Integrity Initiative -- University of Southern California

Explaining Mixtures of Sources in News Articles

Authors: Alexander Spangher, James Youn, Matt DeButts, Nanyun Peng, Emilio Ferrara, Jonathan May

Abstract: Human writers plan, then write. For large language models (LLMs) to play a role in longer-form article generation, we must understand the planning steps humans make before writing. We explore one kind of planning, source-selection in news, as a case-study for evaluating plans in long-form generation. We ask: why do specific stories call for specific kinds of sources? We imagine a generative proces… ▽ More Human writers plan, then write. For large language models (LLMs) to play a role in longer-form article generation, we must understand the planning steps humans make before writing. We explore one kind of planning, source-selection in news, as a case-study for evaluating plans in long-form generation. We ask: why do specific stories call for specific kinds of sources? We imagine a generative process for story writing where a source-selection schema is first selected by a journalist, and then sources are chosen based on categories in that schema. Learning the article's plan means predicting the schema initially chosen by the journalist. Working with professional journalists, we adapt five existing schemata and introduce three new ones to describe journalistic plans for the inclusion of sources in documents. Then, inspired by Bayesian latent-variable modeling, we develop metrics to select the most likely plan, or schema, underlying a story, which we use to compare schemata. We find that two schemata: stance and social affiliation best explain source plans in most documents. However, other schemata like textual entailment explain source plans in factually rich topics like "Science". Finally, we find we can predict the most suitable schema given just the article's headline with reasonable accuracy. We see this as an important case-study for human planning, and provides a framework and approach for evaluating other kinds of plans. We release a corpora, NewsSources, with annotations for 4M articles. △ Less

Submitted 7 November, 2024; originally announced November 2024.

Comments: 9 pages

Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election

Authors: Jinyi Ye, Luca Luceri, Emilio Ferrara

Abstract: Approximately 50% of tweets in X's user timelines are personalized recommendations from accounts they do not follow. This raises a critical question: What political content are users exposed to beyond their established networks, and what implications does this have for democratic discourse online? In this paper, we present a six-week audit of X's algorithmic content recommendations during the 2024… ▽ More Approximately 50% of tweets in X's user timelines are personalized recommendations from accounts they do not follow. This raises a critical question: What political content are users exposed to beyond their established networks, and what implications does this have for democratic discourse online? In this paper, we present a six-week audit of X's algorithmic content recommendations during the 2024 U.S. Presidential Election by deploying 120 sock-puppet monitoring accounts to capture tweets from their personalized "For You" timelines. Our objective is to quantify out-of-network content exposure for right- and left-leaning user profiles and assess any potential inequalities and biases in political exposure. Our findings indicate that X's algorithm skews exposure toward a few high-popularity accounts across all users, with right-leaning users experiencing the highest level of exposure inequality. Both left- and right-leaning users encounter amplified exposure to accounts aligned with their own political views and reduced exposure to opposing viewpoints. Additionally, we observe that new accounts experience a right-leaning bias in exposure within their default timelines. Our work contributes to understanding how content recommendation systems may induce and reinforce biases while exacerbating vulnerabilities among politically polarized user groups. We underscore the importance of transparency-aware algorithms in addressing critical issues such as safeguarding election integrity and fostering a more informed digital public sphere. △ Less

Submitted 20 March, 2025; v1 submitted 4 November, 2024; originally announced November 2024.

Journal ref: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (FAccT '25)

Unfiltered Conversations: A Dataset of 2024 U.S. Presidential Election Discourse on Truth Social

Authors: Kashish Shah, Patrick Gerard, Luca Luceri, Emilio Ferrara

Abstract: Truth Social, launched as a social media platform with a focus on free speech, has become a prominent space for political discourse, attracting a user base with diverse, yet often conservative, viewpoints. As an emerging platform with minimal content moderation, Truth Social has facilitated discussions around contentious social and political issues but has also seen the spread of conspiratorial an… ▽ More Truth Social, launched as a social media platform with a focus on free speech, has become a prominent space for political discourse, attracting a user base with diverse, yet often conservative, viewpoints. As an emerging platform with minimal content moderation, Truth Social has facilitated discussions around contentious social and political issues but has also seen the spread of conspiratorial and hyper-partisan narratives. In this paper, we introduce and release a comprehensive dataset capturing activity on Truth Social related to the upcoming 2024 U.S. Presidential Election, including posts, replies, user interactions, content and media. This dataset comprises 1.5 million posts published between February, 2024 and October 2024, and encompasses key user engagement features and posts metadata. Data collection began in June 2024, though it includes posts published earlier, with the oldest post dating back to February 2022. This offers researchers a unique resource to study communication patterns, the formation of online communities, and the dissemination of information within Truth Social in the run-up to the election. By providing an in-depth view of Truth Social's user dynamics and content distribution, this dataset aims to support further research on political discourse within an alt-tech social media platform. The dataset is publicly available at https://github.com/kashish-s/TruthSocial_2024ElectionInitiative △ Less

Submitted 2 November, 2024; originally announced November 2024.

Comments: HUMANS Lab -- Working Paper No. 2024.8 -- The 2024 Election Integrity Initiative -- University of Southern California

A Public Dataset Tracking Social Media Discourse about the 2024 U.S. Presidential Election on Twitter/X

Authors: Ashwin Balasubramanian, Vito Zou, Hitesh Narayana, Christina You, Luca Luceri, Emilio Ferrara

Abstract: In this paper, we introduce the first release of a large-scale dataset capturing discourse on $\mathbb{X}$ (a.k.a., Twitter) related to the upcoming 2024 U.S. Presidential Election. Our dataset comprises 22 million publicly available posts on X.com, collected from May 1, 2024, to July 31, 2024, using a custom-built scraper, which we describe in detail. By employing targeted keywords linked to key… ▽ More In this paper, we introduce the first release of a large-scale dataset capturing discourse on $\mathbb{X}$ (a.k.a., Twitter) related to the upcoming 2024 U.S. Presidential Election. Our dataset comprises 22 million publicly available posts on X.com, collected from May 1, 2024, to July 31, 2024, using a custom-built scraper, which we describe in detail. By employing targeted keywords linked to key political figures, events, and emerging issues, we aligned data collection with the election cycle to capture evolving public sentiment and the dynamics of political engagement on social media. This dataset offers researchers a robust foundation to investigate critical questions about the influence of social media in shaping political discourse, the propagation of election-related narratives, and the spread of misinformation. We also present a preliminary analysis that highlights prominent hashtags and keywords within the dataset, offering initial insights into the dominant themes and conversations occurring in the lead-up to the election. Our dataset is available at: url{https://github.com/sinking8/usc-x-24-us-election △ Less

Submitted 1 November, 2024; originally announced November 2024.

Unearthing a Billion Telegram Posts about the 2024 U.S. Presidential Election: Development of a Public Dataset

Authors: Leonardo Blas, Luca Luceri, Emilio Ferrara

Abstract: With its lenient moderation policies and long-standing associations with potentially unlawful activities, Telegram has become an incubator for problematic content, frequently featuring conspiratorial, hyper-partisan, and fringe narratives. In the political sphere, these concerns are amplified by reports of Telegram channels being used to organize violent acts, such as those that occurred during th… ▽ More With its lenient moderation policies and long-standing associations with potentially unlawful activities, Telegram has become an incubator for problematic content, frequently featuring conspiratorial, hyper-partisan, and fringe narratives. In the political sphere, these concerns are amplified by reports of Telegram channels being used to organize violent acts, such as those that occurred during the Capitol Hill attack on January 6, 2021. As the 2024 U.S. election approaches, Telegram remains a focal arena for societal and political discourse, warranting close attention from the research community, regulators, and the media. Based on these premises, we introduce and release a Telegram dataset focused on the 2024 U.S. Presidential Election, featuring over 30,000 chats and half a billion messages, including chat details, profile pictures, messages, and user information. We constructed a network of chats and analyzed the 500 most central ones, examining their shared messages. This resource represents the largest public Telegram dataset to date, offering an unprecedented opportunity to study political discussion on Telegram in the lead-up to the 2024 U.S. election. We will continue to collect data until the end of 2024, and routinely update the dataset released at: https://github.com/leonardo-blas/usc-tg-24-us-election △ Less

Submitted 31 October, 2024; originally announced October 2024.

Comments: HUMANS Lab -- Working Paper No. 2024.5 -- The 2024 Election Integrity Initiative -- University of Southern California

Exposing Cross-Platform Coordinated Inauthentic Activity in the Run-Up to the 2024 U.S. Election

Authors: Federico Cinus, Marco Minici, Luca Luceri, Emilio Ferrara

Abstract: Coordinated information operations remain a persistent challenge on social media, despite platform efforts to curb them. While previous research has primarily focused on identifying these operations within individual platforms, this study shows that coordination frequently transcends platform boundaries. Leveraging newly collected data of online conversations related to the 2024 U.S. Election acro… ▽ More Coordinated information operations remain a persistent challenge on social media, despite platform efforts to curb them. While previous research has primarily focused on identifying these operations within individual platforms, this study shows that coordination frequently transcends platform boundaries. Leveraging newly collected data of online conversations related to the 2024 U.S. Election across $\mathbb{X}$ (formerly, Twitter), Facebook, and Telegram, we construct similarity networks to detect coordinated communities exhibiting suspicious sharing behaviors within and across platforms. Proposing an advanced coordination detection model, we reveal evidence of potential foreign interference, with Russian-affiliated media being systematically promoted across Telegram and $\mathbb{X}$. Our analysis also uncovers substantial intra- and cross-platform coordinated inauthentic activity, driving the spread of highly partisan, low-credibility, and conspiratorial content. These findings highlight the urgent need for regulatory measures that extend beyond individual platforms to effectively address the growing challenge of cross-platform coordinated influence campaigns. △ Less

Submitted 24 April, 2025; v1 submitted 30 October, 2024; originally announced October 2024.

Comments: HUMANS Lab -- Working Paper No. 2024.7 -- The 2024 Election Integrity Initiative -- University of Southern California - Updated Version of WWW '25 Submission

Uncovering Coordinated Cross-Platform Information Operations Threatening the Integrity of the 2024 U.S. Presidential Election Online Discussion

Authors: Marco Minici, Luca Luceri, Federico Cinus, Emilio Ferrara

Abstract: Information Operations (IOs) pose a significant threat to the integrity of democratic processes, with the potential to influence election-related online discourse. In anticipation of the 2024 U.S. presidential election, we present a study aimed at uncovering the digital traces of coordinated IOs on $\mathbb{X}$ (formerly Twitter). Using our machine learning framework for detecting online coordinat… ▽ More Information Operations (IOs) pose a significant threat to the integrity of democratic processes, with the potential to influence election-related online discourse. In anticipation of the 2024 U.S. presidential election, we present a study aimed at uncovering the digital traces of coordinated IOs on $\mathbb{X}$ (formerly Twitter). Using our machine learning framework for detecting online coordination, we analyze a dataset comprising election-related conversations on $\mathbb{X}$ from May 2024. This reveals a network of coordinated inauthentic actors, displaying notable similarities in their link-sharing behaviors. Our analysis shows concerted efforts by these accounts to disseminate misleading, redundant, and biased information across the Web through a coordinated cross-platform information operation: The links shared by this network frequently direct users to other social media platforms or suspicious websites featuring low-quality political content and, in turn, promoting the same $\mathbb{X}$ and YouTube accounts. Members of this network also shared deceptive images generated by AI, accompanied by language attacking political figures and symbolic imagery intended to convey power and dominance. While $\mathbb{X}$ has suspended a subset of these accounts, more than 75% of the coordinated network remains active. Our findings underscore the critical role of developing computational models to scale up the detection of threats on large social media platforms, and emphasize the broader implications of these techniques to detect IOs across the wider Web. △ Less

Submitted 30 October, 2024; v1 submitted 23 September, 2024; originally announced September 2024.

Comments: First Monday 29(11), 2024

Report number: The 2024 Election Integrity Initiative: HUMANS Lab - Working Paper No. 2024.4 - University of Southern California

Large Language Models for Wearable Sensor-Based Human Activity Recognition, Health Monitoring, and Behavioral Modeling: A Survey of Early Trends, Datasets, and Challenges

Authors: Emilio Ferrara

Abstract: The proliferation of wearable technology enables the generation of vast amounts of sensor data, offering significant opportunities for advancements in health monitoring, activity recognition, and personalized medicine. However, the complexity and volume of this data present substantial challenges in data modeling and analysis, which have been tamed with approaches spanning time series modeling to… ▽ More The proliferation of wearable technology enables the generation of vast amounts of sensor data, offering significant opportunities for advancements in health monitoring, activity recognition, and personalized medicine. However, the complexity and volume of this data present substantial challenges in data modeling and analysis, which have been tamed with approaches spanning time series modeling to deep learning techniques. The latest frontier in this domain is the adoption of Large Language Models (LLMs), such as GPT-4 and Llama, for data analysis, modeling, understanding, and generation of human behavior through the lens of wearable sensor data. This survey explores current trends and challenges in applying LLMs for sensor-based human activity recognition and behavior modeling. We discuss the nature of wearable sensors data, the capabilities and limitations of LLMs to model them and their integration with traditional machine learning techniques. We also identify key challenges, including data quality, computational requirements, interpretability, and privacy concerns. By examining case studies and successful applications, we highlight the potential of LLMs in enhancing the analysis and interpretation of wearable sensors data. Finally, we propose future directions for research, emphasizing the need for improved preprocessing techniques, more efficient and scalable models, and interdisciplinary collaboration. This survey aims to provide a comprehensive overview of the intersection between wearable sensors data and LLMs, offering insights into the current state and future prospects of this emerging field. △ Less

Submitted 31 July, 2024; v1 submitted 9 July, 2024; originally announced July 2024.

Journal ref: Sensors, 2024

Tracking the 2024 US Presidential Election Chatter on Tiktok: A Public Multimodal Dataset

Authors: Gabriela Pinto, Charles Bickham, Tanishq Salkar, Luca Luceri, Emilio Ferrara

Abstract: This paper documents our release of a large-scale data collection of TikTok posts related to the upcoming 2024 U.S. Presidential Election. Our current data comprises 1.8 million videos published between November 1, 2023, and May 26, 2024. Its exploratory analysis identifies the most common keywords, hashtags, and bigrams in both Spanish and English posts, focusing on the election and the two main… ▽ More This paper documents our release of a large-scale data collection of TikTok posts related to the upcoming 2024 U.S. Presidential Election. Our current data comprises 1.8 million videos published between November 1, 2023, and May 26, 2024. Its exploratory analysis identifies the most common keywords, hashtags, and bigrams in both Spanish and English posts, focusing on the election and the two main Presidential candidates, President Joe Biden and Donald Trump. We utilized the TikTok Research API, incorporating various election-related keywords and hashtags, to capture the full scope of relevant content. To address the limitations of the TikTok Research API, we also employed third-party scrapers to expand our dataset. The dataset is publicly available at https://github.com/gabbypinto/US2024PresElectionTikToks △ Less

Submitted 2 July, 2024; v1 submitted 1 July, 2024; originally announced July 2024.

Comments: The 2024 Election Integrity Initiative

Report number: HUMANS Lab -- Working Paper No. 2024.3

Tracing the Unseen: Uncovering Human Trafficking Patterns in Job Listings

Authors: Siyi Zhou, Jiankun Peng, Emilio Ferrara

Abstract: In the shadow of the digital revolution, the insidious issue of human trafficking has found new breeding grounds within the realms of social media and online job boards. Previous research efforts have predominantly centered on identifying victims via the analysis of escort advertisements. However, our work shifts the focus towards enabling a proactive approach: pinpointing potential traffickers be… ▽ More In the shadow of the digital revolution, the insidious issue of human trafficking has found new breeding grounds within the realms of social media and online job boards. Previous research efforts have predominantly centered on identifying victims via the analysis of escort advertisements. However, our work shifts the focus towards enabling a proactive approach: pinpointing potential traffickers before they lure their preys through false job opportunities. In this study, we collect and analyze a vast dataset comprising over a quarter million job postings collected from eight relevant regions across the United States, spanning nearly two decades (2006-2024). The job boards we considered are specifically catered towards Chinese-speaking immigrants in the US. We classify the job posts into distinct groups based on the self-reported information of the posting user. Our investigation into the types of advertised opportunities, the modes of preferred contact, and the frequency of postings uncovers the patterns characterizing suspicious ads. Additionally, we highlight how external events such as health emergencies and conflicts appear to strongly correlate with increased volume of suspicious job posts: traffickers are more likely to prey upon vulnerable populations in times of crises. This research underscores the imperative for a deeper dive into how online job boards and communication platforms could be unwitting facilitators of human trafficking. More importantly, it calls for the urgent formulation of targeted strategies to dismantle these digital conduits of exploitation. △ Less

Submitted 18 June, 2024; originally announced June 2024.

The Susceptibility Paradox in Online Social Influence

Authors: Luca Luceri, Jinyi Ye, Julie Jiang, Emilio Ferrara

Abstract: Understanding susceptibility to online influence is crucial for mitigating the spread of misinformation and protecting vulnerable audiences. This paper investigates susceptibility to influence within social networks, focusing on the differential effects of influence-driven versus spontaneous behaviors on user content adoption. Our analysis reveals that influence-driven adoption exhibits high homop… ▽ More Understanding susceptibility to online influence is crucial for mitigating the spread of misinformation and protecting vulnerable audiences. This paper investigates susceptibility to influence within social networks, focusing on the differential effects of influence-driven versus spontaneous behaviors on user content adoption. Our analysis reveals that influence-driven adoption exhibits high homophily, indicating that individuals prone to influence often connect with similarly susceptible peers, thereby reinforcing peer influence dynamics, whereas spontaneous adoption shows significant but lower homophily. Additionally, we extend the Generalized Friendship Paradox to influence-driven behaviors, demonstrating that users' friends are generally more susceptible to influence than the users themselves, de facto establishing the notion of Susceptibility Paradox in online social influence. This pattern does not hold for spontaneous behaviors, where friends exhibit fewer spontaneous adoptions. We find that susceptibility to influence can be predicted using friends' susceptibility alone, while predicting spontaneous adoption requires additional features, such as user metadata. These findings highlight the complex interplay between user engagement and characteristics in spontaneous content adoption. Our results provide new insights into social influence mechanisms and offer implications for designing more effective moderation strategies to protect vulnerable audiences. △ Less

Submitted 13 May, 2025; v1 submitted 17 June, 2024; originally announced June 2024.

Journal ref: Proceedings of the Nineteenth International AAAI Conference on Web and Social Media (ICWSM 2025)

Charting the Landscape of Nefarious Uses of Generative Artificial Intelligence for Online Election Interference

Authors: Emilio Ferrara

Abstract: Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) pose significant risks, particularly in the realm of online election interference. This paper explores the nefarious applications of GenAI, highlighting their potential to disrupt democratic processes through deepfakes, botnets, targeted misinformation campaigns, and synthetic identities. By examining recent case studies a… ▽ More Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) pose significant risks, particularly in the realm of online election interference. This paper explores the nefarious applications of GenAI, highlighting their potential to disrupt democratic processes through deepfakes, botnets, targeted misinformation campaigns, and synthetic identities. By examining recent case studies and public incidents, we illustrate how malicious actors exploit these technologies to try influencing voter behavior, spread disinformation, and undermine public trust in electoral systems. The paper also discusses the societal implications of these threats, emphasizing the urgent need for robust mitigation strategies and international cooperation to safeguard democratic integrity. △ Less

Submitted 4 April, 2025; v1 submitted 3 June, 2024; originally announced June 2024.

Comments: First Monday, 2025

Journal ref: First Monday 30(1), June 2025

Hidden in Plain Sight: Exploring the Intersections of Mental Health, Eating Disorders, and Content Moderation on TikTok

Authors: Charles Bickham, Kia Kazemi-Nia, Luca Luceri, Kristina Lerman, Emilio Ferrara

Abstract: Social media platforms actively moderate content glorifying harmful behaviors like eating disorders, which include anorexia and bulimia. However, users have adapted to evade moderation by using coded hashtags. Our study investigates the prevalence of moderation evaders on the popular social media platform TikTok and contrasts their use and emotional valence with mainstream hashtags. We notice that… ▽ More Social media platforms actively moderate content glorifying harmful behaviors like eating disorders, which include anorexia and bulimia. However, users have adapted to evade moderation by using coded hashtags. Our study investigates the prevalence of moderation evaders on the popular social media platform TikTok and contrasts their use and emotional valence with mainstream hashtags. We notice that moderation evaders and mainstream hashtags appear together, indicating that vulnerable users might inadvertently encounter harmful content even when searching for mainstream terms. Additionally, through an analysis of emotional expressions in video descriptions and comments, we find that mainstream hashtags generally promote positive engagement, while moderation evaders evoke a wider range of emotions, including heightened negativity. These findings provide valuable insights for content creators, platform moderation efforts, and interventions aimed at cultivating a supportive online environment for discussions on mental health and eating disorders. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: 10 pages, 5 figures, 2 tables

FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs

Authors: Eun Cheol Choi, Emilio Ferrara

Abstract: Our society is facing rampant misinformation harming public health and trust. To address the societal challenge, we introduce FACT-GPT, a system leveraging Large Language Models (LLMs) to automate the claim matching stage of fact-checking. FACT-GPT, trained on a synthetic dataset, identifies social media content that aligns with, contradicts, or is irrelevant to previously debunked claims. Our eva… ▽ More Our society is facing rampant misinformation harming public health and trust. To address the societal challenge, we introduce FACT-GPT, a system leveraging Large Language Models (LLMs) to automate the claim matching stage of fact-checking. FACT-GPT, trained on a synthetic dataset, identifies social media content that aligns with, contradicts, or is irrelevant to previously debunked claims. Our evaluation shows that our specialized LLMs can match the accuracy of larger models in identifying related claims, closely mirroring human judgment. This research provides an automated solution for efficient claim matching, demonstrates the potential of LLMs in supporting fact-checkers, and offers valuable resources for further research in the field. △ Less

Submitted 8 February, 2024; originally announced February 2024.

GET-Tok: A GenAI-Enriched Multimodal TikTok Dataset Documenting the 2022 Attempted Coup in Peru

Authors: Gabriela Pinto, Keith Burghardt, Kristina Lerman, Emilio Ferrara

Abstract: TikTok is one of the largest and fastest-growing social media sites in the world. TikTok features, however, such as voice transcripts, are often missing and other important features, such as OCR or video descriptions, do not exist. We introduce the Generative AI Enriched TikTok (GET-Tok) data, a pipeline for collecting TikTok videos and enriched data by augmenting the TikTok Research API with gene… ▽ More TikTok is one of the largest and fastest-growing social media sites in the world. TikTok features, however, such as voice transcripts, are often missing and other important features, such as OCR or video descriptions, do not exist. We introduce the Generative AI Enriched TikTok (GET-Tok) data, a pipeline for collecting TikTok videos and enriched data by augmenting the TikTok Research API with generative AI models. As a case study, we collect videos about the attempted coup in Peru initiated by its former President, Pedro Castillo, and its accompanying protests. The data includes information on 43,697 videos published from November 20, 2022 to March 1, 2023 (102 days). Generative AI augments the collected data via transcripts of TikTok videos, text descriptions of what is shown in the videos, what text is displayed within the video, and the stances expressed in the video. Overall, this pipeline will contribute to a better understanding of online discussion in a multimodal setting with applications of Generative AI, especially outlining the utility of this pipeline in non-English-language social media. Our code used to produce the pipeline is in a public Github repository: https://github.com/gabbypinto/GET-Tok-Peru. △ Less

Submitted 8 February, 2024; originally announced February 2024.

Comments: Github repository: https://github.com/gabbypinto/GET-Tok-Peru

Coordinated Activity Modulates the Behavior and Emotions of Organic Users: A Case Study on Tweets about the Gaza Conflict

Authors: Priyanka Dey, Luca Luceri, Emilio Ferrara

Abstract: Social media has become a crucial conduit for the swift dissemination of information during global crises. However, this also paves the way for the manipulation of narratives by malicious actors. This research delves into the interaction dynamics between coordinated (malicious) entities and organic (regular) users on Twitter amidst the Gaza conflict. Through the analysis of approximately 3.5 milli… ▽ More Social media has become a crucial conduit for the swift dissemination of information during global crises. However, this also paves the way for the manipulation of narratives by malicious actors. This research delves into the interaction dynamics between coordinated (malicious) entities and organic (regular) users on Twitter amidst the Gaza conflict. Through the analysis of approximately 3.5 million tweets from over 1.3 million users, our study uncovers that coordinated users significantly impact the information landscape, successfully disseminating their content across the network: a substantial fraction of their messages is adopted and shared by organic users. Furthermore, the study documents a progressive increase in organic users' engagement with coordinated content, which is paralleled by a discernible shift towards more emotionally polarized expressions in their subsequent communications. These results highlight the critical need for vigilance and a nuanced understanding of information manipulation on social media platforms. △ Less

Submitted 8 February, 2024; originally announced February 2024.

"Can You Play Anything Else?" Understanding Play Style Flexibility in League of Legends

Authors: Emily Chen, Alexander Bisberg, Emilio Ferrara

Abstract: This study investigates the concept of flexibility within League of Legends, a popular online multiplayer game, focusing on the relationship between user adaptability and team success. Utilizing a dataset encompassing players of varying skill levels and play styles, we calculate two measures of flexibility for each player: overall flexibility and temporal flexibility. Our findings suggest that the… ▽ More This study investigates the concept of flexibility within League of Legends, a popular online multiplayer game, focusing on the relationship between user adaptability and team success. Utilizing a dataset encompassing players of varying skill levels and play styles, we calculate two measures of flexibility for each player: overall flexibility and temporal flexibility. Our findings suggest that the flexibility of a user is dependent upon a user's preferred play style, and flexibility does impact match outcome. This work also shows that skill level not only indicates how willing a player is to adapt their play style but also how their adaptability changes over time. This paper highlights the duality and balance of specialization versus flexibility, providing insights that can inform strategic planning, collaboration and resource allocation in competitive environments. △ Less

Submitted 10 July, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Moral Values Underpinning COVID-19 Online Communication Patterns

Authors: Julie Jiang, Luca Luceri, Emilio Ferrara

Abstract: The COVID-19 pandemic has triggered profound societal changes, extending beyond its health impacts to the moralization of behaviors. Leveraging insights from moral psychology, this study delves into the moral fabric shaping online discussions surrounding COVID-19 over a span of nearly two years. Our investigation identifies four distinct user groups characterized by differences in morality, politi… ▽ More The COVID-19 pandemic has triggered profound societal changes, extending beyond its health impacts to the moralization of behaviors. Leveraging insights from moral psychology, this study delves into the moral fabric shaping online discussions surrounding COVID-19 over a span of nearly two years. Our investigation identifies four distinct user groups characterized by differences in morality, political ideology, and communication styles. We underscore the intricate relationship between moral differences and political ideologies, revealing a nuanced picture where moral orientations do not rigidly separate users politically. Furthermore, we uncover patterns of moral homophily within the social network, highlighting the existence of one potential moral echo chamber. Analyzing the moral themes embedded in messages, we observe that messages featuring moral foundations not typically favored by their authors, as well as those incorporating multiple moral foundations, resonate more effectively with out-group members. This research contributes valuable insights into the complex interplay between moral foundations, communication dynamics, and network structures on Twitter. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: 11 pages, 8 figures, 2 tables

Social-LLM: Modeling User Behavior at Scale using Language Models and Social Network Data

Authors: Julie Jiang, Emilio Ferrara

Abstract: The proliferation of social network data has unlocked unprecedented opportunities for extensive, data-driven exploration of human behavior. The structural intricacies of social networks offer insights into various computational social science issues, particularly concerning social influence and information diffusion. However, modeling large-scale social network data comes with computational challe… ▽ More The proliferation of social network data has unlocked unprecedented opportunities for extensive, data-driven exploration of human behavior. The structural intricacies of social networks offer insights into various computational social science issues, particularly concerning social influence and information diffusion. However, modeling large-scale social network data comes with computational challenges. Though large language models make it easier than ever to model textual content, any advanced network representation methods struggle with scalability and efficient deployment to out-of-sample users. In response, we introduce a novel approach tailored for modeling social network data in user detection tasks. This innovative method integrates localized social network interactions with the capabilities of large language models. Operating under the premise of social network homophily, which posits that socially connected users share similarities, our approach is designed to address these challenges. We conduct a thorough evaluation of our method across seven real-world social network datasets, spanning a diverse range of topics and detection tasks, showcasing its applicability to advance research in computational social science. △ Less

Submitted 31 December, 2023; originally announced January 2024.

Comments: 10 pages, 5 figures, 2 tables

Social Bots: Detection and Challenges

Authors: Kai-Cheng Yang, Onur Varol, Alexander C. Nwala, Mohsen Sayyadiharikandeh, Emilio Ferrara, Alessandro Flammini, Filippo Menczer

Abstract: While social media are a key source of data for computational social science, their ease of manipulation by malicious actors threatens the integrity of online information exchanges and their analysis. In this Chapter, we focus on malicious social bots, a prominent vehicle for such manipulation. We start by discussing recent studies about the presence and actions of social bots in various online di… ▽ More While social media are a key source of data for computational social science, their ease of manipulation by malicious actors threatens the integrity of online information exchanges and their analysis. In this Chapter, we focus on malicious social bots, a prominent vehicle for such manipulation. We start by discussing recent studies about the presence and actions of social bots in various online discussions to show their real-world implications and the need for detection methods. Then we discuss the challenges of bot detection methods and use Botometer, a publicly available bot detection tool, as a case study to describe recent developments in this area. We close with a practical guide on how to handle social bots in social media research. △ Less

Submitted 28 December, 2023; originally announced December 2023.

Comments: This is a draft of the chapter. The final version will be available in the Handbook of Computational Social Science edited by Taha Yasseri, forthcoming 2024, Edward Elgar Publishing Ltd. The material cannot be used for any other purpose without further permission of the publisher and is for private use only

Can Language Model Moderators Improve the Health of Online Discourse?

Authors: Hyundong Cho, Shuai Liu, Taiwei Shi, Darpan Jain, Basem Rizk, Yuyang Huang, Zixun Lu, Nuan Wen, Jonathan Gratch, Emilio Ferrara, Jonathan May

Abstract: Conversational moderation of online communities is crucial to maintaining civility for a constructive environment, but it is challenging to scale and harmful to moderators. The inclusion of sophisticated natural language generation modules as a force multiplier to aid human moderators is a tantalizing prospect, but adequate evaluation approaches have so far been elusive. In this paper, we establis… ▽ More Conversational moderation of online communities is crucial to maintaining civility for a constructive environment, but it is challenging to scale and harmful to moderators. The inclusion of sophisticated natural language generation modules as a force multiplier to aid human moderators is a tantalizing prospect, but adequate evaluation approaches have so far been elusive. In this paper, we establish a systematic definition of conversational moderation effectiveness grounded on moderation literature and establish design criteria for conducting realistic yet safe evaluation. We then propose a comprehensive evaluation framework to assess models' moderation capabilities independently of human intervention. With our framework, we conduct the first known study of language models as conversational moderators, finding that appropriately prompted models that incorporate insights from social science can provide specific and fair feedback on toxic behavior but struggle to influence users to increase their levels of respect and cooperation. △ Less

Submitted 6 May, 2024; v1 submitted 16 November, 2023; originally announced November 2023.

Comments: 9 pages, NAACL 2024 Main

Tracking the Newsworthiness of Public Documents

Authors: Alexander Spangher, Emilio Ferrara, Ben Welsh, Nanyun Peng, Serdar Tumgoren, Jonathan May

Abstract: Journalists must find stories in huge amounts of textual data (e.g. leaks, bills, press releases) as part of their jobs: determining when and why text becomes news can help us understand coverage patterns and help us build assistive tools. Yet, this is challenging because very few labelled links exist, language use between corpora is very different, and text may be covered for a variety of reasons… ▽ More Journalists must find stories in huge amounts of textual data (e.g. leaks, bills, press releases) as part of their jobs: determining when and why text becomes news can help us understand coverage patterns and help us build assistive tools. Yet, this is challenging because very few labelled links exist, language use between corpora is very different, and text may be covered for a variety of reasons. In this work we focus on news coverage of local public policy in the San Francisco Bay Area by the San Francisco Chronicle. First, we gather news articles, public policy documents and meeting recordings and link them using probabilistic relational modeling, which we show is a low-annotation linking methodology that outperforms other retrieval-based baselines. Second, we define a new task: newsworthiness prediction, to predict if a policy item will get covered. We show that different aspects of public policy discussion yield different newsworthiness signals. Finally we perform human evaluation with expert journalists and show our systems identify policies they consider newsworthy with 68% F1 and our coverage recommendations are helpful with an 84% win-rate. △ Less

Submitted 16 November, 2023; originally announced November 2023.

Comments: 9 pages, 7 pages appendix

Leveraging Large Language Models to Detect Influence Campaigns in Social Media

Authors: Luca Luceri, Eric Boniardi, Emilio Ferrara

Abstract: Social media influence campaigns pose significant challenges to public discourse and democracy. Traditional detection methods fall short due to the complexity and dynamic nature of social media. Addressing this, we propose a novel detection method using Large Language Models (LLMs) that incorporates both user metadata and network structures. By converting these elements into a text format, our app… ▽ More Social media influence campaigns pose significant challenges to public discourse and democracy. Traditional detection methods fall short due to the complexity and dynamic nature of social media. Addressing this, we propose a novel detection method using Large Language Models (LLMs) that incorporates both user metadata and network structures. By converting these elements into a text format, our approach effectively processes multilingual content and adapts to the shifting tactics of malicious campaign actors. We validate our model through rigorous testing on multiple datasets, showcasing its superior performance in identifying influence efforts. This research not only offers a powerful tool for detecting campaigns, but also sets the stage for future enhancements to keep up with the fast-paced evolution of social media-based influence tactics. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Susceptibility to Unreliable Information Sources: Swift Adoption with Minimal Exposure

Authors: Jinyi Ye, Luca Luceri, Julie Jiang, Emilio Ferrara

Abstract: Misinformation proliferation on social media platforms is a pervasive threat to the integrity of online public discourse. Genuine users, susceptible to others' influence, often unknowingly engage with, endorse, and re-share questionable pieces of information, collectively amplifying the spread of misinformation. In this study, we introduce an empirical framework to investigate users' susceptibilit… ▽ More Misinformation proliferation on social media platforms is a pervasive threat to the integrity of online public discourse. Genuine users, susceptible to others' influence, often unknowingly engage with, endorse, and re-share questionable pieces of information, collectively amplifying the spread of misinformation. In this study, we introduce an empirical framework to investigate users' susceptibility to influence when exposed to unreliable and reliable information sources. Leveraging two datasets on political and public health discussions on Twitter, we analyze the impact of exposure on the adoption of information sources, examining how the reliability of the source modulates this relationship. Our findings provide evidence that increased exposure augments the likelihood of adoption. Users tend to adopt low-credibility sources with fewer exposures than high-credibility sources, a trend that persists even among non-partisan users. Furthermore, the number of exposures needed for adoption varies based on the source credibility, with extreme ends of the spectrum (very high or low credibility) requiring fewer exposures for adoption. Additionally, we reveal that the adoption of information sources often mirrors users' prior exposure to sources with comparable credibility levels. Our research offers critical insights for mitigating the endorsement of misinformation by vulnerable users, offering a framework to study the dynamics of content exposure and adoption on social media platforms. △ Less

Submitted 29 January, 2025; v1 submitted 9 November, 2023; originally announced November 2023.

Journal ref: Proceedings of the ACM Web Conference 2024

Unmasking the Web of Deceit: Uncovering Coordinated Activity to Expose Information Operations on Twitter

Authors: Luca Luceri, Valeria Pantè, Keith Burghardt, Emilio Ferrara

Abstract: Social media platforms, particularly Twitter, have become pivotal arenas for influence campaigns, often orchestrated by state-sponsored information operations (IOs). This paper delves into the detection of key players driving IOs by employing similarity graphs constructed from behavioral pattern data. We unveil that well-known, yet underutilized network properties can help accurately identify coor… ▽ More Social media platforms, particularly Twitter, have become pivotal arenas for influence campaigns, often orchestrated by state-sponsored information operations (IOs). This paper delves into the detection of key players driving IOs by employing similarity graphs constructed from behavioral pattern data. We unveil that well-known, yet underutilized network properties can help accurately identify coordinated IO drivers. Drawing from a comprehensive dataset of 49 million tweets from six countries, which includes multiple verified IOs, our study reveals that traditional network filtering techniques do not consistently pinpoint IO drivers across campaigns. We first propose a framework based on node pruning that emerges superior, particularly when combining multiple behavioral indicators across different networks. Then, we introduce a supervised machine learning model that harnesses a vector representation of the fused similarity network. This model, which boasts a precision exceeding 0.95, adeptly classifies IO drivers on a global scale and reliably forecasts their temporal engagements. Our findings are crucial in the fight against deceptive influence campaigns on social media, helping us better understand and detect them. △ Less

Submitted 15 October, 2023; originally announced October 2023.

Comments: Accepted at the 2024 ACM Web Conference

Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation

Authors: Eun Cheol Choi, Emilio Ferrara

Abstract: In today's digital era, the rapid spread of misinformation poses threats to public well-being and societal trust. As online misinformation proliferates, manual verification by fact checkers becomes increasingly challenging. We introduce FACT-GPT (Fact-checking Augmentation with Claim matching Task-oriented Generative Pre-trained Transformer), a framework designed to automate the claim matching pha… ▽ More In today's digital era, the rapid spread of misinformation poses threats to public well-being and societal trust. As online misinformation proliferates, manual verification by fact checkers becomes increasingly challenging. We introduce FACT-GPT (Fact-checking Augmentation with Claim matching Task-oriented Generative Pre-trained Transformer), a framework designed to automate the claim matching phase of fact-checking using Large Language Models (LLMs). This framework identifies new social media content that either supports or contradicts claims previously debunked by fact-checkers. Our approach employs GPT-4 to generate a labeled dataset consisting of simulated social media posts. This data set serves as a training ground for fine-tuning more specialized LLMs. We evaluated FACT-GPT on an extensive dataset of social media content related to public health. The results indicate that our fine-tuned LLMs rival the performance of larger pre-trained LLMs in claim matching tasks, aligning closely with human annotations. This study achieves three key milestones: it provides an automated framework for enhanced fact-checking; demonstrates the potential of LLMs to complement human expertise; offers public resources, including datasets and models, to further research and applications in the fact-checking domain. △ Less

Submitted 13 October, 2023; originally announced October 2023.

Social Approval and Network Homophily as Motivators of Online Toxicity

Authors: Julie Jiang, Luca Luceri, Joseph B. Walther, Emilio Ferrara

Abstract: Online hate messaging is a pervasive issue plaguing the well-being of social media users. This research empirically investigates a novel theory positing that online hate may be driven primarily by the pursuit of social approval rather than a direct desire to harm the targets. Results show that toxicity is homophilous in users' social networks and that a user's propensity for hostility can be predi… ▽ More Online hate messaging is a pervasive issue plaguing the well-being of social media users. This research empirically investigates a novel theory positing that online hate may be driven primarily by the pursuit of social approval rather than a direct desire to harm the targets. Results show that toxicity is homophilous in users' social networks and that a user's propensity for hostility can be predicted by their social networks. We also illustrate how receiving greater or fewer social engagements in the form of likes, retweets, quotes, and replies affects a user's subsequent toxicity. We establish a clear connection between receiving social approval signals and increases in subsequent toxicity. Being retweeted plays a particularly prominent role in escalating toxicity. Results also show that not receiving expected levels of social approval leads to decreased toxicity. We discuss the important implications of our research and opportunities to combat online hate. △ Less

Submitted 29 February, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

Factuality Challenges in the Era of Large Language Models

Authors: Isabelle Augenstein, Timothy Baldwin, Meeyoung Cha, Tanmoy Chakraborty, Giovanni Luca Ciampaglia, David Corney, Renee DiResta, Emilio Ferrara, Scott Hale, Alon Halevy, Eduard Hovy, Heng Ji, Filippo Menczer, Ruben Miguez, Preslav Nakov, Dietram Scheufele, Shivam Sharma, Giovanni Zagni

Abstract: The emergence of tools based on Large Language Models (LLMs), such as OpenAI's ChatGPT, Microsoft's Bing Chat, and Google's Bard, has garnered immense public attention. These incredibly useful, natural-sounding tools mark significant advances in natural language generation, yet they exhibit a propensity to generate false, erroneous, or misleading content -- commonly referred to as "hallucinations.… ▽ More The emergence of tools based on Large Language Models (LLMs), such as OpenAI's ChatGPT, Microsoft's Bing Chat, and Google's Bard, has garnered immense public attention. These incredibly useful, natural-sounding tools mark significant advances in natural language generation, yet they exhibit a propensity to generate false, erroneous, or misleading content -- commonly referred to as "hallucinations." Moreover, LLMs can be exploited for malicious applications, such as generating false but credible-sounding content and profiles at scale. This poses a significant challenge to society in terms of the potential deception of users and the increasing dissemination of inaccurate information. In light of these risks, we explore the kinds of technological innovations, regulatory reforms, and AI literacy initiatives needed from fact-checkers, news organizations, and the broader research and policy communities. By identifying the risks, the imminent threats, and some viable solutions, we seek to shed light on navigating various aspects of veracity in the era of generative AI. △ Less

Submitted 9 October, 2023; v1 submitted 8 October, 2023; originally announced October 2023.

Comments: Our article offers a comprehensive examination of the challenges and risks associated with Large Language Models (LLMs), focusing on their potential impact on the veracity of information in today's digital landscape

GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models

Authors: Emilio Ferrara

Abstract: Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identiti… ▽ More Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are marvels of technology; celebrated for their prowess in natural language processing and multimodal content generation, they promise a transformative future. But as with all powerful tools, they come with their shadows. Picture living in a world where deepfakes are indistinguishable from reality, where synthetic identities orchestrate malicious campaigns, and where targeted misinformation or scams are crafted with unparalleled precision. Welcome to the darker side of GenAI applications. This article is not just a journey through the meanders of potential misuse of GenAI and LLMs, but also a call to recognize the urgency of the challenges ahead. As we navigate the seas of misinformation campaigns, malicious content generation, and the eerie creation of sophisticated malware, we'll uncover the societal implications that ripple through the GenAI revolution we are witnessing. From AI-powered botnets on social media platforms to the unnerving potential of AI to generate fabricated identities, or alibis made of synthetic realities, the stakes have never been higher. The lines between the virtual and the real worlds are blurring, and the consequences of potential GenAI's nefarious applications impact us all. This article serves both as a synthesis of rigorous research presented on the risks of GenAI and misuse of LLMs and as a thought-provoking vision of the different types of harmful GenAI applications we might encounter in the near future, and some ways we can prepare for them. △ Less

Submitted 22 January, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Comments: Accepted in: Journal of Computational Social Science

Journal ref: J Comput Soc Sc (2024)

The Butterfly Effect in Artificial Intelligence Systems: Implications for AI Bias and Fairness

Authors: Emilio Ferrara

Abstract: The Butterfly Effect, a concept originating from chaos theory, underscores how small changes can have significant and unpredictable impacts on complex systems. In the context of AI fairness and bias, the Butterfly Effect can stem from a variety of sources, such as small biases or skewed data inputs during algorithm development, saddle points in training, or distribution shifts in data between trai… ▽ More The Butterfly Effect, a concept originating from chaos theory, underscores how small changes can have significant and unpredictable impacts on complex systems. In the context of AI fairness and bias, the Butterfly Effect can stem from a variety of sources, such as small biases or skewed data inputs during algorithm development, saddle points in training, or distribution shifts in data between training and testing phases. These seemingly minor alterations can lead to unexpected and substantial unfair outcomes, disproportionately affecting underrepresented individuals or groups and perpetuating pre-existing inequalities. Moreover, the Butterfly Effect can amplify inherent biases within data or algorithms, exacerbate feedback loops, and create vulnerabilities for adversarial attacks. Given the intricate nature of AI systems and their societal implications, it is crucial to thoroughly examine any changes to algorithms or input data for potential unintended consequences. In this paper, we envision both algorithmic and empirical strategies to detect, quantify, and mitigate the Butterfly Effect in AI systems, emphasizing the importance of addressing these challenges to promote fairness and ensure responsible AI development. △ Less

Submitted 2 February, 2024; v1 submitted 11 July, 2023; originally announced July 2023.

Comments: Cite as: Machine Learning with Applications, Volume 15, 2024, 100525 10.1016/j.mlwa.2024.100525

Journal ref: Machine Learning with Applications, Volume 15, 2024, 100525

Controlled Text Generation with Hidden Representation Transformations

Authors: Vaibhav Kumar, Hana Koorehdavoudi, Masud Moshtaghi, Amita Misra, Ankit Chadha, Emilio Ferrara

Abstract: We propose CHRT (Control Hidden Representation Transformation) - a controlled language generation framework that steers large language models to generate text pertaining to certain attributes (such as toxicity). CHRT gains attribute control by modifying the hidden representation of the base model through learned transformations. We employ a contrastive-learning framework to learn these transformat… ▽ More We propose CHRT (Control Hidden Representation Transformation) - a controlled language generation framework that steers large language models to generate text pertaining to certain attributes (such as toxicity). CHRT gains attribute control by modifying the hidden representation of the base model through learned transformations. We employ a contrastive-learning framework to learn these transformations that can be combined to gain multi-attribute control. The effectiveness of CHRT is experimentally shown by comparing it with seven baselines over three attributes. CHRT outperforms all the baselines in the task of detoxification, positive sentiment steering, and text simplification while minimizing the loss in linguistic qualities. Further, our approach has the lowest inference latency of only 0.01 seconds more than the base model, making it the most suitable for high-performance production environments. We open-source our code and release two novel datasets to further propel controlled language generation research. △ Less

Submitted 31 May, 2023; v1 submitted 30 May, 2023; originally announced May 2023.

Comments: Accepted at ACL 2023 as a long paper (Findings)

Identifying Informational Sources in News Articles

Authors: Alexander Spangher, Nanyun Peng, Jonathan May, Emilio Ferrara

Abstract: News articles are driven by the informational sources journalists use in reporting. Modeling when, how and why sources get used together in stories can help us better understand the information we consume and even help journalists with the task of producing it. In this work, we take steps toward this goal by constructing the largest and widest-ranging annotated dataset, to date, of informational s… ▽ More News articles are driven by the informational sources journalists use in reporting. Modeling when, how and why sources get used together in stories can help us better understand the information we consume and even help journalists with the task of producing it. In this work, we take steps toward this goal by constructing the largest and widest-ranging annotated dataset, to date, of informational sources used in news writing. We show that our dataset can be used to train high-performing models for information detection and source attribution. We further introduce a novel task, source prediction, to study the compositionality of sources in news articles. We show good performance on this task, which we argue is an important proof for narrative science exploring the internal structure of news articles and aiding in planning-based language generation, and an important step towards a source-recommendation system to aid journalists. △ Less

Submitted 24 May, 2023; originally announced May 2023.

Comments: 13 pages

Fairness And Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, And Mitigation Strategies

Authors: Emilio Ferrara

Abstract: The significant advancements in applying Artificial Intelligence (AI) to healthcare decision-making, medical diagnosis, and other domains have simultaneously raised concerns about the fairness and bias of AI systems. This is particularly critical in areas like healthcare, employment, criminal justice, credit scoring, and increasingly, in generative AI models (GenAI) that produce synthetic media. S… ▽ More The significant advancements in applying Artificial Intelligence (AI) to healthcare decision-making, medical diagnosis, and other domains have simultaneously raised concerns about the fairness and bias of AI systems. This is particularly critical in areas like healthcare, employment, criminal justice, credit scoring, and increasingly, in generative AI models (GenAI) that produce synthetic media. Such systems can lead to unfair outcomes and perpetuate existing inequalities, including generative biases that affect the representation of individuals in synthetic data. This survey paper offers a succinct, comprehensive overview of fairness and bias in AI, addressing their sources, impacts, and mitigation strategies. We review sources of bias, such as data, algorithm, and human decision biases - highlighting the emergent issue of generative AI bias where models may reproduce and amplify societal stereotypes. We assess the societal impact of biased AI systems, focusing on the perpetuation of inequalities and the reinforcement of harmful stereotypes, especially as generative AI becomes more prevalent in creating content that influences public perception. We explore various proposed mitigation strategies, discussing the ethical considerations of their implementation and emphasizing the need for interdisciplinary collaboration to ensure effectiveness. Through a systematic literature review spanning multiple academic disciplines, we present definitions of AI bias and its different types, including a detailed look at generative AI bias. We discuss the negative impacts of AI bias on individuals and society and provide an overview of current approaches to mitigate AI bias, including data pre-processing, model selection, and post-processing. We emphasize the unique challenges presented by generative AI models and the importance of strategies specifically tailored to address these. △ Less

Submitted 7 December, 2023; v1 submitted 15 April, 2023; originally announced April 2023.

Journal ref: Sci 2024, 6(1), 3