-
Rethinking stance detection: A theoretically-informed research agenda for user-level inference using language models
Authors:
Prasanta Bhattacharya,
Hong Zhang,
Yiming Cao,
Wei Gao,
Brandon Siyuan Loh,
Joseph J. P. Simons,
Liang Ze Wong
Abstract:
Stance detection has emerged as a popular task in natural language processing research, enabled largely by the abundance of target-specific social media data. While there has been considerable research on the development of stance detection models, datasets, and application, we highlight important gaps pertaining to (i) a lack of theoretical conceptualization of stance, and (ii) the treatment of s…
▽ More
Stance detection has emerged as a popular task in natural language processing research, enabled largely by the abundance of target-specific social media data. While there has been considerable research on the development of stance detection models, datasets, and application, we highlight important gaps pertaining to (i) a lack of theoretical conceptualization of stance, and (ii) the treatment of stance at an individual- or user-level, as opposed to message-level. In this paper, we first review the interdisciplinary origins of stance as an individual-level construct to highlight relevant attributes (e.g., psychological features) that might be useful to incorporate in stance detection models. Further, we argue that recent pre-trained and large language models (LLMs) might offer a way to flexibly infer such user-level attributes and/or incorporate them in modelling stance. To better illustrate this, we briefly review and synthesize the emerging corpus of studies on using LLMs for inferring stance, and specifically on incorporating user attributes in such tasks. We conclude by proposing a four-point agenda for pursuing stance detection research that is theoretically informed, inclusive, and practically impactful.
△ Less
Submitted 4 February, 2025;
originally announced February 2025.
-
From traces to measures: Large language models as a tool for psychological measurement from text
Authors:
Joseph J. P. Simons,
Wong Liang Ze,
Prasanta Bhattacharya,
Brandon Siyuan Loh,
Wei Gao
Abstract:
Large language models are increasingly being used to label or rate psychological features in text data. This approach helps address one of the limiting factors of digital trace data - their lack of an inherent target of measurement. However, this approach is also a form of psychological measurement (using observable variables to quantify a hypothetical latent construct). As such, these ratings are…
▽ More
Large language models are increasingly being used to label or rate psychological features in text data. This approach helps address one of the limiting factors of digital trace data - their lack of an inherent target of measurement. However, this approach is also a form of psychological measurement (using observable variables to quantify a hypothetical latent construct). As such, these ratings are subject to the same psychometric considerations of reliability and validity as more standard psychological measures. Here we present a workflow for developing and evaluating large language model based measures of psychological features which incorporate these considerations. We also provide an example, attempting to measure the previously established constructs of attitude certainty, importance and moralization from text. Using a pool of prompts adapted from existing measurement instruments, we find they have good levels of internal consistency but only partially meet validity criteria.
△ Less
Submitted 13 October, 2024; v1 submitted 12 May, 2024;
originally announced May 2024.
-
Enhancing Stance Classification on Social Media Using Quantified Moral Foundations
Authors:
Hong Zhang,
Quoc-Nam Nguyen,
Prasanta Bhattacharya,
Wei Gao,
Liang Ze Wong,
Brandon Siyuan Loh,
Joseph J. P. Simons,
Jisun An
Abstract:
This study enhances stance detection on social media by incorporating deeper psychological attributes, specifically individuals' moral foundations. These theoretically-derived dimensions aim to provide a comprehensive profile of an individual's moral concerns which, in recent work, has been linked to behaviour in a range of domains, including society, politics, health, and the environment. In this…
▽ More
This study enhances stance detection on social media by incorporating deeper psychological attributes, specifically individuals' moral foundations. These theoretically-derived dimensions aim to provide a comprehensive profile of an individual's moral concerns which, in recent work, has been linked to behaviour in a range of domains, including society, politics, health, and the environment. In this paper, we investigate how moral foundation dimensions can contribute to predicting an individual's stance on a given target. Specifically we incorporate moral foundation features extracted from text, along with message semantic features, to classify stances at both message- and user-levels using both traditional machine learning models and large language models. Our preliminary results suggest that encoding moral foundations can enhance the performance of stance detection tasks and help illuminate the associations between specific moral foundations and online stances on target topics. The results highlight the importance of considering deeper psychological attributes in stance analysis and underscores the role of moral foundations in guiding online social behavior.
△ Less
Submitted 29 September, 2024; v1 submitted 15 October, 2023;
originally announced October 2023.
-
Psychological Frameworks for Persuasive Information and Communications Technologies
Authors:
Joseph J. P. Simons
Abstract:
When developing devices to encourage positive change in users, social psychology can offer useful conceptual resources. This article outlines three major theories from the discipline and discusses their implications for designing persuasive technologies.
When developing devices to encourage positive change in users, social psychology can offer useful conceptual resources. This article outlines three major theories from the discipline and discusses their implications for designing persuasive technologies.
△ Less
Submitted 28 August, 2018;
originally announced August 2018.
