-
Conversate: Supporting Reflective Learning in Interview Practice Through Interactive Simulation and Dialogic Feedback
Authors:
Taufiq Daryanto,
Xiaohan Ding,
Lance T. Wilhelm,
Sophia Stil,
Kirk McInnis Knutsen,
Eugenia H. Rho
Abstract:
Job interviews play a critical role in shaping one's career, yet practicing interview skills can be challenging, especially without access to human coaches or peers for feedback. Recent advancements in large language models (LLMs) present an opportunity to enhance the interview practice experience. Yet, little research has explored the effectiveness and user perceptions of such systems or the bene…
▽ More
Job interviews play a critical role in shaping one's career, yet practicing interview skills can be challenging, especially without access to human coaches or peers for feedback. Recent advancements in large language models (LLMs) present an opportunity to enhance the interview practice experience. Yet, little research has explored the effectiveness and user perceptions of such systems or the benefits and challenges of using LLMs for interview practice. Furthermore, while prior work and recent commercial tools have demonstrated the potential of AI to assist with interview practice, they often deliver one-way feedback, where users only receive information about their performance. By contrast, dialogic feedback, a concept developed in learning sciences, is a two-way interaction feedback process that allows users to further engage with and learn from the provided feedback through interactive dialogue. This paper introduces Conversate, a web-based application that supports reflective learning in job interview practice by leveraging large language models (LLMs) for interactive interview simulations and dialogic feedback. To start the interview session, the user provides the title of a job position (e.g., entry-level software engineer) in the system. Then, our system will initialize the LLM agent to start the interview simulation by asking the user an opening interview question and following up with questions carefully adapted to subsequent user responses. After the interview session, our back-end LLM framework will then analyze the user's responses and highlight areas for improvement. Users can then annotate the transcript by selecting specific sections and writing self-reflections. Finally, the user can interact with the system for dialogic feedback, conversing with the LLM agent to learn from and iteratively refine their answers based on the agent's guidance.
△ Less
Submitted 1 November, 2024; v1 submitted 7 October, 2024;
originally announced October 2024.
-
CounterQuill: Investigating the Potential of Human-AI Collaboration in Online Counterspeech Writing
Authors:
Xiaohan Ding,
Kaike Ping,
Uma Sushmitha Gunturi,
Buse Carik,
Sophia Stil,
Lance T Wilhelm,
Taufiq Daryanto,
James Hawdon,
Sang Won Lee,
Eugenia H Rho
Abstract:
Online hate speech has become increasingly prevalent on social media, causing harm to individuals and society. While automated content moderation has received considerable attention, user-driven counterspeech remains a less explored yet promising approach. However, many people face difficulties in crafting effective responses. We introduce CounterQuill, a human-AI collaborative system that helps e…
▽ More
Online hate speech has become increasingly prevalent on social media, causing harm to individuals and society. While automated content moderation has received considerable attention, user-driven counterspeech remains a less explored yet promising approach. However, many people face difficulties in crafting effective responses. We introduce CounterQuill, a human-AI collaborative system that helps everyday users with writing empathetic counterspeech, not by generating automatic replies, but by educating them through reflection and response. CounterQuill follows a three-stage workflow grounded in computational thinking: (1) a learning session to build understanding of hate speech and counterspeech, (2) a brainstorming session to identify harmful patterns and ideate counterspeech ideas, and (3) a co-writing session that helps users refine their counter responses while preserving personal voice. Through a user study \r{ho}(N=20), we found that CounterQuill helped participants develop the skills to brainstorm and draft counterspeech with increased confidence and control throughout the process. Our findings highlight how AI systems can scaffold complex communication tasks through structured, human-centered workflows that educate users on how to recognize, reflect on, and respond to online hate speech.
△ Less
Submitted 16 May, 2025; v1 submitted 3 October, 2024;
originally announced October 2024.
-
Detecting Frames in News Headlines and Lead Images in U.S. Gun Violence Coverage
Authors:
Isidora Chara Tourni,
Lei Guo,
Hengchang Hu,
Edward Halim,
Prakash Ishwar,
Taufiq Daryanto,
Mona Jalal,
Boqi Chen,
Margrit Betke,
Fabian Zhafransyah,
Sha Lai,
Derry Tanti Wijaya
Abstract:
News media structure their reporting of events or issues using certain perspectives.
When describing an incident involving gun violence, for example, some journalists may focus on mental health or gun regulation, while others may emphasize the discussion of gun rights. Such perspectives are called \say{frames} in communication research. We study, for the first time, the value of combining lead i…
▽ More
News media structure their reporting of events or issues using certain perspectives.
When describing an incident involving gun violence, for example, some journalists may focus on mental health or gun regulation, while others may emphasize the discussion of gun rights. Such perspectives are called \say{frames} in communication research. We study, for the first time, the value of combining lead images and their contextual information with text to identify the frame of a given news article. We observe that using multiple modes of information(article- and image-derived features) improves prediction of news frames over any single mode of information when the images are relevant to the frames of the headlines. We also observe that frame image relevance is related to the ease of conveying frames via images, which we call frame concreteness. Additionally, we release the first multimodal news framing dataset related to gun violence in the U.S., curated and annotated by communication researchers. The dataset will allow researchers to further examine the use of multiple information modalities for studying media framing.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Referring Expressions with Rational Speech Act Framework: A Probabilistic Approach
Authors:
Hieu Le,
Taufiq Daryanto,
Fabian Zhafransyah,
Derry Wijaya,
Elizabeth Coppock,
Sang Chin
Abstract:
This paper focuses on a referring expression generation (REG) task in which the aim is to pick out an object in a complex visual scene. One common theoretical approach to this problem is to model the task as a two-agent cooperative scheme in which a `speaker' agent would generate the expression that best describes a targeted area and a `listener' agent would identify the target. Several recent REG…
▽ More
This paper focuses on a referring expression generation (REG) task in which the aim is to pick out an object in a complex visual scene. One common theoretical approach to this problem is to model the task as a two-agent cooperative scheme in which a `speaker' agent would generate the expression that best describes a targeted area and a `listener' agent would identify the target. Several recent REG systems have used deep learning approaches to represent the speaker/listener agents. The Rational Speech Act framework (RSA), a Bayesian approach to pragmatics that can predict human linguistic behavior quite accurately, has been shown to generate high quality and explainable expressions on toy datasets involving simple visual scenes. Its application to large scale problems, however, remains largely unexplored. This paper applies a combination of the probabilistic RSA framework and deep learning approaches to larger datasets involving complex visual scenes in a multi-step process with the aim of generating better-explained expressions. We carry out experiments on the RefCOCO and RefCOCO+ datasets and compare our approach with other end-to-end deep learning approaches as well as a variation of RSA to highlight our key contribution. Experimental results show that while achieving lower accuracy than SOTA deep learning methods, our approach outperforms similar RSA approach in human comprehension and has an advantage over end-to-end deep learning under limited data scenario. Lastly, we provide a detailed analysis on the expression generation process with concrete examples, thus providing a systematic view on error types and deficiencies in the generation process and identifying possible areas for future improvements.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
