-
"Who Has the Time?": Understanding Receptivity to Health Chatbots among Underserved Women in India
Authors:
Manvi S,
Roshini Deva,
Neha Madhiwalla,
Azra Ismail
Abstract:
Access to health information and services among women continues to be a major challenge in many communities globally. In recent years, there has been a growing interest in the potential of chatbots to address this information and access gap. We conducted interviews and focus group discussions with underserved women in urban India to understand their receptivity towards the use of chatbots for mate…
▽ More
Access to health information and services among women continues to be a major challenge in many communities globally. In recent years, there has been a growing interest in the potential of chatbots to address this information and access gap. We conducted interviews and focus group discussions with underserved women in urban India to understand their receptivity towards the use of chatbots for maternal and child health, as well as barriers to their adoption. Our findings uncover gaps in digital access and literacies, and perceived conflict with various responsibilities that women are burdened with, which shape their interactions with digital technology. Our paper offers insights into the design of chatbots for community health that can meet the lived realities of women in underserved settings.
△ Less
Submitted 21 February, 2025;
originally announced February 2025.
-
"Kya family planning after marriage hoti hai?": Integrating Cultural Sensitivity in an LLM Chatbot for Reproductive Health
Authors:
Roshini Deva,
Dhruv Ramani,
Tanvi Divate,
Suhani Jalota,
Azra Ismail
Abstract:
Access to sexual and reproductive health information remains a challenge in many communities globally, due to cultural taboos and limited availability of healthcare providers. Public health organizations are increasingly turning to Large Language Models (LLMs) to improve access to timely and personalized information. However, recent HCI scholarship indicates that significant challenges remain in i…
▽ More
Access to sexual and reproductive health information remains a challenge in many communities globally, due to cultural taboos and limited availability of healthcare providers. Public health organizations are increasingly turning to Large Language Models (LLMs) to improve access to timely and personalized information. However, recent HCI scholarship indicates that significant challenges remain in incorporating context awareness and mitigating bias in LLMs. In this paper, we study the development of a culturally-appropriate LLM-based chatbot for reproductive health with underserved women in urban India. Through user interactions, focus groups, and interviews with multiple stakeholders, we examine the chatbot's response to sensitive and highly contextual queries on reproductive health. Our findings reveal strengths and limitations of the system in capturing local context, and complexities around what constitutes "culture". Finally, we discuss how local context might be better integrated, and present a framework to inform the design of culturally-sensitive chatbots for community health.
△ Less
Submitted 21 February, 2025;
originally announced February 2025.
-
A Mixed-Methods Evaluation of LLM-Based Chatbots for Menopause
Authors:
Roshini Deva,
Manvi S,
Jasmine Zhou,
Elizabeth Britton Chahine,
Agena Davenport-Nicholson,
Nadi Nina Kaonga,
Selen Bozkurt,
Azra Ismail
Abstract:
The integration of Large Language Models (LLMs) into healthcare settings has gained significant attention, particularly for question-answering tasks. Given the high-stakes nature of healthcare, it is essential to ensure that LLM-generated content is accurate and reliable to prevent adverse outcomes. However, the development of robust evaluation metrics and methodologies remains a matter of much de…
▽ More
The integration of Large Language Models (LLMs) into healthcare settings has gained significant attention, particularly for question-answering tasks. Given the high-stakes nature of healthcare, it is essential to ensure that LLM-generated content is accurate and reliable to prevent adverse outcomes. However, the development of robust evaluation metrics and methodologies remains a matter of much debate. We examine the performance of publicly available LLM-based chatbots for menopause-related queries, using a mixed-methods approach to evaluate safety, consensus, objectivity, reproducibility, and explainability. Our findings highlight the promise and limitations of traditional evaluation metrics for sensitive health topics. We propose the need for customized and ethically grounded evaluation frameworks to assess LLMs to advance safe and effective use in healthcare.
△ Less
Submitted 5 February, 2025;
originally announced February 2025.
