-
Effects of Theory of Mind and Prosocial Beliefs on Steering Human-Aligned Behaviors of LLMs in Ultimatum Games
Authors:
Neemesh Yadav,
Palakorn Achananuparp,
Jing Jiang,
Ee-Peng Lim
Abstract:
Large Language Models (LLMs) have shown potential in simulating human behaviors and performing theory-of-mind (ToM) reasoning, a crucial skill for complex social interactions. In this study, we investigate the role of ToM reasoning in aligning agentic behaviors with human norms in negotiation tasks, using the ultimatum game as a controlled environment. We initialized LLM agents with different pros…
▽ More
Large Language Models (LLMs) have shown potential in simulating human behaviors and performing theory-of-mind (ToM) reasoning, a crucial skill for complex social interactions. In this study, we investigate the role of ToM reasoning in aligning agentic behaviors with human norms in negotiation tasks, using the ultimatum game as a controlled environment. We initialized LLM agents with different prosocial beliefs (including Greedy, Fair, and Selfless) and reasoning methods like chain-of-thought (CoT) and varying ToM levels, and examined their decision-making processes across diverse LLMs, including reasoning models like o3-mini and DeepSeek-R1 Distilled Qwen 32B. Results from 2,700 simulations indicated that ToM reasoning enhances behavior alignment, decision-making consistency, and negotiation outcomes. Consistent with previous findings, reasoning models exhibit limited capability compared to models with ToM reasoning, different roles of the game benefits with different orders of ToM reasoning. Our findings contribute to the understanding of ToM's role in enhancing human-AI interaction and cooperative decision-making. The code used for our experiments can be found at https://github.com/Stealth-py/UltimatumToM.
△ Less
Submitted 30 May, 2025;
originally announced May 2025.
-
A Multi-Stage Framework with Taxonomy-Guided Reasoning for Occupation Classification Using Large Language Models
Authors:
Palakorn Achananuparp,
Ee-Peng Lim
Abstract:
Automatically annotating job data with standardized occupations from taxonomies, known as occupation classification, is crucial for labor market analysis. However, this task is often hindered by data scarcity and the challenges of manual annotations. While large language models (LLMs) hold promise due to their extensive world knowledge and in-context learning capabilities, their effectiveness depe…
▽ More
Automatically annotating job data with standardized occupations from taxonomies, known as occupation classification, is crucial for labor market analysis. However, this task is often hindered by data scarcity and the challenges of manual annotations. While large language models (LLMs) hold promise due to their extensive world knowledge and in-context learning capabilities, their effectiveness depends on their knowledge of occupational taxonomies, which remains unclear. In this study, we assess the ability of LLMs to generate precise taxonomic entities from taxonomy, highlighting their limitations. To address these challenges, we propose a multi-stage framework consisting of inference, retrieval, and reranking stages, which integrates taxonomy-guided reasoning examples to enhance performance by aligning outputs with taxonomic knowledge. Evaluations on a large-scale dataset show significant improvements in classification accuracy. Furthermore, we demonstrate the framework's adaptability for multi-label skill classification. Our results indicate that the framework outperforms existing LLM-based methods, offering a practical and scalable solution for occupation classification and related tasks across LLMs.
△ Less
Submitted 17 March, 2025;
originally announced March 2025.
-
CAMI: A Counselor Agent Supporting Motivational Interviewing through State Inference and Topic Exploration
Authors:
Yizhe Yang,
Palakorn Achananuparp,
Heyan Huang,
Jing Jiang,
Kit Phey Leng,
Nicholas Gabriel Lim,
Cameron Tan Shi Ern,
Ee-peng Lim
Abstract:
Conversational counselor agents have become essential tools for addressing the rising demand for scalable and accessible mental health support. This paper introduces CAMI, a novel automated counselor agent grounded in Motivational Interviewing (MI) -- a client-centered counseling approach designed to address ambivalence and facilitate behavior change. CAMI employs a novel STAR framework, consistin…
▽ More
Conversational counselor agents have become essential tools for addressing the rising demand for scalable and accessible mental health support. This paper introduces CAMI, a novel automated counselor agent grounded in Motivational Interviewing (MI) -- a client-centered counseling approach designed to address ambivalence and facilitate behavior change. CAMI employs a novel STAR framework, consisting of client's state inference, motivation topic exploration, and response generation modules, leveraging large language models (LLMs). These components work together to evoke change talk, aligning with MI principles and improving counseling outcomes for clients from diverse backgrounds. We evaluate CAMI's performance through both automated and manual evaluations, utilizing simulated clients to assess MI skill competency, client's state inference accuracy, topic exploration proficiency, and overall counseling success. Results show that CAMI not only outperforms several state-of-the-art methods but also shows more realistic counselor-like behavior. Additionally, our ablation study underscores the critical roles of state inference and topic exploration in achieving this performance.
△ Less
Submitted 4 February, 2025;
originally announced February 2025.
-
Consistent Client Simulation for Motivational Interviewing-based Counseling
Authors:
Yizhe Yang,
Palakorn Achananuparp,
Heyan Huang,
Jing Jiang,
John Pinto,
Jenny Giam,
Kit Phey Leng,
Nicholas Gabriel Lim,
Cameron Tan Shi Ern,
Ee-peng Lim
Abstract:
Simulating human clients in mental health counseling is crucial for training and evaluating counselors (both human or simulated) in a scalable manner. Nevertheless, past research on client simulation did not focus on complex conversation tasks such as mental health counseling. In these tasks, the challenge is to ensure that the client's actions (i.e., interactions with the counselor) are consisten…
▽ More
Simulating human clients in mental health counseling is crucial for training and evaluating counselors (both human or simulated) in a scalable manner. Nevertheless, past research on client simulation did not focus on complex conversation tasks such as mental health counseling. In these tasks, the challenge is to ensure that the client's actions (i.e., interactions with the counselor) are consistent with with its stipulated profiles and negative behavior settings. In this paper, we propose a novel framework that supports consistent client simulation for mental health counseling. Our framework tracks the mental state of a simulated client, controls its state transitions, and generates for each state behaviors consistent with the client's motivation, beliefs, preferred plan to change, and receptivity. By varying the client profile and receptivity, we demonstrate that consistent simulated clients for different counseling scenarios can be effectively created. Both our automatic and expert evaluations on the generated counseling sessions also show that our client simulation method achieves higher consistency than previous methods.
△ Less
Submitted 4 February, 2025;
originally announced February 2025.
-
Speaker Verification in Agent-Generated Conversations
Authors:
Yizhe Yang,
Palakorn Achananuparp,
Heyan Huang,
Jing Jiang,
Ee-Peng Lim
Abstract:
The recent success of large language models (LLMs) has attracted widespread interest to develop role-playing conversational agents personalized to the characteristics and styles of different speakers to enhance their abilities to perform both general and special purpose dialogue tasks. However, the ability to personalize the generated utterances to speakers, whether conducted by human or LLM, has…
▽ More
The recent success of large language models (LLMs) has attracted widespread interest to develop role-playing conversational agents personalized to the characteristics and styles of different speakers to enhance their abilities to perform both general and special purpose dialogue tasks. However, the ability to personalize the generated utterances to speakers, whether conducted by human or LLM, has not been well studied. To bridge this gap, our study introduces a novel evaluation challenge: speaker verification in agent-generated conversations, which aimed to verify whether two sets of utterances originate from the same speaker. To this end, we assemble a large dataset collection encompassing thousands of speakers and their utterances. We also develop and evaluate speaker verification models under experiment setups. We further utilize the speaker verification models to evaluate the personalization abilities of LLM-based role-playing models. Comprehensive experiments suggest that the current role-playing models fail in accurately mimicking speakers, primarily due to their inherent linguistic characteristics.
△ Less
Submitted 5 June, 2024; v1 submitted 16 May, 2024;
originally announced May 2024.
-
Transforming Facial Weight of Real Images by Editing Latent Space of StyleGAN
Authors:
V N S Rama Krishna Pinnimty,
Matt Zhao,
Palakorn Achananuparp,
Ee-Peng Lim
Abstract:
We present an invert-and-edit framework to automatically transform facial weight of an input face image to look thinner or heavier by leveraging semantic facial attributes encoded in the latent space of Generative Adversarial Networks (GANs). Using a pre-trained StyleGAN as the underlying generator, we first employ an optimization-based embedding method to invert the input image into the StyleGAN…
▽ More
We present an invert-and-edit framework to automatically transform facial weight of an input face image to look thinner or heavier by leveraging semantic facial attributes encoded in the latent space of Generative Adversarial Networks (GANs). Using a pre-trained StyleGAN as the underlying generator, we first employ an optimization-based embedding method to invert the input image into the StyleGAN latent space. Then, we identify the facial-weight attribute direction in the latent space via supervised learning and edit the inverted latent code by moving it positively or negatively along the extracted feature axis. Our framework is empirically shown to produce high-quality and realistic facial-weight transformations without requiring training GANs with a large amount of labeled face images from scratch. Ultimately, our framework can be utilized as part of an intervention to motivate individuals to make healthier food choices by visualizing the future impacts of their behavior on appearance.
△ Less
Submitted 4 November, 2020;
originally announced November 2020.
-
Foodbot: A Goal-Oriented Just-in-Time Healthy Eating Interventions Chatbot
Authors:
Philips Kokoh Prasetyo,
Palakorn Achananuparp,
Ee-Peng Lim
Abstract:
Recent research has identified a few design flaws in popular mobile health (mHealth) applications for promoting healthy eating lifestyle, such as mobile food journals. These include tediousness of manual food logging, inadequate food database coverage, and a lack of healthy dietary goal setting. To address these issues, we present Foodbot, a chatbot-based mHealth application for goal-oriented just…
▽ More
Recent research has identified a few design flaws in popular mobile health (mHealth) applications for promoting healthy eating lifestyle, such as mobile food journals. These include tediousness of manual food logging, inadequate food database coverage, and a lack of healthy dietary goal setting. To address these issues, we present Foodbot, a chatbot-based mHealth application for goal-oriented just-in-time (JIT) healthy eating interventions. Powered by a large-scale food knowledge graph, Foodbot utilizes automatic speech recognition and mobile messaging interface to record food intake. Moreover, Foodbot allows users to set goals and guides their behavior toward the goals via JIT notification prompts, interactive dialogues, and personalized recommendation. Altogether, the Foodbot framework demonstrates the use of open-source data, tools, and platforms to build a practical mHealth solution for supporting healthy eating lifestyle in the general population.
△ Less
Submitted 11 September, 2020;
originally announced September 2020.
-
Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images and Recipes with Semantic Consistency and Attention Mechanism
Authors:
Hao Wang,
Doyen Sahoo,
Chenghao Liu,
Ke Shu,
Palakorn Achananuparp,
Ee-peng Lim,
Steven C. H. Hoi
Abstract:
Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such…
▽ More
Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another. Two major challenges in addressing this problem are 1) large intra-variance and small inter-variance across cross-modal food data; and 2) difficulties in obtaining discriminative recipe representations. To address these two problems, we propose Semantic-Consistent and Attention-based Networks (SCAN), which regularize the embeddings of the two modalities through aligning output semantic probabilities. Besides, we exploit a self-attention mechanism to improve the embedding of recipes. We evaluate the performance of the proposed method on the large-scale Recipe1M dataset, and show that we can outperform several state-of-the-art cross-modal retrieval strategies for food images and cooking recipes by a significant margin.
△ Less
Submitted 20 September, 2021; v1 submitted 9 March, 2020;
originally announced March 2020.
-
RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System
Authors:
Helena H. Lee,
Ke Shu,
Palakorn Achananuparp,
Philips Kokoh Prasetyo,
Yue Liu,
Ee-Peng Lim,
Lav R. Varshney
Abstract:
Interests in the automatic generation of cooking recipes have been growing steadily over the past few years thanks to a large amount of online cooking recipes. We present RecipeGPT, a novel online recipe generation and evaluation system. The system provides two modes of text generations: (1) instruction generation from given recipe title and ingredients; and (2) ingredient generation from recipe t…
▽ More
Interests in the automatic generation of cooking recipes have been growing steadily over the past few years thanks to a large amount of online cooking recipes. We present RecipeGPT, a novel online recipe generation and evaluation system. The system provides two modes of text generations: (1) instruction generation from given recipe title and ingredients; and (2) ingredient generation from recipe title and cooking instructions. Its back-end text generation module comprises a generative pre-trained language model GPT-2 fine-tuned on a large cooking recipe dataset. Moreover, the recipe evaluation module allows the users to conveniently inspect the quality of the generated recipe contents and store the results for future reference. RecipeGPT can be accessed online at https://recipegpt.org/.
△ Less
Submitted 5 March, 2020;
originally announced March 2020.
-
FoodAI: Food Image Recognition via Deep Learning for Smart Food Logging
Authors:
Doyen Sahoo,
Wang Hao,
Shu Ke,
Wu Xiongwei,
Hung Le,
Palakorn Achananuparp,
Ee-Peng Lim,
Steven C. H. Hoi
Abstract:
An important aspect of health monitoring is effective logging of food consumption. This can help management of diet-related diseases like obesity, diabetes, and even cardiovascular diseases. Moreover, food logging can help fitness enthusiasts, and people who wanting to achieve a target weight. However, food-logging is cumbersome, and requires not only taking additional effort to note down the food…
▽ More
An important aspect of health monitoring is effective logging of food consumption. This can help management of diet-related diseases like obesity, diabetes, and even cardiovascular diseases. Moreover, food logging can help fitness enthusiasts, and people who wanting to achieve a target weight. However, food-logging is cumbersome, and requires not only taking additional effort to note down the food item consumed regularly, but also sufficient knowledge of the food item consumed (which is difficult due to the availability of a wide variety of cuisines). With increasing reliance on smart devices, we exploit the convenience offered through the use of smart phones and propose a smart-food logging system: FoodAI, which offers state-of-the-art deep-learning based image recognition capabilities. FoodAI has been developed in Singapore and is particularly focused on food items commonly consumed in Singapore. FoodAI models were trained on a corpus of 400,000 food images from 756 different classes. In this paper we present extensive analysis and insights into the development of this system. FoodAI has been deployed as an API service and is one of the components powering Healthy 365, a mobile app developed by Singapore's Heath Promotion Board. We have over 100 registered organizations (universities, companies, start-ups) subscribing to this service and actively receive several API requests a day. FoodAI has made food logging convenient, aiding smart consumption and a healthy lifestyle.
△ Less
Submitted 26 September, 2019;
originally announced September 2019.
-
Estimating Glycemic Impact of Cooking Recipes via Online Crowdsourcing and Machine Learning
Authors:
Helena Lee,
Palakorn Achananuparp,
Yue Liu,
Ee-Peng Lim,
Lav R. Varshney
Abstract:
Consumption of diets with low glycemic impact is highly recommended for diabetics and pre-diabetics as it helps maintain their blood glucose levels. However, laboratory analysis of dietary glycemic potency is time-consuming and expensive. In this paper, we explore a data-driven approach utilizing online crowdsourcing and machine learning to estimate the glycemic impact of cooking recipes. We show…
▽ More
Consumption of diets with low glycemic impact is highly recommended for diabetics and pre-diabetics as it helps maintain their blood glucose levels. However, laboratory analysis of dietary glycemic potency is time-consuming and expensive. In this paper, we explore a data-driven approach utilizing online crowdsourcing and machine learning to estimate the glycemic impact of cooking recipes. We show that a commonly used healthiness metric may not always be effective in determining recipes suitable for diabetics, thus emphasizing the importance of the glycemic-impact estimation task. Our best classification model, trained on nutritional and crowdsourced data obtained from Amazon Mechanical Turk (AMT), can accurately identify recipes which are unhealthful for diabetics.
△ Less
Submitted 17 September, 2019;
originally announced September 2019.
-
Characterizing and Predicting Repeat Food Consumption Behavior for Just-in-Time Interventions
Authors:
Yue Liu,
Helena Lee,
Palakorn Achananuparp,
Ee-Peng Lim,
Tzu-Ling Cheng,
Shou-De Lin
Abstract:
Human beings are creatures of habit. In their daily life, people tend to repeatedly consume similar types of food items over several days and occasionally switch to consuming different types of items when the consumptions become overly monotonous. However, the novel and repeat consumption behaviors have not been studied in food recommendation research. More importantly, the ability to predict dail…
▽ More
Human beings are creatures of habit. In their daily life, people tend to repeatedly consume similar types of food items over several days and occasionally switch to consuming different types of items when the consumptions become overly monotonous. However, the novel and repeat consumption behaviors have not been studied in food recommendation research. More importantly, the ability to predict daily eating habits of individuals is crucial to improve the effectiveness of food recommender systems in facilitating healthy lifestyle change. In this study, we analyze the patterns of repeat food consumptions using large-scale consumption data from a popular online fitness community called MyFitnessPal (MFP), conduct an offline evaluation of various state-of-the-art algorithms in predicting the next-day food consumption, and analyze their performance across different demographic groups and contexts. The experiment results show that algorithms incorporating the exploration-and-exploitation and temporal dynamics are more effective in the next-day recommendation task than most state-of-the-art algorithms.
△ Less
Submitted 17 September, 2019;
originally announced September 2019.
-
Eat & Tell: A Randomized Trial of Random-Loss Incentive to Increase Dietary Self-Tracking Compliance
Authors:
Palakorn Achananuparp,
Ee-Peng Lim,
Vibhanshu Abhishek,
Tianjiao Yun
Abstract:
A growing body of evidence has shown that incorporating behavioral economics principles into the design of financial incentive programs helps improve their cost-effectiveness, promote individuals' short-term engagement, and increase compliance in health behavior interventions. Yet, their effects on long-term engagement have not been fully examined. In study designs where repeated administration of…
▽ More
A growing body of evidence has shown that incorporating behavioral economics principles into the design of financial incentive programs helps improve their cost-effectiveness, promote individuals' short-term engagement, and increase compliance in health behavior interventions. Yet, their effects on long-term engagement have not been fully examined. In study designs where repeated administration of incentives is required to ensure the regularity of behaviors, the effectiveness of subsequent incentives may decrease as a result of the law of diminishing marginal utility. In this paper, we introduce random-loss incentive -- a new financial incentive based on loss aversion and unpredictability principles -- to address the problem of individuals' growing insensitivity to repeated interventions over time. We evaluate the new incentive design by conducting a randomized controlled trial to measure the influences of random losses on participants' dietary self-tracking and self-reporting compliance using a mobile web application called Eat & Tell. The results show that random losses are significantly more effective than fixed losses in encouraging long-term engagement.
△ Less
Submitted 3 May, 2018;
originally announced May 2018.
-
Does Journaling Encourage Healthier Choices? Analyzing Healthy Eating Behaviors of Food Journalers
Authors:
Palakorn Achananuparp,
Ee-Peng Lim,
Vibhanshu Abhishek
Abstract:
Past research has shown the benefits of food journaling in promoting mindful eating and healthier food choices. However, the links between journaling and healthy eating have not been thoroughly examined. Beyond caloric restriction, do journalers consistently and sufficiently consume healthful diets? How different are their eating habits compared to those of average consumers who tend to be less co…
▽ More
Past research has shown the benefits of food journaling in promoting mindful eating and healthier food choices. However, the links between journaling and healthy eating have not been thoroughly examined. Beyond caloric restriction, do journalers consistently and sufficiently consume healthful diets? How different are their eating habits compared to those of average consumers who tend to be less conscious about health? In this study, we analyze the healthy eating behaviors of active food journalers using data from MyFitnessPal. Surprisingly, our findings show that food journalers do not eat as healthily as they should despite their proclivity to health eating and their food choices resemble those of the general populace. Furthermore, we find that the journaling duration is only a marginal determinant of healthy eating outcomes and sociodemographic factors, such as gender and regions of residence, are much more predictive of healthy food choices.
△ Less
Submitted 3 May, 2018;
originally announced May 2018.
-
Extracting Food Substitutes From Food Diary via Distributional Similarity
Authors:
Palakorn Achananuparp,
Ingmar Weber
Abstract:
In this paper, we explore the problem of identifying substitute relationship between food pairs from real-world food consumption data as the first step towards the healthier food recommendation. Our method is inspired by the distributional hypothesis in linguistics. Specifically, we assume that foods that are consumed in similar contexts are more likely to be similar dietarily. For example, a turk…
▽ More
In this paper, we explore the problem of identifying substitute relationship between food pairs from real-world food consumption data as the first step towards the healthier food recommendation. Our method is inspired by the distributional hypothesis in linguistics. Specifically, we assume that foods that are consumed in similar contexts are more likely to be similar dietarily. For example, a turkey sandwich can be considered a suitable substitute for a chicken sandwich if both tend to be consumed with french fries and salad. To evaluate our method, we constructed a real-world food consumption dataset from MyFitnessPal's public food diary entries and obtained ground-truth human judgements of food substitutes from a crowdsourcing service. The experiment results suggest the effectiveness of the method in identifying suitable substitutes.
△ Less
Submitted 29 July, 2016;
originally announced July 2016.
-
Insights from Machine-Learned Diet Success Prediction
Authors:
Ingmar Weber,
Palakorn Achananuparp
Abstract:
To support people trying to lose weight and stay healthy, more and more fitness apps have sprung up including the ability to track both calories intake and expenditure. Users of such apps are part of a wider ``quantified self'' movement and many opt-in to publicly share their logged data. In this paper, we use public food diaries of more than 4,000 long-term active MyFitnessPal users to study the…
▽ More
To support people trying to lose weight and stay healthy, more and more fitness apps have sprung up including the ability to track both calories intake and expenditure. Users of such apps are part of a wider ``quantified self'' movement and many opt-in to publicly share their logged data. In this paper, we use public food diaries of more than 4,000 long-term active MyFitnessPal users to study the characteristics of a (un-)successful diet. Concretely, we train a machine learning model to predict repeatedly being over or under self-set daily calories goals and then look at which features contribute to the model's prediction. Our findings include both expected results, such as the token ``mcdonalds'' or the category ``dessert'' being indicative for being over the calories goal, but also less obvious ones such as the difference between pork and poultry concerning dieting success, or the use of the ``quick added calories'' functionality being indicative of over-shooting calorie-wise. This study also hints at the feasibility of using such data for more in-depth data mining, e.g., looking at the interaction between consumed foods such as mixing protein- and carbohydrate-rich foods. To the best of our knowledge, this is the first systematic study of public food diaries.
△ Less
Submitted 16 October, 2015;
originally announced October 2015.
