-
"Mango Mango, How to Let The Lettuce Dry Without A Spinner?'': Exploring User Perceptions of Using An LLM-Based Conversational Assistant Toward Cooking Partner
Authors:
Szeyi Chan,
Jiachen Li,
Bingsheng Yao,
Amama Mahmood,
Chien-Ming Huang,
Holly Jimison,
Elizabeth D Mynatt,
Dakuo Wang
Abstract:
The rapid advancement of the Large Language Model (LLM) has created numerous potentials for integration with conversational assistants (CAs) assisting people in their daily tasks, particularly due to their extensive flexibility. However, users' real-world experiences interacting with these assistants remain unexplored. In this research, we chose cooking, a complex daily task, as a scenario to inve…
▽ More
The rapid advancement of the Large Language Model (LLM) has created numerous potentials for integration with conversational assistants (CAs) assisting people in their daily tasks, particularly due to their extensive flexibility. However, users' real-world experiences interacting with these assistants remain unexplored. In this research, we chose cooking, a complex daily task, as a scenario to investigate people's successful and unsatisfactory experiences while receiving assistance from an LLM-based CA, Mango Mango. We discovered that participants value the system's ability to provide extensive information beyond the recipe, offer customized instructions based on context, and assist them in dynamically planning the task. However, they expect the system to be more adaptive to oral conversation and provide more suggestive responses to keep users actively involved. Recognizing that users began treating our LLM-CA as a personal assistant or even a partner rather than just a recipe-reading tool, we propose several design considerations for future development.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
Evaluating Alignment Approaches in Superimposed Time-Series and Temporal Event-Sequence Visualizations
Authors:
Yixuan Zhang,
Sara Di Bartolomeo,
Fangfang Sheng,
Holly Jimison,
Cody Dunne
Abstract:
Composite temporal event sequence visualizations have included sentinel event alignment techniques to cope with data volume and variety. Prior work has demonstrated the utility of using single-event alignment for understanding the precursor, co-occurring, and aftereffect events surrounding a sentinel event. However, the usefulness of single-event alignment has not been sufficiently evaluated in co…
▽ More
Composite temporal event sequence visualizations have included sentinel event alignment techniques to cope with data volume and variety. Prior work has demonstrated the utility of using single-event alignment for understanding the precursor, co-occurring, and aftereffect events surrounding a sentinel event. However, the usefulness of single-event alignment has not been sufficiently evaluated in composite visualizations. Furthermore, recently proposed dual-event alignment techniques have not been empirically evaluated. In this work, we designed tasks around temporal event sequence and timing analysis and conducted a controlled experiment on Amazon Mechanical Turk to examine four sentinel event alignment approaches: no sentinel event alignment (NoAlign), single-event alignment (SingleAlign), dual-event alignment with left justification (DualLeft), and dual-event alignment with stretch justification (DualStretch). Differences between approaches were most pronounced with more rows of data. For understanding intermediate events between two sentinel events, dual-event alignment was the clear winner for correctness---71% vs. 18% for NoAlign and SingleAlign. For understanding the duration between two sentinel events, NoAlign was the clear winner: correctness---88% vs. 36% for DualStretch---completion time---55 seconds vs. 101 seconds for DualLeft---and error---1.5% vs. 8.4% for DualStretch. For understanding precursor and aftereffect events, there was no significant difference among approaches. A free copy of this paper, the evaluation stimuli and data, and source code are available at https://osf.io/78fs5
△ Less
Submitted 20 August, 2019;
originally announced August 2019.
-
The shape of health: A comparison of five alternative ways of visualizing personal health and wellbeing
Authors:
Andres Ledesma,
Hannu Nieminen,
Päivi Valve,
Miikka Ermes,
Holly Jimison,
Misha Pavel
Abstract:
The combination of clinical and personal health and wellbeing data can tell us much about our behaviors, risks and overall status. The way this data is visualized may affect our understanding of our own health. To study this effect, we conducted a small experiment with 30 participants in which we presented a holistic overview of the health and wellbeing of two modeled individuals, one of them with…
▽ More
The combination of clinical and personal health and wellbeing data can tell us much about our behaviors, risks and overall status. The way this data is visualized may affect our understanding of our own health. To study this effect, we conducted a small experiment with 30 participants in which we presented a holistic overview of the health and wellbeing of two modeled individuals, one of them with metabolic syndrome. We used an insight-based methodology to assess the effectiveness of the visualizations. The results show that adequate visualization of holistic health data helps users without medical background to better understand the overall health situation and possible health risks related to lifestyles. Furthermore, we found that the application of insight-based methodology in the health and wellbeing domain remains unexplored and additional research and methodology development are needed.
△ Less
Submitted 6 April, 2016;
originally announced April 2016.
-
Assessing Executive Function Using a Computer Game: Computational Modeling of Cognitive Processes
Authors:
Stuart Hagler,
Holly B. Jimison,
Misha Pavel
Abstract:
Early and reliable detection of cognitive decline is one of the most important challenges of current healthcare. In this project we developed an approach whereby a frequently played computer game can be used to assess a variety of cognitive processes and estimate the results of the pen-and-paper Trail-Making Test (TMT) - known to measure executive function, as well as visual pattern recognition, s…
▽ More
Early and reliable detection of cognitive decline is one of the most important challenges of current healthcare. In this project we developed an approach whereby a frequently played computer game can be used to assess a variety of cognitive processes and estimate the results of the pen-and-paper Trail-Making Test (TMT) - known to measure executive function, as well as visual pattern recognition, speed of processing, working memory, and set-switching ability. We developed a computational model of the TMT based on a decomposition of the test into several independent processes, each characterized by a set of parameters that can be estimated from play of a computer game designed to resemble the TMT. An empirical evaluation of the model suggests that it is possible to use the game data to estimate the parameters of the underlying cognitive processes and using the values of the parameters to estimate the TMT performance. Cognitive measures and trends in these measures can be used to identify individuals for further assessment, to provide a mechanism for improving the early detection of neurological problems, and to provide feedback and monitoring for cognitive interventions in the home.
△ Less
Submitted 11 March, 2016;
originally announced March 2016.
-
Quantification of Human Movement for Assessment in Automated Exercise Coaching
Authors:
Stuart Hagler,
Holly B. Jimison,
Ruzena Bajczy,
Misha Pavel
Abstract:
Quantification of human movement is a challenge in many areas, ranging from physical therapy to robotics. We quantify of human movement for the purpose of providing automated exercise coaching in the home. We developed a model-based assessment and inference process that combines biomechanical constraints with movement assessment based on the Microsoft Kinect camera. To illustrate the approach, we…
▽ More
Quantification of human movement is a challenge in many areas, ranging from physical therapy to robotics. We quantify of human movement for the purpose of providing automated exercise coaching in the home. We developed a model-based assessment and inference process that combines biomechanical constraints with movement assessment based on the Microsoft Kinect camera. To illustrate the approach, we quantify the performance of a simple squatting exercise using two model-based metrics that are related to strength and endurance, and provide an estimate of the strength and energy-expenditure of each exercise session. We look at data for 5 subjects, and show that for some subjects the metrics indicate a trend consistent with improved exercise performance.
△ Less
Submitted 7 March, 2016;
originally announced March 2016.
-
A Perspective on Confidence and Its Use in Focusing Attention During Knowledge Acquisition
Authors:
David Heckerman,
Holly B. Jimison
Abstract:
We present a representation of partial confidence in belief and preference that is consistent with the tenets of decision-theory. The fundamental insight underlying the representation is that if a person is not completely confident in a probability or utility assessment, additional modeling of the assessment may improve decisions to which it is relevant. We show how a traditional decision-analytic…
▽ More
We present a representation of partial confidence in belief and preference that is consistent with the tenets of decision-theory. The fundamental insight underlying the representation is that if a person is not completely confident in a probability or utility assessment, additional modeling of the assessment may improve decisions to which it is relevant. We show how a traditional decision-analytic approach can be used to balance the benefits of additional modeling with associated costs. The approach can be used during knowledge acquisition to focus the attention of a knowledge engineer or expert on parts of a decision model that deserve additional refinement.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.
-
A Representation of Uncertainty to Aid Insight into Decision Models
Authors:
Holly B. Jimison
Abstract:
Many real world models can be characterized as weak, meaning that there is significant uncertainty in both the data input and inferences. This lack of determinism makes it especially difficult for users of computer decision aids to understand and have confidence in the models. This paper presents a representation for uncertainty and utilities that serves as a framework for graphical summary and…
▽ More
Many real world models can be characterized as weak, meaning that there is significant uncertainty in both the data input and inferences. This lack of determinism makes it especially difficult for users of computer decision aids to understand and have confidence in the models. This paper presents a representation for uncertainty and utilities that serves as a framework for graphical summary and computer-generated explanation of decision models. The application described that tests the methodology is a computer decision aid designed to enhance the clinician-patient consultation process for patients with angina (chest pain due to lack of blood flow to the heart muscle). The angina model is represented as a Bayesian decision network. Additionally, the probabilities and utilities are treated as random variables with probability distributions on their range of possible values. The initial distributions represent information on all patients with anginal symptoms, and the approach allows for rapid tailoring to more patientspecific distributions. This framework provides a metric for judging the importance of each variable in the model dynamically.
△ Less
Submitted 27 March, 2013;
originally announced April 2013.
