Showing 1–14 of 14 results for author: Saket, B

Towards Leveraging Large Language Models for Automated Medical Q&A Evaluation

Authors: Jack Krolik, Herprit Mahal, Feroz Ahmad, Gaurav Trivedi, Bahador Saket

Abstract: This paper explores the potential of using Large Language Models (LLMs) to automate the evaluation of responses in medical Question and Answer (Q\&A) systems, a crucial form of Natural Language Processing. Traditionally, human evaluation has been indispensable for assessing the quality of these responses. However, manual evaluation by medical professionals is time-consuming and costly. Our study e… ▽ More This paper explores the potential of using Large Language Models (LLMs) to automate the evaluation of responses in medical Question and Answer (Q\&A) systems, a crucial form of Natural Language Processing. Traditionally, human evaluation has been indispensable for assessing the quality of these responses. However, manual evaluation by medical professionals is time-consuming and costly. Our study examines whether LLMs can reliably replicate human evaluations by using questions derived from patient data, thereby saving valuable time for medical experts. While the findings suggest promising results, further research is needed to address more specific or complex questions that were beyond the scope of this initial investigation. △ Less

Submitted 3 September, 2024; originally announced September 2024.

Comments: 10 pages, 3 figures, 3 tables

ACM Class: I.2.7; J.3

Understanding the Effects of Visualizing Missing Values on Visual Data Exploration

Authors: Hayeong Song, Yu Fu, Bahador Saket, John Stasko

Abstract: When performing data analysis, people often confront data sets containing missing values. We conducted an empirical study to understand the effects of visualizing those missing values on participants' decision-making processes while performing a visual data exploration task. More specifically, our study participants purchased a hypothetical portfolio of stocks based on a dataset where some stocks… ▽ More When performing data analysis, people often confront data sets containing missing values. We conducted an empirical study to understand the effects of visualizing those missing values on participants' decision-making processes while performing a visual data exploration task. More specifically, our study participants purchased a hypothetical portfolio of stocks based on a dataset where some stocks had missing values for attributes such as PE ratio, beta, and EPS. The experiment used scatterplots to communicate the stock data. For one group of participants, stocks with missing values simply were not shown, while the second group saw such stocks depicted with estimated values as points with error bars. We measured participants' cognitive load involved in decision-making with data with missing values. Our results indicate that their decision-making workflow was different across two conditions. △ Less

Submitted 17 September, 2021; originally announced September 2021.

Leveraging WiFi Network Logs to Infer Student Collocation and its Relationship with Academic Performance

Authors: V. Das Swain, H. Kwon, S. Sargolzaei, B. Saket, M. Bin Morshed, K. Tran, D. Patel, Y. Tian, J. Philipose, Y. Cui, T. Plötz, M. De Choudhury, G. D. Abowd

Abstract: A comprehensive understanding of collocation can help understand performance outcomes. For university cohorts, this needs data that describes large groups over a long period. Harnessing user devices to infer this, while tempting, is challenged by privacy concerns, power consumption, and maintenance issues. Alternatively, embedding new sensors in the environment is limited by the expense of coverin… ▽ More A comprehensive understanding of collocation can help understand performance outcomes. For university cohorts, this needs data that describes large groups over a long period. Harnessing user devices to infer this, while tempting, is challenged by privacy concerns, power consumption, and maintenance issues. Alternatively, embedding new sensors in the environment is limited by the expense of covering the entire campus. We investigate the feasibility of leveraging WiFi association logs for this purpose. While these provide coarse approximations of location, these are easily obtainable and depict multiple users on campus over a semester. We explore how these coarse collocations are related to individual performance. Specifically, we inspect the association between individual performance and the collocation behaviors of project group members. We study 163 students (in 54 project groups) over 14 weeks. After describing how we determine collocation with the WiFi logs, we present a study to analyze how collocation within groups relates to a student's final score. We find collocation behaviors show a significant correlation (Pearson's r = 0.24) with performance -- better than both peer feedback or individual behaviors like attendance. Finally, we discuss how repurposing WiFi logs can facilitate applications for domains like mental wellbeing and physical health. △ Less

Submitted 5 May, 2021; v1 submitted 22 May, 2020; originally announced May 2020.

Comments: 25 pages, 10 figures, 5 tables

ACM Class: J.4

Geono-Cluster: Interactive Visual Cluster Analysis for Biologists

Authors: Bahador Saket, Subhajit Das, Bum Chul Kwon, Alex Endert

Abstract: Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or throug… ▽ More Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study. △ Less

Submitted 3 November, 2019; originally announced November 2019.

Investigating Direct Manipulation of Graphical Encodings as a Method for User Interaction

Authors: Bahador Saket, Samuel Huron, Charles Perin, Alex Endert

Abstract: We investigate direct manipulation of graphical encodings as a method for interacting with visualizations. There is an increasing interest in developing visualization tools that enable users to perform operations by directly manipulating graphical encodings rather than external widgets such as checkboxes and sliders. Designers of such tools must decide which direct manipulation operations should b… ▽ More We investigate direct manipulation of graphical encodings as a method for interacting with visualizations. There is an increasing interest in developing visualization tools that enable users to perform operations by directly manipulating graphical encodings rather than external widgets such as checkboxes and sliders. Designers of such tools must decide which direct manipulation operations should be supported, and identify how each operation can be invoked. However, we lack empirical guidelines for how people convey their intended operations using direct manipulation of graphical encodings. We address this issue by conducting a qualitative study that examines how participants perform 15 operations using direct manipulation of standard graphical encodings. From this study, we 1) identify a list of strategies people employ to perform each operation, 2) observe commonalities in strategies across operations, and 3) derive implications to help designers leverage direct manipulation of graphical encoding as a method for user interaction. △ Less

Submitted 1 August, 2019; originally announced August 2019.

Liger: Combining Interaction Paradigms for Visual Analysis

Authors: Bahador Saket, Lei Jiang, Charles Perin, Alex Endert

Abstract: Visualization tools usually leverage a single interaction paradigm (e.g., manual view specification, visualization by demonstration, etc.), which fosters the process of visualization construction. A large body of work has investigated the effectiveness of individual interaction paradigms, building an understanding of advantages and disadvantages of each in isolation. However, how can we leverage t… ▽ More Visualization tools usually leverage a single interaction paradigm (e.g., manual view specification, visualization by demonstration, etc.), which fosters the process of visualization construction. A large body of work has investigated the effectiveness of individual interaction paradigms, building an understanding of advantages and disadvantages of each in isolation. However, how can we leverage the benefits of multiple interaction paradigms by combining them into a single tool? We currently lack a holistic view of how interaction paradigms that use the same input modality (e.g., mouse) can be combined into a single tool and how people use such tools. To investigate opportunities and challenges in combining paradigms, we first created a multi-paradigm prototype (Liger) that combines two mouse-based interaction paradigms (manual view specification and visualization by demonstration) in a unified tool. We then conducted an exploratory study with Liger, providing initial evidence that people 1) use both paradigms interchangeably, 2) seamlessly switch between paradigms based on the operation at hand, and 3) choose to successfully complete a single operation using a combination of both paradigms. △ Less

Submitted 21 July, 2019; v1 submitted 18 July, 2019; originally announced July 2019.

A User-based Visual Analytics Workflow for Exploratory Model Analysis

Authors: Dylan Cashman, Shah Rukh Humayoun, Florian Heimerl, Kendall Park, Subhajit Das, John Thompson, Bahador Saket, Abigail Mosca, John Stasko, Alex Endert, Michael Gleicher, Remco Chang

Abstract: Many visual analytics systems allow users to interact with machine learning models towards the goals of data exploration and insight generation on a given dataset. However, in some situations, insights may be less important than the production of an accurate predictive model for future use. In that case, users are more interested in generating of diverse and robust predictive models, verifying the… ▽ More Many visual analytics systems allow users to interact with machine learning models towards the goals of data exploration and insight generation on a given dataset. However, in some situations, insights may be less important than the production of an accurate predictive model for future use. In that case, users are more interested in generating of diverse and robust predictive models, verifying their performance on holdout data, and selecting the most suitable model for their usage scenario. In this paper, we consider the concept of Exploratory Model Analysis (EMA), which is defined as the process of discovering and selecting relevant models that can be used to make predictions on a data source. We delineate the differences between EMA and the well-known term exploratory data analysis in terms of the desired outcome of the analytic process: insights into the data or a set of deployable models. The contributions of this work are a visual analytics system workflow for EMA, a user study, and two use cases validating the effectiveness of the workflow. We found that our system workflow enabled users to generate complex models, to assess them for various qualities, and to select the most relevant model for their task. △ Less

Submitted 29 July, 2019; v1 submitted 27 September, 2018; originally announced September 2018.

Journal ref: Computer Graphics Forum 38(3) 2019, The Eurographics Association and John Wiley & Sons Ltd

Beyond Heuristics: Learning Visualization Design

Authors: Bahador Saket, Dominik Moritz, Halden Lin, Victor Dibia, Cagatay Demiralp, Jeffrey Heer

Abstract: In this paper, we describe a research agenda for deriving design principles directly from data. We argue that it is time to go beyond manually curated and applied visualization design guidelines. We propose learning models of visualization design from data collected using graphical perception studies and build tools powered by the learned models. To achieve this vision, we need to 1) develop scala… ▽ More In this paper, we describe a research agenda for deriving design principles directly from data. We argue that it is time to go beyond manually curated and applied visualization design guidelines. We propose learning models of visualization design from data collected using graphical perception studies and build tools powered by the learned models. To achieve this vision, we need to 1) develop scalable methods for collecting training data, 2) collect different forms of training data, 3) advance interpretability of machine learning models, and 4) develop adaptive models that evolve as more data becomes available. △ Less

Submitted 15 August, 2018; v1 submitted 17 July, 2018; originally announced July 2018.

Evaluation of Visualization by Demonstration and Manual View Specification

Authors: Bahador Saket, Alex Endert

Abstract: We present an exploratory study comparing the visualization construction and data exploration processes of people using two visualization tools, each implementing a different interaction paradigm. One of the visualization tools implements the manual view specification paradigm (Polestar) and another implements the visualization by demonstration paradigm (VisExemplar). Findings of our study indicat… ▽ More We present an exploratory study comparing the visualization construction and data exploration processes of people using two visualization tools, each implementing a different interaction paradigm. One of the visualization tools implements the manual view specification paradigm (Polestar) and another implements the visualization by demonstration paradigm (VisExemplar). Findings of our study indicate that the interaction paradigms implemented in these tools influence: 1) approaches used for constructing visualizations, 2) how users form goals, 3) how many visualization alternatives are considered and created, and 4) the feeling of control during the visualization construction process. △ Less

Submitted 7 May, 2018; originally announced May 2018.

Task-Based Effectiveness of Basic Visualizations

Authors: Bahador Saket, Alex Endert, Cagatay Demiralp

Abstract: Visualizations of tabular data are widely used; understanding their effectiveness in different task and data contexts is fundamental to scaling their impact. However, little is known about how basic tabular data visualizations perform across varying data analysis tasks and data attribute types. In this paper, we report results from a crowdsourced experiment to evaluate the effectiveness of five vi… ▽ More Visualizations of tabular data are widely used; understanding their effectiveness in different task and data contexts is fundamental to scaling their impact. However, little is known about how basic tabular data visualizations perform across varying data analysis tasks and data attribute types. In this paper, we report results from a crowdsourced experiment to evaluate the effectiveness of five visualization types --- Table, Line Chart, Bar Chart, Scatterplot, and Pie Chart --- across ten common data analysis tasks and three data attribute types using two real-world datasets. We found the effectiveness of these visualization types significantly varies across task and data attribute types, suggesting that visualization design would benefit from considering context dependent effectiveness. Based on our findings, we derive recommendations on which visualizations to choose based on different tasks. We finally train a decision tree on the data we collected to drive a recommender, showcasing how to effectively engineer experimental user data into practical visualization systems. △ Less

Submitted 24 April, 2018; v1 submitted 25 September, 2017; originally announced September 2017.

VisAR: Bringing Interactivity to Static Data Visualizations through Augmented Reality

Authors: Taeheon Kim, Bahador Saket, Alex Endert, Blair MacIntyre

Abstract: Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to be… ▽ More Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to being static (e.g., visualizations on presentation slides or posters). We believe in many of these cases, static visualizations will benefit from allowing users to perform interactive tasks on them. Inspired by the introduction of numerous commercial personal augmented reality (AR) devices, we propose an AR solution that allows interactive data exploration of datasets on static visualizations. In particular, we present a prototype system named VisAR that uses the Microsoft Hololens to enable users to complete interactive tasks on static visualizations. △ Less

Submitted 4 August, 2017; originally announced August 2017.

Towards Understanding Enjoyment and Flow in Information Visualization

Authors: Bahador Saket, Carlos Scheidegger, Stephen Kobourov

Abstract: Traditionally, evaluation studies in information visualization have measured effectiveness by assessing performance time and accuracy. More recently, there has been a concerted effort to understand aspects beyond time and errors. In this paper we study enjoyment, which, while arguably not the primary goal of visualization, has been shown to impact performance and memorability. Different models of… ▽ More Traditionally, evaluation studies in information visualization have measured effectiveness by assessing performance time and accuracy. More recently, there has been a concerted effort to understand aspects beyond time and errors. In this paper we study enjoyment, which, while arguably not the primary goal of visualization, has been shown to impact performance and memorability. Different models of enjoyment have been proposed in psychology, education and gaming; yet there is no standard approach to evaluate and measure enjoyment in visualization. In this paper we relate the flow model of Csikszentmihalyi to Munzner's nested model of visualization evaluation and previous work in the area. We suggest that, even though previous papers tackled individual elements of flow, in order to understand what specifically makes a visualization enjoyable, it might be necessary to measure all specific elements. △ Less

Submitted 2 March, 2015; originally announced March 2015.

Node, Node-Link, and Node-Link-Group Diagrams: An Evaluation

Authors: Bahador Saket, Paolo Simonetto, Stephen Kobourov, Katy Borner

Abstract: Effectively showing the relationships between objects in a dataset is one of the main tasks in information visualization. Typically there is a well-defined notion of distance between pairs of objects, and traditional approaches such as principal component analysis or multi-dimensional scaling are used to place the objects as points in 2D space, so that similar objects are close to each other. In a… ▽ More Effectively showing the relationships between objects in a dataset is one of the main tasks in information visualization. Typically there is a well-defined notion of distance between pairs of objects, and traditional approaches such as principal component analysis or multi-dimensional scaling are used to place the objects as points in 2D space, so that similar objects are close to each other. In another typical setting, the dataset is visualized as a network graph, where related nodes are connected by links. More recently, datasets are also visualized as maps, where in addition to nodes and links, there is an explicit representation of groups and clusters. We consider these three Techniques, characterized by a progressive increase of the amount of encoded information: node diagrams, node-link diagrams and node-link-group diagrams. We assess these three types of diagrams with a controlled experiment that covers nine different tasks falling broadly in three categories: node-based tasks, network-based tasks and group-based tasks. Our findings indicate that adding links, or links and group representations, does not negatively impact performance (time and accuracy) of node-based tasks. Similarly, adding group representations does not negatively impact the performance of network-based tasks. Node-link-group diagrams outperform the others on group-based tasks. These conclusions contradict results in other studies, in similar but subtly different settings. Taken together, however, such results can have significant implications for the design of standard and domain specific visualizations tools. △ Less

Submitted 7 April, 2014; originally announced April 2014.

Group-Level Graph Visualization Taxonomy

Authors: Bahador Saket, Paolo Simonetto, Stephen Kobourov

Abstract: Task taxonomies for graph and network visualizations focus on tasks commonly encountered when analyzing graph connectivity and topology. However, in many application fields such as the social sciences (social networks), biology (protein interaction models), software engineering (program call graphs), connectivity and topology information is intertwined with group, clustering, and hierarchical info… ▽ More Task taxonomies for graph and network visualizations focus on tasks commonly encountered when analyzing graph connectivity and topology. However, in many application fields such as the social sciences (social networks), biology (protein interaction models), software engineering (program call graphs), connectivity and topology information is intertwined with group, clustering, and hierarchical information. Several recent visualization techniques, such as BubbleSets, LineSets and GMap, make explicit use of grouping and clustering, but evaluating such visualization has been difficult due to the lack of standardized group-level tasks. With this in mind, our goal is to define a new set of tasks that assess group-level comprehension. We propose several types of group-level tasks and provide several examples of each type. Finally, we characterize some of the proposed tasks using the multi-level typology of abstract visualization tasks. We believe that adding group-level tasks to the task taxonomy for graph visualization would make the taxonomy more useful for the recent graph visualization techniques. It would help evaluators define and categorize new tasks, and it would help generalize individual results collected in controlled experiments. △ Less

Submitted 21 March, 2014; originally announced March 2014.