Showing 1–6 of 6 results for author: Law, P

Causal Perception in Question-Answering Systems

Authors: Po-Ming Law, Leo Yu-Ho Lo, Alex Endert, John Stasko, Huamin Qu

Abstract: Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impac… ▽ More Root cause analysis is a common data analysis task. While question-answering systems enable people to easily articulate a why question (e.g., why students in Massachusetts have high ACT Math scores on average) and obtain an answer, these systems often produce questionable causal claims. To investigate how such claims might mislead users, we conducted two crowdsourced experiments to study the impact of showing different information on user perceptions of a question-answering system. We found that in a system that occasionally provided unreasonable responses, showing a scatterplot increased the plausibility of unreasonable causal claims. Also, simply warning participants that correlation is not causation seemed to lead participants to accept reasonable causal claims more cautiously. We observed a strong tendency among participants to associate correlation with causation. Yet, the warning appeared to reduce the tendency. Grounded in the findings, we propose ways to reduce the illusion of causality when using question-answering systems. △ Less

Submitted 6 January, 2021; v1 submitted 28 December, 2020; originally announced December 2020.

Comments: ACM Conference on Human Factors in Computing Systems (CHI 2021)

Characterizing Automated Data Insights

Authors: Po-Ming Law, Alex Endert, John Stasko

Abstract: Many researchers have explored tools that aim to recommend data insights to users. These tools automatically communicate a rich diversity of data insights and offer such insights for many different purposes. However, there is a lack of structured understanding concerning what researchers of these tools mean by "insight" and what tasks in the analysis workflow these tools aim to support. We conduct… ▽ More Many researchers have explored tools that aim to recommend data insights to users. These tools automatically communicate a rich diversity of data insights and offer such insights for many different purposes. However, there is a lack of structured understanding concerning what researchers of these tools mean by "insight" and what tasks in the analysis workflow these tools aim to support. We conducted a systematic review of existing systems that seek to recommend data insights. Grounded in the review, we propose 12 types of automated insights and four purposes of automating insights. We further discuss the design opportunities emerged from our analysis. △ Less

Submitted 4 September, 2020; v1 submitted 29 August, 2020; originally announced August 2020.

Comments: Published as IEEE VIS 2020 short paper

What are Data Insights to Professional Visualization Users?

Authors: Po-Ming Law, Alex Endert, John Stasko

Abstract: While many visualization researchers have attempted to define data insights, little is known about how visualization users perceive them. We interviewed 23 professional users of end-user visualization platforms (e.g., Tableau and Power BI) about their experiences with data insights. We report on seven characteristics of data insights based on interviewees' descriptions. Grounded in these character… ▽ More While many visualization researchers have attempted to define data insights, little is known about how visualization users perceive them. We interviewed 23 professional users of end-user visualization platforms (e.g., Tableau and Power BI) about their experiences with data insights. We report on seven characteristics of data insights based on interviewees' descriptions. Grounded in these characteristics, we propose practical implications for creating tools that aim to automatically communicate data insights to users. △ Less

Submitted 4 October, 2020; v1 submitted 29 August, 2020; originally announced August 2020.

Comments: Published as IEEE VIS 2020 short paper

The Impact of Presentation Style on Human-In-The-Loop Detection of Algorithmic Bias

Authors: Po-Ming Law, Sana Malik, Fan Du, Moumita Sinha

Abstract: While decision makers have begun to employ machine learning, machine learning models may make predictions that bias against certain demographic groups. Semi-automated bias detection tools often present reports of automatically-detected biases using a recommendation list or visual cues. However, there is a lack of guidance concerning which presentation style to use in what scenarios. We conducted a… ▽ More While decision makers have begun to employ machine learning, machine learning models may make predictions that bias against certain demographic groups. Semi-automated bias detection tools often present reports of automatically-detected biases using a recommendation list or visual cues. However, there is a lack of guidance concerning which presentation style to use in what scenarios. We conducted a small lab study with 16 participants to investigate how presentation style might affect user behaviors in reviewing bias reports. Participants used both a prototype with a recommendation list and a prototype with visual cues for bias detection. We found that participants often wanted to investigate the performance measures that were not automatically detected as biases. Yet, when using the prototype with a recommendation list, they tended to give less consideration to such measures. Grounded in the findings, we propose information load and comprehensiveness as two axes for characterizing bias detection tasks and illustrate how the two axes could be adopted to reason about when to use a recommendation list or visual cues. △ Less

Submitted 9 May, 2020; v1 submitted 26 April, 2020; originally announced April 2020.

Comments: Published at Graphics Interface 2020 (GI 2020)

Designing Tools for Semi-Automated Detection of Machine Learning Biases: An Interview Study

Authors: Po-Ming Law, Sana Malik, Fan Du, Moumita Sinha

Abstract: Machine learning models often make predictions that bias against certain subgroups of input data. When undetected, machine learning biases can constitute significant financial and ethical implications. Semi-automated tools that involve humans in the loop could facilitate bias detection. Yet, little is known about the considerations involved in their design. In this paper, we report on an interview… ▽ More Machine learning models often make predictions that bias against certain subgroups of input data. When undetected, machine learning biases can constitute significant financial and ethical implications. Semi-automated tools that involve humans in the loop could facilitate bias detection. Yet, little is known about the considerations involved in their design. In this paper, we report on an interview study with 11 machine learning practitioners for investigating the needs surrounding semi-automated bias detection tools. Based on the findings, we highlight four considerations in designing to guide system designers who aim to create future tools for bias detection. △ Less

Submitted 17 March, 2020; v1 submitted 12 March, 2020; originally announced March 2020.

Comments: Proceedings of the CHI 2020 Workshop on Detection and Design for Cognitive Biases in People and Computing Systems

Segue: Overviewing Evolution Patterns of Egocentric Networks by Interactive Construction of Spatial Layouts

Authors: Po-Ming Law, Yanhong Wu, Rahul C. Basole

Abstract: Getting the overall picture of how a large number of ego-networks evolve is a common yet challenging task. Existing techniques often require analysts to inspect the evolution patterns of ego-networks one after another. In this study, we explore an approach that allows analysts to interactively create spatial layouts in which each dot is a dynamic ego-network. These spatial layouts provide overview… ▽ More Getting the overall picture of how a large number of ego-networks evolve is a common yet challenging task. Existing techniques often require analysts to inspect the evolution patterns of ego-networks one after another. In this study, we explore an approach that allows analysts to interactively create spatial layouts in which each dot is a dynamic ego-network. These spatial layouts provide overviews of the evolution patterns of ego-networks, thereby revealing different global patterns such as trends, clusters and outliers in evolution patterns. To let analysts interactively construct interpretable spatial layouts, we propose a data transformation pipeline, with which analysts can adjust the spatial layouts and convert dynamic egonetworks into event sequences to aid interpretations of the spatial positions. Based on this transformation pipeline, we developed Segue, a visual analysis system that supports thorough exploration of the evolution patterns of ego-networks. Through two usage scenarios, we demonstrate how analysts can gain insights into the overall evolution patterns of a large collection of ego-networks by interactively creating different spatial layouts. △ Less

Submitted 12 November, 2018; originally announced November 2018.

Comments: Published at IEEE Conference on Visual Analytics Science and Technology (IEEE VAST 2018)