Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making
Authors:
Kori Inkpen,
Shreya Chappidi,
Keri Mallari,
Besmira Nushi,
Divya Ramesh,
Pietro Michelucci,
Vani Mandava,
Libuše Hannah Vepřek,
Gabrielle Quinn
Abstract:
Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. This work examines users' interaction with three simulated algorithmic models, all with similar accuracy…
▽ More
Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user's domain expertise, mental models of an AI system, trust in recommendations, and more. This work examines users' interaction with three simulated algorithmic models, all with similar accuracy but different tuning on their true positive and true negative rates. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled.
Our results show that while recommendations from an AI-Assistant can aid user decision making, factors such as users' baseline performance relative to the AI and complementary tuning of AI error types significantly impact overall team performance. Novice users improved, but not to the accuracy level of the AI. Highly proficient users were generally able to discern when they should follow the AI recommendation and typically maintained or improved their performance. Mid-performers, who had a similar level of accuracy to the AI, were most variable in terms of whether the AI recommendations helped or hurt their performance. In addition, we found that users' perception of the AI's performance relative on their own also had a significant impact on whether their accuracy improved when given AI recommendations. This work provides insights on the complexity of factors related to Human-AI collaboration and provides recommendations on how to develop human-centered AI algorithms to complement users in decision-making tasks.
△ Less
Submitted 16 August, 2022;
originally announced August 2022.
Polynomial phase estimation by phase unwrapping
Authors:
Robby G. McKilliam,
Barry G. Quinn,
I. Vaughan L. Clarkson,
Bill Moran,
Badri N. Vellambi
Abstract:
Estimating the coefficients of a noisy polynomial phase signal is important in fields including radar, biology and radio communications. One approach attempts to perform polynomial regression on the phase of the signal. This is complicated by the fact that the phase is wrapped modulo 2πand must be unwrapped before regression can be performed. In this paper we consider an estimator that performs ph…
▽ More
Estimating the coefficients of a noisy polynomial phase signal is important in fields including radar, biology and radio communications. One approach attempts to perform polynomial regression on the phase of the signal. This is complicated by the fact that the phase is wrapped modulo 2πand must be unwrapped before regression can be performed. In this paper we consider an estimator that performs phase unwrapping in a least squares manner. We describe the asymptotic properties of this estimator, showing that it is strongly consistent and asymptotically normally distributed.
△ Less
Submitted 2 November, 2012;
originally announced November 2012.
An Algorithm to Compute the Nearest Point in the Lattice $A_{n}^*$
Authors:
Robby G. McKilliam,
I. Vaughan L. Clarkson,
Barry G. Quinn
Abstract:
The lattice $A_n^*$ is an important lattice because of its covering properties in low dimensions. Clarkson \cite{Clarkson1999:Anstar} described an algorithm to compute the nearest lattice point in $A_n^*$ that requires $O(n\log{n})$ arithmetic operations. In this paper, we describe a new algorithm. While the complexity is still $O(n\log{n})$, it is significantly simpler to describe and verify. I…
▽ More
The lattice $A_n^*$ is an important lattice because of its covering properties in low dimensions. Clarkson \cite{Clarkson1999:Anstar} described an algorithm to compute the nearest lattice point in $A_n^*$ that requires $O(n\log{n})$ arithmetic operations. In this paper, we describe a new algorithm. While the complexity is still $O(n\log{n})$, it is significantly simpler to describe and verify. In practice, we find that the new algorithm also runs faster.
△ Less
Submitted 29 September, 2008; v1 submitted 9 January, 2008;
originally announced January 2008.
