-
When Two Wrongs Don't Make a Right" -- Examining Confirmation Bias and the Role of Time Pressure During Human-AI Collaboration in Computational Pathology
Authors:
Emely Rosbach,
Jonas Ammeling,
Sebastian Krügel,
Angelika Kießig,
Alexis Fritz,
Jonathan Ganz,
Chloé Puget,
Taryn Donovan,
Andrea Klang,
Maximilian C. Köller,
Pompei Bolfa,
Marco Tecilla,
Daniela Denk,
Matti Kiupel,
Georgios Paraschou,
Mun Keong Kok,
Alexander F. H. Haake,
Ronald R. de Krijger,
Andreas F. -P. Sonnen,
Tanit Kasantikul,
Gerry M. Dorrestein,
Rebecca C. Smedley,
Nikolas Stathonikos,
Matthias Uhl,
Christof A. Bertram
, et al. (2 additional authors not shown)
Abstract:
Artificial intelligence (AI)-based decision support systems hold promise for enhancing diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration can introduce and amplify cognitive biases, such as confirmation bias caused by false confirmation when erroneous human opinions are reinforced by inaccurate AI output. This bias may worsen when time pressure, ubiquito…
▽ More
Artificial intelligence (AI)-based decision support systems hold promise for enhancing diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration can introduce and amplify cognitive biases, such as confirmation bias caused by false confirmation when erroneous human opinions are reinforced by inaccurate AI output. This bias may worsen when time pressure, ubiquitously present in routine pathology, strains practitioners' cognitive resources. We quantified confirmation bias triggered by AI-induced false confirmation and examined the role of time constraints in a web-based experiment, where trained pathology experts (n=28) estimated tumor cell percentages. Our results suggest that AI integration may fuel confirmation bias, evidenced by a statistically significant positive linear-mixed-effects model coefficient linking AI recommendations mirroring flawed human judgment and alignment with system advice. Conversely, time pressure appeared to weaken this relationship. These findings highlight potential risks of AI use in healthcare and aim to support the safe integration of clinical decision support systems.
△ Less
Submitted 1 November, 2024;
originally announced November 2024.
-
Automation Bias in AI-Assisted Medical Decision-Making under Time Pressure in Computational Pathology
Authors:
Emely Rosbach,
Jonathan Ganz,
Jonas Ammeling,
Andreas Riener,
Marc Aubreville
Abstract:
Artificial intelligence (AI)-based clinical decision support systems (CDSS) promise to enhance diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration might introduce automation bias, where users uncritically follow automated cues. This bias may worsen when time pressure strains practitioners' cognitive resources. We quantified automation bias by measuring th…
▽ More
Artificial intelligence (AI)-based clinical decision support systems (CDSS) promise to enhance diagnostic accuracy and efficiency in computational pathology. However, human-AI collaboration might introduce automation bias, where users uncritically follow automated cues. This bias may worsen when time pressure strains practitioners' cognitive resources. We quantified automation bias by measuring the adoption of negative system consultations and examined the role of time pressure in a web-based experiment, where trained pathology experts (n=28) estimated tumor cell percentages. Our results indicate that while AI integration led to a statistically significant increase in overall performance, it also resulted in a 7% automation bias rate, where initially correct evaluations were overturned by erroneous AI advice. Conversely, time pressure did not exacerbate automation bias occurrence, but appeared to increase its severity, evidenced by heightened reliance on the system's negative consultations and subsequent performance decline. These findings highlight potential risks of AI use in healthcare.
△ Less
Submitted 1 November, 2024;
originally announced November 2024.
-
Designing for Passengers' Information Needs on Fellow Travelers: A Comparison of Day and Night Rides in Shared Automated Vehicles
Authors:
Lukas A. Flohr,
Martina Schuß,
Dieter P. Wallach,
Antonio Krüger,
Andreas Riener
Abstract:
Shared automated mobility-on-demand promises efficient, sustainable, and flexible transportation. Nevertheless, security concerns, resilience, and their mutual influence - especially at night - will likely be the most critical barriers to public adoption since passengers have to share rides with strangers without a human driver on board. As related work points out that information about fellow tra…
▽ More
Shared automated mobility-on-demand promises efficient, sustainable, and flexible transportation. Nevertheless, security concerns, resilience, and their mutual influence - especially at night - will likely be the most critical barriers to public adoption since passengers have to share rides with strangers without a human driver on board. As related work points out that information about fellow travelers might mitigate passengers' concerns, we designed two user interface variants to investigate the role of this information in an exploratory within-subjects user study (N = 24). Participants experienced four automated day and night rides with varying personal information about co-passengers in a simulated environment. The results of the mixed-method study indicate that having information about other passengers (e.g., photo, gender, and name) positively affects user experience at night. In contrast, it is less necessary during the day. Considering participants' simultaneously raised privacy demands poses a substantial challenge for resilient system design.
△ Less
Submitted 4 August, 2023;
originally announced August 2023.
