-
Towards A Scalable Solution for Improving Multi-Group Fairness in Compositional Classification
Authors:
James Atwood,
Tina Tian,
Ben Packer,
Meghana Deodhar,
Jilin Chen,
Alex Beutel,
Flavien Prost,
Ahmad Beirami
Abstract:
Despite the rich literature on machine learning fairness, relatively little attention has been paid to remediating complex systems, where the final prediction is the combination of multiple classifiers and where multiple groups are present. In this paper, we first show that natural baseline approaches for improving equal opportunity fairness scale linearly with the product of the number of remedia…
▽ More
Despite the rich literature on machine learning fairness, relatively little attention has been paid to remediating complex systems, where the final prediction is the combination of multiple classifiers and where multiple groups are present. In this paper, we first show that natural baseline approaches for improving equal opportunity fairness scale linearly with the product of the number of remediated groups and the number of remediated prediction labels, rendering them impractical. We then introduce two simple techniques, called {\em task-overconditioning} and {\em group-interleaving}, to achieve a constant scaling in this multi-group multi-label setup. Our experimental results in academic and real-world environments demonstrate the effectiveness of our proposal at mitigation within this environment.
△ Less
Submitted 11 July, 2023;
originally announced July 2023.
-
A Human-ML Collaboration Framework for Improving Video Content Reviews
Authors:
Meghana Deodhar,
Xiao Ma,
Yixin Cai,
Alex Koes,
Alex Beutel,
Jilin Chen
Abstract:
We deal with the problem of localized in-video taxonomic human annotation in the video content moderation domain, where the goal is to identify video segments that violate granular policies, e.g., community guidelines on an online video platform. High quality human labeling is critical for enforcement in content moderation. This is challenging due to the problem of information overload - raters ne…
▽ More
We deal with the problem of localized in-video taxonomic human annotation in the video content moderation domain, where the goal is to identify video segments that violate granular policies, e.g., community guidelines on an online video platform. High quality human labeling is critical for enforcement in content moderation. This is challenging due to the problem of information overload - raters need to apply a large taxonomy of granular policy violations with ambiguous definitions, within a limited review duration to relatively long videos. Our key contribution is a novel human-machine learning (ML) collaboration framework aimed at maximizing the quality and efficiency of human decisions in this setting - human labels are used to train segment-level models, the predictions of which are displayed as "hints" to human raters, indicating probable regions of the video with specific policy violations. The human verified/corrected segment labels can help refine the model further, hence creating a human-ML positive feedback loop. Experiments show improved human video moderation decision quality, and efficiency through more granular annotations submitted within a similar review duration, which enable a 5-8% AUC improvement in the hint generation models.
△ Less
Submitted 17 October, 2022;
originally announced October 2022.
-
NeuraHealth: An Automated Screening Pipeline to Detect Undiagnosed Cognitive Impairment in Electronic Health Records with Deep Learning and Natural Language Processing
Authors:
Tanish Tyagi,
Colin G. Magdamo,
Ayush Noori,
Zhaozhi Li,
Xiao Liu,
Mayuresh Deodhar,
Zhuoqiao Hong,
Wendong Ge,
Elissa M. Ye,
Yi-han Sheu,
Haitham Alabsi,
Laura Brenner,
Gregory K. Robbins,
Sahar Zafar,
Nicole Benson,
Lidia Moura,
John Hsu,
Alberto Serrano-Pozo,
Dimitry Prokopenko,
Rudolph E. Tanzi,
Bradley T. Hyman,
Deborah Blacker,
Shibani S. Mukerji,
M. Brandon Westover,
Sudeshna Das
Abstract:
Dementia related cognitive impairment (CI) is a neurodegenerative disorder, affecting over 55 million people worldwide and growing rapidly at the rate of one new case every 3 seconds. 75% cases go undiagnosed globally with up to 90% in low-and-middle-income countries, leading to an estimated annual worldwide cost of USD 1.3 trillion, forecasted to reach 2.8 trillion by 2030. With no cure, a recurr…
▽ More
Dementia related cognitive impairment (CI) is a neurodegenerative disorder, affecting over 55 million people worldwide and growing rapidly at the rate of one new case every 3 seconds. 75% cases go undiagnosed globally with up to 90% in low-and-middle-income countries, leading to an estimated annual worldwide cost of USD 1.3 trillion, forecasted to reach 2.8 trillion by 2030. With no cure, a recurring failure of clinical trials, and a lack of early diagnosis, the mortality rate is 100%. Information in electronic health records (EHR) can provide vital clues for early detection of CI, but a manual review by experts is tedious and error prone. Several computational methods have been proposed, however, they lack an enhanced understanding of the linguistic context in complex language structures of EHR. Therefore, I propose a novel and more accurate framework, NeuraHealth, to identify patients who had no earlier diagnosis. In NeuraHealth, using patient EHR from Mass General Brigham BioBank, I fine-tuned a bi-directional attention-based deep learning natural language processing model to classify sequences. The sequence predictions were used to generate structured features as input for a patient level regularized logistic regression model. This two-step framework creates high dimensionality, outperforming all existing state-of-the-art computational methods as well as clinical methods. Further, I integrate the models into a real-world product, a web app, to create an automated EHR screening pipeline for scalable and high-speed discovery of undetected CI in EHR, making early diagnosis viable in medical facilities and in regions with scarce health services.
△ Less
Submitted 20 June, 2022; v1 submitted 12 January, 2022;
originally announced February 2022.
-
Using Deep Learning to Identify Patients with Cognitive Impairment in Electronic Health Records
Authors:
Tanish Tyagi,
Colin G. Magdamo,
Ayush Noori,
Zhaozhi Li,
Xiao Liu,
Mayuresh Deodhar,
Zhuoqiao Hong,
Wendong Ge,
Elissa M. Ye,
Yi-han Sheu,
Haitham Alabsi,
Laura Brenner,
Gregory K. Robbins,
Sahar Zafar,
Nicole Benson,
Lidia Moura,
John Hsu,
Alberto Serrano-Pozo,
Dimitry Prokopenko,
Rudolph E. Tanzi,
Bradley T. Hyman,
Deborah Blacker,
Shibani S. Mukerji,
M. Brandon Westover,
Sudeshna Das
Abstract:
Dementia is a neurodegenerative disorder that causes cognitive decline and affects more than 50 million people worldwide. Dementia is under-diagnosed by healthcare professionals - only one in four people who suffer from dementia are diagnosed. Even when a diagnosis is made, it may not be entered as a structured International Classification of Diseases (ICD) diagnosis code in a patient's charts. In…
▽ More
Dementia is a neurodegenerative disorder that causes cognitive decline and affects more than 50 million people worldwide. Dementia is under-diagnosed by healthcare professionals - only one in four people who suffer from dementia are diagnosed. Even when a diagnosis is made, it may not be entered as a structured International Classification of Diseases (ICD) diagnosis code in a patient's charts. Information relevant to cognitive impairment (CI) is often found within electronic health records (EHR), but manual review of clinician notes by experts is both time consuming and often prone to errors. Automated mining of these notes presents an opportunity to label patients with cognitive impairment in EHR data. We developed natural language processing (NLP) tools to identify patients with cognitive impairment and demonstrate that linguistic context enhances performance for the cognitive impairment classification task. We fine-tuned our attention based deep learning model, which can learn from complex language structures, and substantially improved accuracy (0.93) relative to a baseline NLP model (0.84). Further, we show that deep learning NLP can successfully identify dementia patients without dementia-related ICD codes or medications.
△ Less
Submitted 12 November, 2021;
originally announced November 2021.
