-
LLM-based Text Simplification and its Effect on User Comprehension and Cognitive Load
Authors:
Theo Guidroz,
Diego Ardila,
Jimmy Li,
Adam Mansour,
Paul Jhun,
Nina Gonzalez,
Xiang Ji,
Mike Sanchez,
Sujay Kakarmath,
Mathias MJ Bellaiche,
Miguel Ángel Garrido,
Faruk Ahmed,
Divyansh Choudhary,
Jay Hartford,
Chenwei Xu,
Henry Javier Serrano Echeverria,
Yifan Wang,
Jeff Shaffer,
Eric,
Cao,
Yossi Matias,
Avinatan Hassidim,
Dale R Webster,
Yun Liu,
Sho Fujiwara
, et al. (2 additional authors not shown)
Abstract:
Information on the web, such as scientific publications and Wikipedia, often surpasses users' reading level. To help address this, we used a self-refinement approach to develop a LLM capability for minimally lossy text simplification. To validate our approach, we conducted a randomized study involving 4563 participants and 31 texts spanning 6 broad subject areas: PubMed (biomedical scientific arti…
▽ More
Information on the web, such as scientific publications and Wikipedia, often surpasses users' reading level. To help address this, we used a self-refinement approach to develop a LLM capability for minimally lossy text simplification. To validate our approach, we conducted a randomized study involving 4563 participants and 31 texts spanning 6 broad subject areas: PubMed (biomedical scientific articles), biology, law, finance, literature/philosophy, and aerospace/computer science. Participants were randomized to viewing original or simplified texts in a subject area, and answered multiple-choice questions (MCQs) that tested their comprehension of the text. The participants were also asked to provide qualitative feedback such as task difficulty. Our results indicate that participants who read the simplified text answered more MCQs correctly than their counterparts who read the original text (3.9% absolute increase, p<0.05). This gain was most striking with PubMed (14.6%), while more moderate gains were observed for finance (5.5%), aerospace/computer science (3.8%) domains, and legal (3.5%). Notably, the results were robust to whether participants could refer back to the text while answering MCQs. The absolute accuracy decreased by up to ~9% for both original and simplified setups where participants could not refer back to the text, but the ~4% overall improvement persisted. Finally, participants' self-reported perceived ease based on a simplified NASA Task Load Index was greater for those who read the simplified text (absolute change on a 5-point scale 0.33, p<0.05). This randomized study, involving an order of magnitude more participants than prior works, demonstrates the potential of LLMs to make complex information easier to understand. Our work aims to enable a broader audience to better learn and make use of expert knowledge available on the web, improving information accessibility.
△ Less
Submitted 4 May, 2025;
originally announced May 2025.
-
HeAR -- Health Acoustic Representations
Authors:
Sebastien Baur,
Zaid Nabulsi,
Wei-Hung Weng,
Jake Garrison,
Louis Blankemeier,
Sam Fishman,
Christina Chen,
Sujay Kakarmath,
Minyoi Maimbolwa,
Nsala Sanjase,
Brian Shuma,
Yossi Matias,
Greg S. Corrado,
Shwetak Patel,
Shravya Shetty,
Shruthi Prabhakara,
Monde Muyoyeta,
Diego Ardila
Abstract:
Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other t…
▽ More
Health acoustic sounds such as coughs and breaths are known to contain useful health signals with significant potential for monitoring health and disease, yet are underexplored in the medical machine learning community. The existing deep learning systems for health acoustics are often narrowly trained and evaluated on a single task, which is limited by data and may hinder generalization to other tasks. To mitigate these gaps, we develop HeAR, a scalable self-supervised learning-based deep learning system using masked autoencoders trained on a large dataset of 313 million two-second long audio clips. Through linear probes, we establish HeAR as a state-of-the-art health audio embedding model on a benchmark of 33 health acoustic tasks across 6 datasets. By introducing this work, we hope to enable and accelerate further health acoustics research.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Predicting Cardiovascular Disease Risk using Photoplethysmography and Deep Learning
Authors:
Wei-Hung Weng,
Sebastien Baur,
Mayank Daswani,
Christina Chen,
Lauren Harrell,
Sujay Kakarmath,
Mariam Jabara,
Babak Behsaz,
Cory Y. McLean,
Yossi Matias,
Greg S. Corrado,
Shravya Shetty,
Shruthi Prabhakara,
Yun Liu,
Goodarz Danaei,
Diego Ardila
Abstract:
Cardiovascular diseases (CVDs) are responsible for a large proportion of premature deaths in low- and middle-income countries. Early CVD detection and intervention is critical in these populations, yet many existing CVD risk scores require a physical examination or lab measurements, which can be challenging in such health systems due to limited accessibility. Here we investigated the potential to…
▽ More
Cardiovascular diseases (CVDs) are responsible for a large proportion of premature deaths in low- and middle-income countries. Early CVD detection and intervention is critical in these populations, yet many existing CVD risk scores require a physical examination or lab measurements, which can be challenging in such health systems due to limited accessibility. Here we investigated the potential to use photoplethysmography (PPG), a sensing technology available on most smartphones that can potentially enable large-scale screening at low cost, for CVD risk prediction. We developed a deep learning PPG-based CVD risk score (DLS) to predict the probability of having major adverse cardiovascular events (MACE: non-fatal myocardial infarction, stroke, and cardiovascular death) within ten years, given only age, sex, smoking status and PPG as predictors. We compared the DLS with the office-based refit-WHO score, which adopts the shared predictors from WHO and Globorisk scores (age, sex, smoking status, height, weight and systolic blood pressure) but refitted on the UK Biobank (UKB) cohort. In UKB cohort, DLS's C-statistic (71.1%, 95% CI 69.9-72.4) was non-inferior to office-based refit-WHO score (70.9%, 95% CI 69.7-72.2; non-inferiority margin of 2.5%, p<0.01). The calibration of the DLS was satisfactory, with a 1.8% mean absolute calibration error. Adding DLS features to the office-based score increased the C-statistic by 1.0% (95% CI 0.6-1.4). DLS predicts ten-year MACE risk comparable with the office-based refit-WHO score. It provides a proof-of-concept and suggests the potential of a PPG-based approach strategies for community-based primary prevention in resource-limited regions.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Deploying clinical machine learning? Consider the following...
Authors:
Charles Lu,
Ken Chang,
Praveer Singh,
Stuart Pomerantz,
Sean Doyle,
Sujay Kakarmath,
Christopher Bridge,
Jayashree Kalpathy-Cramer
Abstract:
Despite the intense attention and considerable investment into clinical machine learning research, relatively few applications have been deployed at a large-scale in a real-world clinical environment. While research is important in advancing the state-of-the-art, translation is equally important in bringing these techniques and technologies into a position to ultimately impact healthcare. We belie…
▽ More
Despite the intense attention and considerable investment into clinical machine learning research, relatively few applications have been deployed at a large-scale in a real-world clinical environment. While research is important in advancing the state-of-the-art, translation is equally important in bringing these techniques and technologies into a position to ultimately impact healthcare. We believe a lack of appreciation for several considerations are a major cause for this discrepancy between expectation and reality. To better characterize a holistic perspective among researchers and practitioners, we survey several practitioners with commercial experience in developing CML for clinical deployment. Using these insights, we identify several main categories of challenges in order to better design and develop clinical machine learning applications.
△ Less
Submitted 8 June, 2023; v1 submitted 14 September, 2021;
originally announced September 2021.
