Do LLMs Act as Repositories of Causal Knowledge?
Authors:
Nick Huntington-Klein,
Eleanor J. Murray
Abstract:
Large language models (LLMs) offer the potential to automate a large number of tasks that previously have not been possible to automate, including some in science. There is considerable interest in whether LLMs can automate the process of causal inference by providing the information about causal links necessary to build a structural model. We use the case of confounding in the Coronary Drug Proje…
▽ More
Large language models (LLMs) offer the potential to automate a large number of tasks that previously have not been possible to automate, including some in science. There is considerable interest in whether LLMs can automate the process of causal inference by providing the information about causal links necessary to build a structural model. We use the case of confounding in the Coronary Drug Project (CDP), for which there are several studies listing expert-selected confounders that can serve as a ground truth. LLMs exhibit mediocre performance in identifying confounders in this setting, even though text about the ground truth is in their training data. Variables that experts identify as confounders are only slightly more likely to be labeled as confounders by LLMs compared to variables that experts consider non-confounders. Further, LLM judgment on confounder status is highly inconsistent across models, prompts, and irrelevant concerns like multiple-choice option ordering. LLMs do not yet have the ability to automate the reporting of causal links.
△ Less
Submitted 13 December, 2024;
originally announced December 2024.
The economic value of empowering older patients transitioning from hospital to home: Evidence from the 'Your Care Needs You' intervention
Authors:
Alfredo Palacios,
Simon Walker,
Beth Woods,
Catherine Hewitt,
Alison Cracknell,
Jenni Murray,
Rebecca Lawton,
Gerry Richardson
Abstract:
Hospital-to-home transitions are a critical component of effective healthcare delivery, especially for patients aged 75 and older. This study evaluates the cost-effectiveness of the 'Your Care Needs You' (YCNY) intervention, a patient-centred approach designed to empower older adults during discharge, compared to standard care. The analysis adopts the perspective of the National Health Service (NH…
▽ More
Hospital-to-home transitions are a critical component of effective healthcare delivery, especially for patients aged 75 and older. This study evaluates the cost-effectiveness of the 'Your Care Needs You' (YCNY) intervention, a patient-centred approach designed to empower older adults during discharge, compared to standard care. The analysis adopts the perspective of the National Health Service (NHS) and Personal Social Services. Data were drawn from a cluster randomised controlled trial (cRCT) conducted within the UK NHS over a 90-day post-discharge follow-up period. Adjusted differences in costs and Quality-Adjusted Life Years (QALYs) were estimated using Multilevel Mixed-Effects Generalised Linear Models (MME-GLM) to account for the hierarchical structure of the trial design. Alternatively, Seemingly Unrelated Regression (SUR) models were employed to address potential correlations between costs and QALYs. Scenario analyses and probabilistic sensitivity analyses were conducted to assess the robustness of the results. The YCNY intervention reduced costs by £269 and achieved a QALY gain of 0.0057, resulting in a net health benefit (NHB) of 0.0246 at a £15,000/QALY threshold. It demonstrated an 89% probability of cost-effectiveness compared to standard care within the trial's time horizon. Findings remained robust across alternative scenarios and sensitivity analyses. These results suggest that YCNY is a promising and potentially cost-effective strategy for improving hospital-to-home transitions for older adults. By enhancing outcomes and reducing costs, the study supports integrating patient-centred interventions like YCNY into routine NHS practice, with the potential to improve both efficiency and quality of healthcare delivery.
△ Less
Submitted 18 December, 2024; v1 submitted 6 November, 2024;
originally announced November 2024.
