-
The Community Index: A More Comprehensive Approach to Assessing Scholarly Impact
Authors:
Arav Kumar,
Cameron Sabet,
Alessandro Hammond,
Amelia Fiske,
Bhav Jain,
Deirdre Goode,
Dharaa Suresha,
Leo Anthony Celi,
Lisa Soleymani Lehmann,
Ned Mccague,
Rawan Abulibdeh,
Sameer Pradhan
Abstract:
The h index is a widely recognized metric for assessing the research impact of scholars, defined as the maximum value h such that the scholar has published h papers each cited at least h times. While it has proven useful measuring individual scholarly productivity and citation impact, the h index has limitations, such as an inability to account for interdisciplinary collaboration or demographic di…
▽ More
The h index is a widely recognized metric for assessing the research impact of scholars, defined as the maximum value h such that the scholar has published h papers each cited at least h times. While it has proven useful measuring individual scholarly productivity and citation impact, the h index has limitations, such as an inability to account for interdisciplinary collaboration or demographic differences in citation patterns. Moreover, it is sometimes mistakenly treated as a measure of research quality, even though it only reflects how often work has been cited. While metric based evaluations of research have grown in importance in some areas of academia, such as medicine, these evaluations fail to consider other important aspects of intellectual work, such as representational and epistemic diversity in research. In this article, we propose a new metric called the c index, or the community index, which combines multiple dimensions of scholarly impact. This is important because a plurality of perspectives and lived experiences within author teams can promote epistemological reflection and humility as part of the creation and validation of scientific knowledge. The c index is a means of accounting for the often global, and increasingly interdisciplinary nature of contemporary research, in particular, the data that is collected, curated and analyzed in the process of scientific inquiry. While the c index provides a means of quantifying diversity within research teams, diversity is integral to the advancement of scientific excellence and should be actively fostered through formal recognition and valuation. We herein describe the mathematical foundation of the c index and demonstrate its potential to provide a more comprehensive representation and more multidimensional assessment of scientific contributions of research impact as compared to the h index.
△ Less
Submitted 22 August, 2025;
originally announced August 2025.
-
Performance Gains of LLMs With Humans in a World of LLMs Versus Humans
Authors:
Lucas McCullum,
Pelagie Ami Agassi,
Leo Anthony Celi,
Daniel K. Ebner,
Chrystinne Oliveira Fernandes,
Rachel S. Hicklen,
Mkliwa Koumbia,
Lisa Soleymani Lehmann,
David Restrepo
Abstract:
Currently, a considerable research effort is devoted to comparing LLMs to a group of human experts, where the term "expert" is often ill-defined or variable, at best, in a state of constantly updating LLM releases. Without proper safeguards in place, LLMs will threaten to cause harm to the established structure of safe delivery of patient care which has been carefully developed throughout history…
▽ More
Currently, a considerable research effort is devoted to comparing LLMs to a group of human experts, where the term "expert" is often ill-defined or variable, at best, in a state of constantly updating LLM releases. Without proper safeguards in place, LLMs will threaten to cause harm to the established structure of safe delivery of patient care which has been carefully developed throughout history to keep the safety of the patient at the forefront. A key driver of LLM innovation is founded on community research efforts which, if continuing to operate under "humans versus LLMs" principles, will expedite this trend. Therefore, research efforts moving forward must focus on effectively characterizing the safe use of LLMs in clinical settings that persist across the rapid development of novel LLM models. In this communication, we demonstrate that rather than comparing LLMs to humans, there is a need to develop strategies enabling efficient work of humans with LLMs in an almost symbiotic manner.
△ Less
Submitted 13 May, 2025;
originally announced May 2025.
-
The use of large language models to enhance cancer clinical trial educational materials
Authors:
Mingye Gao,
Aman Varshney,
Shan Chen,
Vikram Goddla,
Jack Gallifant,
Patrick Doyle,
Claire Novack,
Maeve Dillon-Martin,
Teresia Perkins,
Xinrong Correia,
Erik Duhaime,
Howard Isenstein,
Elad Sharon,
Lisa Soleymani Lehmann,
David Kozono,
Brian Anthony,
Dmitriy Dligach,
Danielle S. Bitterman
Abstract:
Cancer clinical trials often face challenges in recruitment and engagement due to a lack of participant-facing informational and educational resources. This study investigated the potential of Large Language Models (LLMs), specifically GPT4, in generating patient-friendly educational content from clinical trial informed consent forms. Using data from ClinicalTrials.gov, we employed zero-shot learn…
▽ More
Cancer clinical trials often face challenges in recruitment and engagement due to a lack of participant-facing informational and educational resources. This study investigated the potential of Large Language Models (LLMs), specifically GPT4, in generating patient-friendly educational content from clinical trial informed consent forms. Using data from ClinicalTrials.gov, we employed zero-shot learning for creating trial summaries and one-shot learning for developing multiple-choice questions, evaluating their effectiveness through patient surveys and crowdsourced annotation. Results showed that GPT4-generated summaries were both readable and comprehensive, and may improve patients' understanding and interest in clinical trials. The multiple-choice questions demonstrated high accuracy and agreement with crowdsourced annotators. For both resource types, hallucinations were identified that require ongoing human oversight. The findings demonstrate the potential of LLMs "out-of-the-box" to support the generation of clinical trial education materials with minimal trial-specific engineering, but implementation with a human-in-the-loop is still needed to avoid misinformation risks.
△ Less
Submitted 3 December, 2024; v1 submitted 2 December, 2024;
originally announced December 2024.
-
Framework for understanding quantum computing use cases from a multidisciplinary perspective and future research directions
Authors:
Dandison Ukpabi,
Heikki Karjaluoto,
Astrid Bötticher,
Anastasija Nikiforova,
Dragoş Petrescu,
Paulina Schindler,
Visvaldis Valtenbergs,
Lennard Lehmann,
Abuzer Yakaryilmaz
Abstract:
Recently, there has been increasing awareness of the tremendous opportunities inherent in quantum computing (QC). Specifically, the speed and efficiency of QC will significantly impact the Internet of Things, cryptography, finance, and marketing. Accordingly, there has been increased QC research funding from national and regional governments and private firms. However, critical concerns regarding…
▽ More
Recently, there has been increasing awareness of the tremendous opportunities inherent in quantum computing (QC). Specifically, the speed and efficiency of QC will significantly impact the Internet of Things, cryptography, finance, and marketing. Accordingly, there has been increased QC research funding from national and regional governments and private firms. However, critical concerns regarding legal, political, and business-related policies germane to QC adoption exist. Since this is an emerging and highly technical domain, most of the existing studies focus heavily on the technical aspects of QC, but our study highlights its practical and social uses cases, which are needed for the increased interest of governments. Thus, this study offers a multidisciplinary review of QC, drawing on the expertise of scholars from a wide range of disciplines whose insights coalesce into a framework that simplifies the understanding of QC, identifies possible areas of market disruption and offer empirically based recommendations that are critical for forecasting, planning, and strategically positioning QCs for accelerated diffusion.
△ Less
Submitted 5 January, 2023; v1 submitted 19 December, 2022;
originally announced December 2022.
-
Algorithmic Differentiation of Linear Algebra Functions with Application in Optimum Experimental Design (Extended Version)
Authors:
S. F. Walter,
L. Lehmann
Abstract:
We derive algorithms for higher order derivative computation of the rectangular $QR$ and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of algorithmic differentiation (AD) using univariate Taylor propagation of matrices (UTPM). Linear algebra functions are regarded as elementary functions and not as algorithms. The presented algorithms ar…
▽ More
We derive algorithms for higher order derivative computation of the rectangular $QR$ and eigenvalue decomposition of symmetric matrices with distinct eigenvalues in the forward and reverse mode of algorithmic differentiation (AD) using univariate Taylor propagation of matrices (UTPM). Linear algebra functions are regarded as elementary functions and not as algorithms. The presented algorithms are implemented in the BSD licensed AD tool \texttt{ALGOPY}. Numerical tests show that the UTPM algorithms derived in this paper produce results close to machine precision accuracy. The theory developed in this paper is applied to compute the gradient of an objective function motivated from optimum experimental design: $\nabla_x Φ(C(J(F(x,y))))$, where $Φ= \{λ_1 : λ_1 C\}$, $C = (J^T J)^{-1}$, $J = \frac{\dd F}{\dd y}$ and $F = F(x,y)$.
△ Less
Submitted 19 February, 2010; v1 submitted 11 January, 2010;
originally announced January 2010.
