-
Large language models management of medications: three performance analyses
Authors:
Kelli Henry,
Steven Xu,
Kaitlin Blotske,
Moriah Cargile,
Erin F. Barreto,
Brian Murray,
Susan Smith,
Seth R. Bauer,
Yanjun Gao,
Tianming Liu,
Andrea Sikora
Abstract:
Background: Large language models (LLMs) can be useful in diagnosing medical conditions, but few studies have evaluated their consistency in recommending appropriate medication regimens. The purpose of this evaluation was to test GPT-4o on three medication benchmarking tests including mapping a drug name to its correct formulation, identifying drug-drug interactions using both its internal knowled…
▽ More
Background: Large language models (LLMs) can be useful in diagnosing medical conditions, but few studies have evaluated their consistency in recommending appropriate medication regimens. The purpose of this evaluation was to test GPT-4o on three medication benchmarking tests including mapping a drug name to its correct formulation, identifying drug-drug interactions using both its internal knowledge and using a web search, and preparing a medication order sentence after being given the medication name. Methods: Using GTP-4o three experiments were completed. Accuracy was quantified by computing cosine similarity on TF-IDF vectors, normalized Levenshtein similarity, and ROUGE-1/ROUGE-L F1 between each response and its reference string or by manual evaluation by clinicians. Results: GPT-4o performed poorly on drug-formulation matching, with frequent omissions of available drug formulations (mean 1.23 per medication) and hallucinations of formulations that do not exist (mean 1.14 per medication). Only 49% of tested medications were correctly matched to all available formulations. Accuracy was decreased for medications with more formulations (p<0.0001). GPT-4o was also inconsistent at identifying drug-drug-interactions, although it had better performance with the search-augmented assessment compared to its internal knowledge (54.7% vs. 69.2%, p=0.013). However, allowing a web-search worsened performance when there was no drug-drug interaction (median % correct 100% vs. 40%, p<0.001). Finally, GPT-4o performed moderately with preparing a medication order sentence, with only 65.8% of medication order sentences containing no medication or abbreviation errors. Conclusions: Model performance was overall poor for all tests. This highlights the need for domain-specific training through clinician-annotated datasets and a comprehensive evaluation framework for benchmarking performance.
△ Less
Submitted 26 September, 2025;
originally announced September 2025.
-
Calibration of stochastic, agent-based neuron growth models with Approximate Bayesian Computation
Authors:
Tobias Duswald,
Lukas Breitwieser,
Thomas Thorne,
Barbara Wohlmuth,
Roman Bauer
Abstract:
Understanding how genetically encoded rules drive and guide complex neuronal growth processes is essential to comprehending the brain's architecture, and agent-based models (ABMs) offer a powerful simulation approach to further develop this understanding. However, accurately calibrating these models remains a challenge. Here, we present a novel application of Approximate Bayesian Computation (ABC)…
▽ More
Understanding how genetically encoded rules drive and guide complex neuronal growth processes is essential to comprehending the brain's architecture, and agent-based models (ABMs) offer a powerful simulation approach to further develop this understanding. However, accurately calibrating these models remains a challenge. Here, we present a novel application of Approximate Bayesian Computation (ABC) to address this issue. ABMs are based on parametrized stochastic rules that describe the time evolution of small components -- the so-called agents -- discretizing the system, leading to stochastic simulations that require appropriate treatment. Mathematically, the calibration defines a stochastic inverse problem. We propose to address it in a Bayesian setting using ABC. We facilitate the repeated comparison between data and simulations by quantifying the morphological information of single neurons with so-called morphometrics and resort to statistical distances to measure discrepancies between populations thereof. We conduct experiments on synthetic as well as experimental data. We find that ABC utilizing Sequential Monte Carlo sampling and the Wasserstein distance finds accurate posterior parameter distributions for representative ABMs. We further demonstrate that these ABMs capture specific features of pyramidal cells of the hippocampus (CA1). Overall, this work establishes a robust framework for calibrating agent-based neuronal growth models and opens the door for future investigations using Bayesian techniques for model building, verification, and adequacy assessment.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Recipes for calibration and validation of agent-based models in cancer biomedicine
Authors:
Nicolò Cogno,
Cristian Axenie,
Roman Bauer,
Vasileios Vavourakis
Abstract:
Computational models and simulations are not just appealing because of their intrinsic characteristics across spatiotemporal scales, scalability, and predictive power, but also because the set of problems in cancer biomedicine that can be addressed computationally exceeds the set of those amenable to analytical solutions. Agent-based models and simulations are especially interesting candidates amo…
▽ More
Computational models and simulations are not just appealing because of their intrinsic characteristics across spatiotemporal scales, scalability, and predictive power, but also because the set of problems in cancer biomedicine that can be addressed computationally exceeds the set of those amenable to analytical solutions. Agent-based models and simulations are especially interesting candidates among computational modelling strategies in cancer research due to their capabilities to replicate realistic local and global interaction dynamics at a convenient and relevant scale. Yet, the absence of methods to validate the consistency of the results across scales can hinder adoption by turning fine-tuned models into black boxes. This review compiles relevant literature to explore strategies to leverage high-fidelity simulations of multi-scale, or multi-level, cancer models with a focus on validation approached as simulation calibration. We argue that simulation calibration goes beyond parameter optimization by embedding informative priors to generate plausible parameter configurations across multiple dimensions.
△ Less
Submitted 30 October, 2023;
originally announced October 2023.
-
Visual Ensemble Analysis of Fluid Flow in Porous Media across Simulation Codes and Experiment
Authors:
Ruben Bauer,
Quynh Quang Ngo,
Guido Reina,
Steffen Frey,
Bernd Flemisch,
Helwig Hauser,
Thomas Ertl,
Michael Sedlmair
Abstract:
We study the question of how visual analysis can support the comparison of spatio-temporal ensemble data of liquid and gas flow in porous media. To this end, we focus on a case study, in which nine different research groups concurrently simulated the process of injecting CO2 into the subsurface. We explore different data aggregation and interactive visualization approaches to compare and analyze t…
▽ More
We study the question of how visual analysis can support the comparison of spatio-temporal ensemble data of liquid and gas flow in porous media. To this end, we focus on a case study, in which nine different research groups concurrently simulated the process of injecting CO2 into the subsurface. We explore different data aggregation and interactive visualization approaches to compare and analyze these nine simulations. In terms of data aggregation, one key component is the choice of similarity metrics that define the relation between the different simulations. We test different metrics and find that a fine-tuned machine-learning based metric provides the best visualization results. Based on that, we propose different visualization methods. For overviewing the data, we use dimensionality reduction methods that allow us to plot and compare the different simulations in a scatterplot. To show details about the spatio-temporal data of each individual simulation, we employ a space-time cube volume rendering. We use the resulting interactive, multi-view visual analysis tool to explore the nine simulations and also to compare them to data from experimental setups. Our main findings include new insights into ranking of simulation results with respect to experimental data, and the development of gravity fingers in simulations.
△ Less
Submitted 27 November, 2023; v1 submitted 26 January, 2023;
originally announced February 2023.
-
An Optimization-based Approach for Flow Table Capacity Bottleneck Mitigation in Software-Defined Networks
Authors:
Robert Bauer,
Martina Zitterbart
Abstract:
Flow delegation is a flexible technique to mitigate flow table capacity bottlenecks in Software-defined Networks (SDN). Such bottlenecks occur when SDN switches provide insufficient flow table capacity which leads to performance degradation and network failures. Flow delegation addresses this problem by automatically relocating flow rules from a bottlenecked switch to neighboring switches with spa…
▽ More
Flow delegation is a flexible technique to mitigate flow table capacity bottlenecks in Software-defined Networks (SDN). Such bottlenecks occur when SDN switches provide insufficient flow table capacity which leads to performance degradation and network failures. Flow delegation addresses this problem by automatically relocating flow rules from a bottlenecked switch to neighboring switches with spare capacity. This paper introduces a new algorithm to efficiently perform flow delegation based on a novel delegation template abstraction and multi-period multi-objective optimization. Different from existing work, our approach can include estimated knowledge about future network situations and deal with different optimization criteria such as link and control overhead. We discuss the problem decomposition for the new algorithm and introduce an efficient two-step heuristic. Results show, that our approach performs significantly better than the simple greedy algorithm used in earlier work and is capable of handling flow delegation for networks with hundreds of switches.
△ Less
Submitted 17 September, 2021;
originally announced September 2021.
-
GraspME -- Grasp Manifold Estimator
Authors:
Janik Hager,
Ruben Bauer,
Marc Toussaint,
Jim Mainprice
Abstract:
In this paper, we introduce a Grasp Manifold Estimator (GraspME) to detect grasp affordances for objects directly in 2D camera images. To perform manipulation tasks autonomously it is crucial for robots to have such graspability models of the surrounding objects. Grasp manifolds have the advantage of providing continuously infinitely many grasps, which is not the case when using other grasp repres…
▽ More
In this paper, we introduce a Grasp Manifold Estimator (GraspME) to detect grasp affordances for objects directly in 2D camera images. To perform manipulation tasks autonomously it is crucial for robots to have such graspability models of the surrounding objects. Grasp manifolds have the advantage of providing continuously infinitely many grasps, which is not the case when using other grasp representations such as predefined grasp points. For instance, this property can be leveraged in motion optimization to define goal sets as implicit surface constraints in the robot configuration space. In this work, we restrict ourselves to the case of estimating possible end-effector positions directly from 2D camera images. To this extend, we define grasp manifolds via a set of key points and locate them in images using a Mask R-CNN backbone. Using learned features allows generalizing to different view angles, with potentially noisy images, and objects that were not part of the training set. We rely on simulation data only and perform experiments on simple and complex objects, including unseen ones. Our framework achieves an inference speed of 11.5 fps on a GPU, an average precision for keypoint estimation of 94.5% and a mean pixel distance of only 1.29. This shows that we can estimate the objects very well via bounding boxes and segmentation masks as well as approximate the correct grasp manifold's keypoint coordinates.
△ Less
Submitted 5 July, 2021;
originally announced July 2021.
-
BioDynaMo: a general platform for scalable agent-based simulation
Authors:
Lukas Breitwieser,
Ahmad Hesam,
Jean de Montigny,
Vasileios Vavourakis,
Alexandros Iosif,
Jack Jennings,
Marcus Kaiser,
Marco Manca,
Alberto Di Meglio,
Zaid Al-Ars,
Fons Rademakers,
Onur Mutlu,
Roman Bauer
Abstract:
Motivation: Agent-based modeling is an indispensable tool for studying complex biological systems. However, existing simulators do not always take full advantage of modern hardware and often have a field-specific software design.
Results: We present a novel simulation platform called BioDynaMo that alleviates both of these problems. BioDynaMo features a general-purpose and high-performance simul…
▽ More
Motivation: Agent-based modeling is an indispensable tool for studying complex biological systems. However, existing simulators do not always take full advantage of modern hardware and often have a field-specific software design.
Results: We present a novel simulation platform called BioDynaMo that alleviates both of these problems. BioDynaMo features a general-purpose and high-performance simulation engine. We demonstrate that BioDynaMo can be used to simulate use cases in: neuroscience, oncology, and epidemiology. For each use case we validate our findings with experimental data or an analytical solution. Our performance results show that BioDynaMo performs up to three orders of magnitude faster than the state-of-the-art baseline. This improvement makes it feasible to simulate each use case with one billion agents on a single server, showcasing the potential BioDynaMo has for computational biology research.
Availability: BioDynaMo is an open-source project under the Apache 2.0 license and is available at www.biodynamo.org. Instructions to reproduce the results are available in supplementary information.
Contact: [email protected], [email protected], [email protected], [email protected]
Supplementary information: Available at https://doi.org/10.5281/zenodo.4501515
△ Less
Submitted 5 February, 2021; v1 submitted 11 June, 2020;
originally announced June 2020.
-
Augmented Curation of Unstructured Clinical Notes from a Massive EHR System Reveals Specific Phenotypic Signature of Impending COVID-19 Diagnosis
Authors:
FNU Shweta,
Karthik Murugadoss,
Samir Awasthi,
AJ Venkatakrishnan,
Arjun Puranik,
Martin Kang,
Brian W. Pickering,
John C. O'Horo,
Philippe R. Bauer,
Raymund R. Razonable,
Paschalis Vergidis,
Zelalem Temesgen,
Stacey Rizza,
Maryam Mahmood,
Walter R. Wilson,
Douglas Challener,
Praveen Anand,
Matt Liebers,
Zainab Doctor,
Eli Silvert,
Hugo Solomon,
Tyler Wagner,
Gregory J. Gores,
Amy W. Williams,
John Halamka
, et al. (2 additional authors not shown)
Abstract:
Understanding the temporal dynamics of COVID-19 patient phenotypes is necessary to derive fine-grained resolution of pathophysiology. Here we use state-of-the-art deep neural networks over an institution-wide machine intelligence platform for the augmented curation of 15.8 million clinical notes from 30,494 patients subjected to COVID-19 PCR diagnostic testing. By contrasting the Electronic Health…
▽ More
Understanding the temporal dynamics of COVID-19 patient phenotypes is necessary to derive fine-grained resolution of pathophysiology. Here we use state-of-the-art deep neural networks over an institution-wide machine intelligence platform for the augmented curation of 15.8 million clinical notes from 30,494 patients subjected to COVID-19 PCR diagnostic testing. By contrasting the Electronic Health Record (EHR)-derived clinical phenotypes of COVID-19-positive (COVIDpos, n=635) versus COVID-19-negative (COVIDneg, n=29,859) patients over each day of the week preceding the PCR testing date, we identify anosmia/dysgeusia (37.4-fold), myalgia/arthralgia (2.6-fold), diarrhea (2.2-fold), fever/chills (2.1-fold), respiratory difficulty (1.9-fold), and cough (1.8-fold) as significantly amplified in COVIDpos over COVIDneg patients. The specific combination of cough and diarrhea has a 3.2-fold amplification in COVIDpos patients during the week prior to PCR testing, and along with anosmia/dysgeusia, constitutes the earliest EHR-derived signature of COVID-19 (4-7 days prior to typical PCR testing date). This study introduces an Augmented Intelligence platform for the real-time synthesis of institutional knowledge captured in EHRs. The platform holds tremendous potential for scaling up curation throughput, with minimal need for retraining underlying neural networks, thus promising EHR-powered early diagnosis for a broad spectrum of diseases.
△ Less
Submitted 28 April, 2020; v1 submitted 17 April, 2020;
originally announced April 2020.
-
menoci: Lightweight Extensible Web Portal enabling FAIR Data Management for Biomedical Research Projects
Authors:
Markus Suhr,
Christoph Lehmann,
Christian Robert Bauer,
Theresa Bender,
Cornelius Knopp,
Luca Freckmann,
Björn Öst Hansen,
Christian Henke,
Georg Aschenbrandt,
Lea Kühlborn,
Sophia Rheinländer,
Linus Weber,
Bartlomiej Marzec,
Marcel Hellkamp,
Philipp Wieder,
Harald Kusch,
Ulrich Sax,
Sara Yasemin Nussbeck
Abstract:
Background: Biomedical research projects deal with data management requirements from multiple sources like funding agencies' guidelines, publisher policies, discipline best practices, and their own users' needs. We describe functional and quality requirements based on many years of experience implementing data management for the CRC 1002 and CRC 1190. A fully equipped data management software shou…
▽ More
Background: Biomedical research projects deal with data management requirements from multiple sources like funding agencies' guidelines, publisher policies, discipline best practices, and their own users' needs. We describe functional and quality requirements based on many years of experience implementing data management for the CRC 1002 and CRC 1190. A fully equipped data management software should improve documentation of experiments and materials, enable data storage and sharing according to the FAIR Guiding Principles while maximizing usability, information security, as well as software sustainability and reusability. Results: We introduce the modular web portal software menoci for data collection, experiment documentation, data publication, sharing, and preservation in biomedical research projects. Menoci modules are based on the Drupal content management system which enables lightweight deployment and setup, and creates the possibility to combine research data management with a customisable project home page or collaboration platform. Conclusions: Management of research data and digital research artefacts is transforming from individual researcher or groups best practices towards project- or organisation-wide service infrastructures. To enable and support this structural transformation process, a vital ecosystem of open source software tools is needed. Menoci is a contribution to this ecosystem of research data management tools that is specifically designed to support biomedical research projects.
△ Less
Submitted 7 February, 2020;
originally announced February 2020.
-
The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency Response System
Authors:
Ankit Agrawal,
Sophia Abraham,
Benjamin Burger,
Chichi Christine,
Luke Fraser,
John Hoeksema,
Sara Hwang,
Elizabeth Travnik,
Shreya Kumar,
Walter Scheirer,
Jane Cleland-Huang,
Michael Vierhauser,
Ryan Bauer,
Steve Cox
Abstract:
The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, m…
▽ More
The use of semi-autonomous Unmanned Aerial Vehicles (UAV) to support emergency response scenarios, such as fire surveillance and search and rescue, offers the potential for huge societal benefits. However, designing an effective solution in this complex domain represents a "wicked design" problem, requiring a careful balance between trade-offs associated with drone autonomy versus human control, mission functionality versus safety, and the diverse needs of different stakeholders. This paper focuses on designing for situational awareness (SA) using a scenario-driven, participatory design process. We developed SA cards describing six common design-problems, known as SA demons, and three new demons of importance to our domain. We then used these SA cards to equip domain experts with SA knowledge so that they could more fully engage in the design process. We designed a potentially reusable solution for achieving SA in multi-stakeholder, multi-UAV, emergency response applications.
△ Less
Submitted 11 January, 2020;
originally announced January 2020.
-
A Dual Encoder Sequence to Sequence Model for Open-Domain Dialogue Modeling
Authors:
Sharath T. S.,
Shubhangi Tandon,
Ryan Bauer
Abstract:
Ever since the successful application of sequence to sequence learning for neural machine translation systems, interest has surged in its applicability towards language generation in other problem domains. Recent work has investigated the use of these neural architectures towards modeling open-domain conversational dialogue, where it has been found that although these models are capable of learnin…
▽ More
Ever since the successful application of sequence to sequence learning for neural machine translation systems, interest has surged in its applicability towards language generation in other problem domains. Recent work has investigated the use of these neural architectures towards modeling open-domain conversational dialogue, where it has been found that although these models are capable of learning a good distributional language model, dialogue coherence is still of concern. Unlike translation, conversation is much more a one-to-many mapping from utterance to a response, and it is even more pressing that the model be aware of the preceding flow of conversation. In this paper we propose to tackle this problem by introducing previous conversational context in terms of latent representations of dialogue acts over time. We inject the latent context representations into a sequence to sequence neural network in the form of dialog acts using a second encoder to enhance the quality and the coherence of the conversations generated. The main task of this research work is to show that adding latent variables that capture discourse relations does indeed result in more coherent responses when compared to conventional sequence to sequence models.
△ Less
Submitted 28 October, 2017;
originally announced October 2017.
-
A hybrid approach to supervised machine learning for algorithmic melody composition
Authors:
Rouven Bauer
Abstract:
In this work we present an algorithm for composing monophonic melodies similar in style to those of a given, phrase annotated, sample of melodies. For implementation, a hybrid approach incorporating parametric Markov models of higher order and a contour concept of phrases is used. This work is based on the master thesis of Thayabaran Kathiresan (2015). An online listening test conducted shows that…
▽ More
In this work we present an algorithm for composing monophonic melodies similar in style to those of a given, phrase annotated, sample of melodies. For implementation, a hybrid approach incorporating parametric Markov models of higher order and a contour concept of phrases is used. This work is based on the master thesis of Thayabaran Kathiresan (2015). An online listening test conducted shows that enhancing a pure Markov model with musically relevant context, like count and planed melody contour, improves the result significantly.
△ Less
Submitted 29 December, 2016;
originally announced December 2016.
-
The BioDynaMo Project: Creating a Platform for Large-Scale Reproducible Biological Simulations
Authors:
Lukas Breitwieser,
Roman Bauer,
Alberto Di Meglio,
Leonard Johard,
Marcus Kaiser,
Marco Manca,
Manuel Mazzara,
Fons Rademakers,
Max Talanov
Abstract:
Computer simulations have become a very powerful tool for scientific research. In order to facilitate research in computational biology, the BioDynaMo project aims at a general platform for biological computer simulations, which should be executable on hybrid cloud computing systems. This paper describes challenges and lessons learnt during the early stages of the software development process, in…
▽ More
Computer simulations have become a very powerful tool for scientific research. In order to facilitate research in computational biology, the BioDynaMo project aims at a general platform for biological computer simulations, which should be executable on hybrid cloud computing systems. This paper describes challenges and lessons learnt during the early stages of the software development process, in the context of implementation issues and the international nature of the collaboration.
△ Less
Submitted 17 August, 2016;
originally announced August 2016.
-
An Evolutionary Algorithm to Learn SPARQL Queries for Source-Target-Pairs: Finding Patterns for Human Associations in DBpedia
Authors:
Jörn Hees,
Rouven Bauer,
Joachim Folz,
Damian Borth,
Andreas Dengel
Abstract:
Efficient usage of the knowledge provided by the Linked Data community is often hindered by the need for domain experts to formulate the right SPARQL queries to answer questions. For new questions they have to decide which datasets are suitable and in which terminology and modelling style to phrase the SPARQL query.
In this work we present an evolutionary algorithm to help with this challenging…
▽ More
Efficient usage of the knowledge provided by the Linked Data community is often hindered by the need for domain experts to formulate the right SPARQL queries to answer questions. For new questions they have to decide which datasets are suitable and in which terminology and modelling style to phrase the SPARQL query.
In this work we present an evolutionary algorithm to help with this challenging task. Given a training list of source-target node-pair examples our algorithm can learn patterns (SPARQL queries) from a SPARQL endpoint. The learned patterns can be visualised to form the basis for further investigation, or they can be used to predict target nodes for new source nodes.
Amongst others, we apply our algorithm to a dataset of several hundred human associations (such as "circle - square") to find patterns for them in DBpedia. We show the scalability of the algorithm by running it against a SPARQL endpoint loaded with > 7.9 billion triples. Further, we use the resulting SPARQL queries to mimic human associations with a Mean Average Precision (MAP) of 39.9 % and a Recall@10 of 63.9 %.
△ Less
Submitted 13 September, 2016; v1 submitted 25 July, 2016;
originally announced July 2016.
-
The BioDynaMo Project
Authors:
Roman Bauer,
Lukas Breitwieser,
Alberto Di Meglio,
Leonard Johard,
Marcus Kaiser,
Marco Manca,
Manuel Mazzara,
Max Talanov
Abstract:
Computer simulations have become a very powerful tool for scientific research. Given the vast complexity that comes with many open scientific questions, a purely analytical or experimental approach is often not viable. For example, biological systems (such as the human brain) comprise an extremely complex organization and heterogeneous interactions across different spatial and temporal scales. In…
▽ More
Computer simulations have become a very powerful tool for scientific research. Given the vast complexity that comes with many open scientific questions, a purely analytical or experimental approach is often not viable. For example, biological systems (such as the human brain) comprise an extremely complex organization and heterogeneous interactions across different spatial and temporal scales. In order to facilitate research on such problems, the BioDynaMo project (\url{https://biodynamo.web.cern.ch/}) aims at a general platform for computer simulations for biological research. Since the scientific investigations require extensive computer resources, this platform should be executable on hybrid cloud computing systems, allowing for the efficient use of state-of-the-art computing technology. This paper describes challenges during the early stages of the software development process. In particular, we describe issues regarding the implementation and the highly interdisciplinary as well as international nature of the collaboration. Moreover, we explain the methodologies, the approach, and the lessons learnt by the team during these first stages.
△ Less
Submitted 10 July, 2016;
originally announced July 2016.
