-
Deep Reinforcement Learning for Optimal Critical Care Pain Management with Morphine using Dueling Double-Deep Q Networks
Authors:
Daniel Lopez-Martinez,
Patrick Eschenfeldt,
Sassan Ostvar,
Myles Ingram,
Chin Hur,
Rosalind Picard
Abstract:
Opioids are the preferred medications for the treatment of pain in the intensive care unit. While undertreatment leads to unrelieved pain and poor clinical outcomes, excessive use of opioids puts patients at risk of experiencing multiple adverse effects. In this work, we present a sequential decision making framework for opioid dosing based on deep reinforcement learning. It provides real-time cli…
▽ More
Opioids are the preferred medications for the treatment of pain in the intensive care unit. While undertreatment leads to unrelieved pain and poor clinical outcomes, excessive use of opioids puts patients at risk of experiencing multiple adverse effects. In this work, we present a sequential decision making framework for opioid dosing based on deep reinforcement learning. It provides real-time clinically interpretable dosing recommendations, personalized according to each patient's evolving pain and physiological condition. We focus on morphine, one of the most commonly prescribed opioids. To train and evaluate the model, we used retrospective data from the publicly available MIMIC-3 database. Our results demonstrate that reinforcement learning may be used to aid decision making in the intensive care setting by providing personalized pain management interventions.
△ Less
Submitted 24 April, 2019;
originally announced April 2019.
-
A hierarchy of models for simulating experimental results from a 3D heterogeneous porous medium
Authors:
Daniel Vogler,
Sassan Ostvar,
Rebecca Paustian,
Brian Wood
Abstract:
In this work we examine the dispersion of conservative tracers (bromide and fluorescein) in an experimentally-constructed three-dimensional dual-porosity porous medium. The medium is highly heterogeneous ($σ_Y^2=5.7$), and consists of spherical, low-hydraulic-conductivity inclusions embedded in a high-hydraulic-conductivity matrix. The bi-modal medium was saturated with tracers, and then flushed w…
▽ More
In this work we examine the dispersion of conservative tracers (bromide and fluorescein) in an experimentally-constructed three-dimensional dual-porosity porous medium. The medium is highly heterogeneous ($σ_Y^2=5.7$), and consists of spherical, low-hydraulic-conductivity inclusions embedded in a high-hydraulic-conductivity matrix. The bi-modal medium was saturated with tracers, and then flushed with tracer-free fluid while the effluent breakthrough curves were measured. The focus for this work is to examine a hierarchy of four models (in the absence of adjustable parameters) with decreasing complexity to assess their ability to accurately represent the measured breakthrough curves. The most information-rich model was (1) a direct numerical simulation of the system in which the geometry, boundary and initial conditions, and medium properties were fully independently characterized experimentally with high fidelity. The reduced models included; (2) a simplified numerical model identical to the fully-resolved direct numerical simulation (DNS) model, but using a domain that was one-tenth the size; (3) an upscaled mobile-immobile model that allowed for a time-dependent mass-transfer coefficient; and, (4) an upscaled mobile-immobile model that assumed a space-time constant mass-transfer coefficient. The results illustrated that all four models provided accurate representations of the experimental breakthrough curves as measured by global RMS error. The primary component of error induced in the upscaled models appeared to arise from the neglect of convection within the inclusions. Interestingly, these results suggested that the conventional convection-dispersion equation, when applied in a way that resolves the heterogeneities, yields models with high fidelity without requiring the imposition of a more complex non-Fickian model.
△ Less
Submitted 3 February, 2018;
originally announced February 2018.
-
Series Solutions for Orthotropic Diffusion in a Cube
Authors:
Brian D. Wood,
Sassan Ostvar
Abstract:
Analytical solutions to heat or diffusion type equations are numerous, but there are rather few explicit solutions for conditions where the thermal conductivity or diffusion tensors are anisotropic. Such solutions have some use in making predictions for idealization of real systems, but are perhaps most useful for providing benchmark solutions which can be used to validate numerical codes. In this…
▽ More
Analytical solutions to heat or diffusion type equations are numerous, but there are rather few explicit solutions for conditions where the thermal conductivity or diffusion tensors are anisotropic. Such solutions have some use in making predictions for idealization of real systems, but are perhaps most useful for providing benchmark solutions which can be used to validate numerical codes. In this short paper, we present the transient solution to the diffusion equation in a cube under conditions of orthotropic anisotropy in the effective thermal conductivity or diffusion tensor. In particular, we consider the physically-relevant case of transport in a cube with no-flux boundaries for several initial conditions including: (1) a delta function, (2) a truncated Gaussian function, (3) a step function, and (4) a planar function. The potential relevance for each of these initial conditions in the context of validating numerical codes is discussed.
△ Less
Submitted 8 July, 2015;
originally announced July 2015.
