-
VeriFact: Verifying Facts in LLM-Generated Clinical Text with Electronic Health Records
Authors:
Philip Chung,
Akshay Swaminathan,
Alex J. Goodell,
Yeasul Kim,
S. Momsen Reincke,
Lichy Han,
Ben Deverett,
Mohammad Amin Sadeghi,
Abdel-Badih Ariss,
Marc Ghanem,
David Seong,
Andrew A. Lee,
Caitlin E. Coombes,
Brad Bradshaw,
Mahir A. Sufian,
Hyo Jung Hong,
Teresa P. Nguyen,
Mohammad R. Rasouli,
Komal Kamra,
Mark A. Burbridge,
James C. McAvoy,
Roya Saffary,
Stephen P. Ma,
Dev Dash,
James Xie
, et al. (4 additional authors not shown)
Abstract:
Methods to ensure factual accuracy of text generated by large language models (LLM) in clinical medicine are lacking. VeriFact is an artificial intelligence system that combines retrieval-augmented generation and LLM-as-a-Judge to verify whether LLM-generated text is factually supported by a patient's medical history based on their electronic health record (EHR). To evaluate this system, we introd…
▽ More
Methods to ensure factual accuracy of text generated by large language models (LLM) in clinical medicine are lacking. VeriFact is an artificial intelligence system that combines retrieval-augmented generation and LLM-as-a-Judge to verify whether LLM-generated text is factually supported by a patient's medical history based on their electronic health record (EHR). To evaluate this system, we introduce VeriFact-BHC, a new dataset that decomposes Brief Hospital Course narratives from discharge summaries into a set of simple statements with clinician annotations for whether each statement is supported by the patient's EHR clinical notes. Whereas highest agreement between clinicians was 88.5%, VeriFact achieves up to 92.7% agreement when compared to a denoised and adjudicated average human clinican ground truth, suggesting that VeriFact exceeds the average clinician's ability to fact-check text against a patient's medical record. VeriFact may accelerate the development of LLM-based EHR applications by removing current evaluation bottlenecks.
△ Less
Submitted 27 January, 2025;
originally announced January 2025.
-
AI for Investment: A Platform Disruption
Authors:
Mohammad Rasouli,
Ravi Chiruvolu,
Ali Risheh
Abstract:
With the investment landscape becoming more competitive, efficiently scaling deal sourcing and improving deal insights have become a dominant strategy for funds. While funds are already spending significant efforts on these two tasks, they cannot be scaled with traditional approaches; hence, there is a surge in automating them. Many third party software providers have emerged recently to address t…
▽ More
With the investment landscape becoming more competitive, efficiently scaling deal sourcing and improving deal insights have become a dominant strategy for funds. While funds are already spending significant efforts on these two tasks, they cannot be scaled with traditional approaches; hence, there is a surge in automating them. Many third party software providers have emerged recently to address this need with productivity solutions, but they fail due to a lack of personalization for the fund, privacy constraints, and natural limits of software use cases. Therefore, most major funds and many smaller funds have started developing their in-house AI platforms: a game changer for the industry. These platforms grow smarter by direct interactions with the fund and can be used to provide personalized use cases. Recent developments in large language models, e.g. ChatGPT, have provided an opportunity for other funds to also develop their own AI platforms. While not having an AI platform now is not a competitive disadvantage, it will be in two years. Funds require a practical plan and corresponding risk assessments for such AI platforms.
△ Less
Submitted 6 September, 2023;
originally announced November 2023.
-
Low-rank LQR Optimal Control Design over Wireless Communication Networks
Authors:
Myung Cho,
Abdallah Abdallah,
Mohammad Rasouli
Abstract:
This paper considers a LQR optimal control design problem for distributed control systems with multi-agents. To control large-scale distributed systems such as smart-grid and multi-agent robotic systems over wireless communication networks, it is desired to design a feedback controller by considering various constraints on communication such as limited power, limited energy, or limited communicati…
▽ More
This paper considers a LQR optimal control design problem for distributed control systems with multi-agents. To control large-scale distributed systems such as smart-grid and multi-agent robotic systems over wireless communication networks, it is desired to design a feedback controller by considering various constraints on communication such as limited power, limited energy, or limited communication bandwidth, etc. In this paper, we focus on the reduction of communication energy in an LQR optimal control design problem on wireless communication networks. By considering the characteristic of wireless communication, i.e., Radio Frequency (RF) signal can spread in all directions in a broadcast way, we formulate a low-rank LQR optimal control model to reduce the communication energy in the distributed feedback control system. To solve the problem, we propose an Alternating Direction Method of Multipliers (ADMM) based algorithm. Through various numerical experiments, we demonstrate that a feedback controller designed using low-rank structure can outperform the previous work on sparse LQR optimal control design, which focuses on reducing the number of communication links in a network, in terms of energy consumption, system stability margin against noise and error in communication.
△ Less
Submitted 31 January, 2023;
originally announced January 2023.
-
Data Sharing Markets
Authors:
Mohammad Rasouli,
Michael I. Jordan
Abstract:
With the growing use of distributed machine learning techniques, there is a growing need for data markets that allows agents to share data with each other. Nevertheless data has unique features that separates it from other commodities including replicability, cost of sharing, and ability to distort. We study a setup where each agent can be both buyer and seller of data. For this setup, we consider…
▽ More
With the growing use of distributed machine learning techniques, there is a growing need for data markets that allows agents to share data with each other. Nevertheless data has unique features that separates it from other commodities including replicability, cost of sharing, and ability to distort. We study a setup where each agent can be both buyer and seller of data. For this setup, we consider two cases: bilateral data exchange (trading data with data) and unilateral data exchange (trading data with money). We model bilateral sharing as a network formation game and show the existence of strongly stable outcome under the top agents property by allowing limited complementarity. We propose ordered match algorithm which can find the stable outcome in O(N^2) (N is the number of agents). For the unilateral sharing, under the assumption of additive cost structure, we construct competitive prices that can implement any social welfare maximizing outcome. Finally for this setup when agents have private information, we propose mixed-VCG mechanism which uses zero cost data distortion of data sharing with its isolated impact to achieve budget balance while truthfully implementing socially optimal outcomes to the exact level of budget imbalance of standard VCG mechanisms. Mixed-VCG uses data distortions as data money for this purpose. We further relax zero cost data distortion assumption by proposing distorted-mixed-VCG. We also extend our model and results to data sharing via incremental inquiries and differential privacy costs.
△ Less
Submitted 20 July, 2021; v1 submitted 19 July, 2021;
originally announced July 2021.
-
FedGAN: Federated Generative Adversarial Networks for Distributed Data
Authors:
Mohammad Rasouli,
Tao Sun,
Ram Rajagopal
Abstract:
We propose Federated Generative Adversarial Network (FedGAN) for training a GAN across distributed sources of non-independent-and-identically-distributed data sources subject to communication and privacy constraints. Our algorithm uses local generators and discriminators which are periodically synced via an intermediary that averages and broadcasts the generator and discriminator parameters. We th…
▽ More
We propose Federated Generative Adversarial Network (FedGAN) for training a GAN across distributed sources of non-independent-and-identically-distributed data sources subject to communication and privacy constraints. Our algorithm uses local generators and discriminators which are periodically synced via an intermediary that averages and broadcasts the generator and discriminator parameters. We theoretically prove the convergence of FedGAN with both equal and two time-scale updates of generator and discriminator, under standard assumptions, using stochastic approximations and communication efficient stochastic gradient descents. We experiment FedGAN on toy examples (2D system, mixed Gaussian, and Swiss role), image datasets (MNIST, CIFAR-10, and CelebA), and time series datasets (household electricity consumption and electric vehicle charging sessions). We show FedGAN converges and has similar performance to general distributed GAN, while reduces communication complexity. We also show its robustness to reduced communications.
△ Less
Submitted 15 June, 2020; v1 submitted 12 June, 2020;
originally announced June 2020.
-
Pose-Invariant Object Recognition for Event-Based Vision with Slow-ELM
Authors:
Rohan Ghosh,
Siyi Tang,
Mahdi Rasouli,
Nitish Thakor,
Sunil Kukreja
Abstract:
Neuromorphic image sensors produce activity-driven spiking output at every pixel. These low-power consuming imagers which encode visual change information in the form of spikes help reduce computational overhead and realize complex real-time systems; object recognition and pose-estimation to name a few. However, there exists a lack of algorithms in event-based vision aimed towards capturing invari…
▽ More
Neuromorphic image sensors produce activity-driven spiking output at every pixel. These low-power consuming imagers which encode visual change information in the form of spikes help reduce computational overhead and realize complex real-time systems; object recognition and pose-estimation to name a few. However, there exists a lack of algorithms in event-based vision aimed towards capturing invariance to transformations. In this work, we propose a methodology for recognizing objects invariant to their pose with the Dynamic Vision Sensor (DVS). A novel slow-ELM architecture is proposed which combines the effectiveness of Extreme Learning Machines and Slow Feature Analysis. The system, tested on an Intel Core i5-4590 CPU, can perform 10,000 classifications per second and achieves 1% classification error for 8 objects with views accumulated over 90 degrees of 2D pose.
△ Less
Submitted 19 March, 2019;
originally announced March 2019.
-
A Game-Theoretic Framework for Studying Dynamics of Multi Decision-maker Systems
Authors:
Mohammad Rasouli
Abstract:
System Dynamics (SD) main aim is to study dynamic behavior of systems based on causal relations. The other purpose of the science is to design policies, both in initial values and causal relation, to change system behavior as we desire. Especially we are interested in making systems behavior a convergent one. Although now SD is mainly used in situations of single policy maker, there are major part…
▽ More
System Dynamics (SD) main aim is to study dynamic behavior of systems based on causal relations. The other purpose of the science is to design policies, both in initial values and causal relation, to change system behavior as we desire. Especially we are interested in making systems behavior a convergent one. Although now SD is mainly used in situations of single policy maker, there are major parts of situations in which there are multi policy makers playing role. Game Theory (GT) is an appropriate tool for studying such cases.GT is the theory of studying multi decision-maker conditions. In this paper we will introduce GT and explain how to apply it in SD. Also we will provide some examples of microeconomic systems and show how to use GT for studying and simulating dynamics of these example systems. We will also have a short discuss on how SD can help GT studies.
△ Less
Submitted 24 December, 2014;
originally announced December 2014.
-
A Supervisory Control Approach to Dynamic Cyber-Security
Authors:
Mohammad Rasouli,
Erik Miehling,
Demosthenis Teneketzis
Abstract:
An analytical approach for a dynamic cyber-security problem that captures progressive attacks to a computer network is presented. We formulate the dynamic security problem from the defender's point of view as a supervisory control problem with imperfect information, modeling the computer network's operation by a discrete event system. We consider a min-max performance criterion and use dynamic pro…
▽ More
An analytical approach for a dynamic cyber-security problem that captures progressive attacks to a computer network is presented. We formulate the dynamic security problem from the defender's point of view as a supervisory control problem with imperfect information, modeling the computer network's operation by a discrete event system. We consider a min-max performance criterion and use dynamic programming to determine, within a restricted set of policies, an optimal policy for the defender. We study and interpret the behavior of this optimal policy as we vary certain parameters of the supervisory control problem.
△ Less
Submitted 10 September, 2014; v1 submitted 2 September, 2014;
originally announced September 2014.
-
Electricity Pooling Markets with Strategic Producers Possessing Asymmetric Information II: Inelastic Demand
Authors:
Mohammad Rasouli,
Demosthenis Teneketzis
Abstract:
In the restructured electricity industry, electricity pooling markets are an oligopoly with strategic producers possessing private information (private production cost function). We focus on pooling markets where aggregate demand is represented by a non-strategic agent.
Inelasticity of demand is a main difficulty in electricity markets which can potentially result in market failure and high pric…
▽ More
In the restructured electricity industry, electricity pooling markets are an oligopoly with strategic producers possessing private information (private production cost function). We focus on pooling markets where aggregate demand is represented by a non-strategic agent.
Inelasticity of demand is a main difficulty in electricity markets which can potentially result in market failure and high prices. We consider demand to be inelastic.
We propose a market mechanism that has the following features. (F1) It is individually rational. (F2) It is budget balanced. (F3) It is price efficient, that is, at equilibrium the price of electricity is equal to the marginal cost of production. (F4) The energy production profile corresponding to every non-zero Nash equilibrium of the game induced by the mechanism is a solution of the corresponding centralized problem where the objective is the maximization of the sum of the producers' and consumers' utilities.
We identify some open problems associated with our approach to electricity pooling markets.
△ Less
Submitted 5 October, 2014; v1 submitted 21 April, 2014;
originally announced April 2014.
-
Electricity Pooling Markets with Strategic Producers Possessing Asymmetric Information I: Elastic Demand
Authors:
Mohammad Rasouli,
Demosthenis Teneketzis
Abstract:
In the restructured electricity industry, electricity pooling markets are an oligopoly with strategic producers possessing private information (private production cost function). We focus on pooling markets where aggregate demand is represented by a non-strategic agent. We consider demand to be elastic.
We propose a market mechanism that has the following features. (F1) It is individually ration…
▽ More
In the restructured electricity industry, electricity pooling markets are an oligopoly with strategic producers possessing private information (private production cost function). We focus on pooling markets where aggregate demand is represented by a non-strategic agent. We consider demand to be elastic.
We propose a market mechanism that has the following features. (F1) It is individually rational. (F2) It is budget balanced. (F3) It is price efficient, that is, at equilibrium the price of electricity is equal to the marginal cost of production. (F4) The energy production profile corresponding to every non-zero Nash equilibrium of the game induced by the mechanism is a solution of the corresponding centralized problem where the objective is the maximization of the sum of the producers' and consumers' utilities.
We identify some open problems associated with our approach to electricity pooling markets.
△ Less
Submitted 16 January, 2014;
originally announced January 2014.
