-
A prototype hybrid prediction market for estimating replicability of published work
Authors:
Tatiana Chakravorti,
Robert Fraleigh,
Timothy Fritton,
Michael McLaughlin,
Vaibhav Singh,
Christopher Griffin,
Anthony Kwasnica,
David Pennock,
C. Lee Giles,
Sarah Rajtmajer
Abstract:
We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. We build on prior work proposing artificial prediction markets as a novel machine-learning algorithm. In an artificial prediction market, trained AI agents buy and sell outcomes of future events. Classification decisions can be framed as outcomes of future events, and acc…
▽ More
We present a prototype hybrid prediction market and demonstrate the avenue it represents for meaningful human-AI collaboration. We build on prior work proposing artificial prediction markets as a novel machine-learning algorithm. In an artificial prediction market, trained AI agents buy and sell outcomes of future events. Classification decisions can be framed as outcomes of future events, and accordingly, the price of an asset corresponding to a given classification outcome can be taken as a proxy for the confidence of the system in that decision. By embedding human participants in these markets alongside bot traders, we can bring together insights from both. In this paper, we detail pilot studies with prototype hybrid markets for the prediction of replication study outcomes. We highlight challenges and opportunities, share insights from semi-structured interviews with hybrid market participants, and outline a vision for ongoing and future work.
△ Less
Submitted 1 March, 2023;
originally announced March 2023.
-
Artificial prediction markets present a novel opportunity for human-AI collaboration
Authors:
Tatiana Chakravorti,
Vaibhav Singh,
Sarah Rajtmajer,
Michael McLaughlin,
Robert Fraleigh,
Christopher Griffin,
Anthony Kwasnica,
David Pennock,
C. Lee Giles
Abstract:
Despite high-profile successes in the field of Artificial Intelligence, machine-driven technologies still suffer important limitations, particularly for complex tasks where creativity, planning, common sense, intuition, or learning from limited data is required. These limitations motivate effective methods for human-machine collaboration. Our work makes two primary contributions. We thoroughly exp…
▽ More
Despite high-profile successes in the field of Artificial Intelligence, machine-driven technologies still suffer important limitations, particularly for complex tasks where creativity, planning, common sense, intuition, or learning from limited data is required. These limitations motivate effective methods for human-machine collaboration. Our work makes two primary contributions. We thoroughly experiment with an artificial prediction market model to understand the effects of market parameters on model performance for benchmark classification tasks. We then demonstrate, through simulation, the impact of exogenous agents in the market, where these exogenous agents represent primitive human behaviors. This work lays the foundation for a novel set of hybrid human-AI machine learning algorithms.
△ Less
Submitted 29 November, 2022;
originally announced November 2022.
-
A Synthetic Prediction Market for Estimating Confidence in Published Work
Authors:
Sarah Rajtmajer,
Christopher Griffin,
Jian Wu,
Robert Fraleigh,
Laxmaan Balaji,
Anna Squicciarini,
Anthony Kwasnica,
David Pennock,
Michael McLaughlin,
Timothy Fritton,
Nishanth Nakshatri,
Arjun Menon,
Sai Ajay Modukuri,
Rajal Nivargi,
Xin Wei,
C. Lee Giles
Abstract:
Explainably estimating confidence in published scholarly work offers opportunity for faster and more robust scientific progress. We develop a synthetic prediction market to assess the credibility of published claims in the social and behavioral sciences literature. We demonstrate our system and detail our findings using a collection of known replication projects. We suggest that this work lays the…
▽ More
Explainably estimating confidence in published scholarly work offers opportunity for faster and more robust scientific progress. We develop a synthetic prediction market to assess the credibility of published claims in the social and behavioral sciences literature. We demonstrate our system and detail our findings using a collection of known replication projects. We suggest that this work lays the foundation for a research agenda that creatively uses AI for peer review.
△ Less
Submitted 23 December, 2021;
originally announced January 2022.
-
Improving Emergency Department ESI Acuity Assignment Using Machine Learning and Clinical Natural Language Processing
Authors:
Oleksandr Ivanov,
Lisa Wolf,
Deena Brecher,
Kevin Masek,
Erica Lewis,
Stephen Liu,
Robert B Dunne,
Kevin Klauer,
Kyla Montgomery,
Yurii Andrieiev,
Moss McLaughlin,
Christian Reilly
Abstract:
Effective triage is critical to mitigating the effect of increased volume by accurately determining patient acuity, need for resources, and establishing effective acuity-based patient prioritization. The purpose of this retrospective study was to determine whether historical EHR data can be extracted and synthesized with clinical natural language processing (C-NLP) and the latest ML algorithms (KA…
▽ More
Effective triage is critical to mitigating the effect of increased volume by accurately determining patient acuity, need for resources, and establishing effective acuity-based patient prioritization. The purpose of this retrospective study was to determine whether historical EHR data can be extracted and synthesized with clinical natural language processing (C-NLP) and the latest ML algorithms (KATE) to produce highly accurate ESI predictive models. An ML model (KATE) for the triage process was developed using 166,175 patient encounters from two participating hospitals. The model was then tested against a gold set that was derived from a random sample of triage encounters at the study sites and correct acuity assignments were recorded by study clinicians using the Emergency Severity Index (ESI) standard as a guide. At the two study sites, KATE predicted accurate ESI acuity assignments 75.9% of the time, compared to nurses (59.8%) and average individual study clinicians (75.3%). KATE accuracy was 26.9% higher than the average nurse accuracy (p-value < 0.0001). On the boundary between ESI 2 and ESI 3 acuity assignments, which relates to the risk of decompensation, KATE was 93.2% higher with 80% accuracy, compared to triage nurses with 41.4% accuracy (p-value < 0.0001). KATE provides a triage acuity assignment substantially more accurate than the triage nurses in this study sample. KATE operates independently of contextual factors, unaffected by the external pressures that can cause under triage and may mitigate the racial and social biases that can negatively affect the accuracy of triage assignment. Future research should focus on the impact of KATE providing feedback to triage nurses in real time, KATEs impact on mortality and morbidity, ED throughput, resource optimization, and nursing outcomes.
△ Less
Submitted 22 September, 2020; v1 submitted 29 March, 2020;
originally announced April 2020.
-
Petascale computations for Large-scale Atomic and Molecular collisions
Authors:
Brendan M. McLaughlin,
Connor P. Ballance
Abstract:
Petaflop architectures are currently being utilized efficiently to perform large scale computations in Atomic, Molecular and Optical Collisions. We solve the Schroedinger or Dirac equation for the appropriate collision problem using the R-matrix or R-matrix with pseudo-states approach. We briefly outline the parallel methodology used and implemented for the current suite of Breit-Pauli and DARC co…
▽ More
Petaflop architectures are currently being utilized efficiently to perform large scale computations in Atomic, Molecular and Optical Collisions. We solve the Schroedinger or Dirac equation for the appropriate collision problem using the R-matrix or R-matrix with pseudo-states approach. We briefly outline the parallel methodology used and implemented for the current suite of Breit-Pauli and DARC codes. Various examples are shown of our theoretical results compared with those obtained from Synchrotron Radiation facilities and from Satellite observations. We also indicate future directions and implementation of the R-matrix codes on emerging GPU architectures.
△ Less
Submitted 15 August, 2014; v1 submitted 21 March, 2014;
originally announced March 2014.
-
A Framework for Enabling Distributed Applications on the Internet
Authors:
Mark Anthony McLaughlin
Abstract:
The last five years have seen the rapid rise in popularity of what we term internet distributed applications (IDAs). These are internet applications with which many users interact simultaneously. IDAs range from P2P file-sharing applications, to collaborative distributed computing projects, to massively multiplayer online games (MMOGs). Currently, there is no framework that combines IDAs collectiv…
▽ More
The last five years have seen the rapid rise in popularity of what we term internet distributed applications (IDAs). These are internet applications with which many users interact simultaneously. IDAs range from P2P file-sharing applications, to collaborative distributed computing projects, to massively multiplayer online games (MMOGs). Currently, there is no framework that combines IDAs collectively within a single context. We provide a basis for such a framework here.
In considering IDAs collectively, we found that there was no generic description that had been applied to them as a group. We have therefore put forward such a description here. In our description, IDAs are functionality separated into three logic layers, which are designed and built individually. Each layer is represented by functionality on the software client running on each participating computer, which together comprise the overall IDA.
The core contribution of this work is a framework, called the Internet Distributed Application Framework (IDAF), which outlines how IDAs can be designed, built and run. The IDAF outlines a set of constraints that each implementing software system must abide by. To verify the IDAF, we have built a system prototype implementation called the Internet Distributed Application System (IDAS). The IDAS includes an implementation of the IDAF layer model, which specifies IDAs are built. The IDAS also includes a generic software client that is capable of simultaneously running and managing arbitrary IDAs. We provide sample IDAs and demonstrations to verify both that the IDAF is implementable and that the IDAS is a workable usable system.
△ Less
Submitted 14 June, 2011;
originally announced June 2011.
-
Digital Identity in The Absence of Authorities: A New Socio-Technical Approach
Authors:
Mark McLaughlin,
Gerard Briscoe,
Paul Malone
Abstract:
On the Internet large service providers tend to control the digital identities of users. These defacto identity authorities wield significant power over users, compelling them to comply with non-negotiable terms, before access to services is granted. In doing so, users expose themselves to privacy risks, manipulation and exploitation via direct marketing. Against this backdrop, the emerging areas…
▽ More
On the Internet large service providers tend to control the digital identities of users. These defacto identity authorities wield significant power over users, compelling them to comply with non-negotiable terms, before access to services is granted. In doing so, users expose themselves to privacy risks, manipulation and exploitation via direct marketing. Against this backdrop, the emerging areas of Digital Ecosystems and user-centric identity emphasise decentralised environments with independent self-determining entities that control their own data and identity. We show that recent advances in user-centric identity, federated identity and trust have prepared the ground for decentralised identity provisioning. We show how social trust, rather than blind deference to authorities, can provide a basis for identity, where risks can be weighed and compared rather than merely accepted. Fundamentally, we are considering the move from authority-centric centralised identity provisioning to user-centric distributed identity provisioning. Finally, we highlight the potential impacts of distributed identity provisioning in the Information Society and give a brief roadmap for its general implementation and adoption.
△ Less
Submitted 31 October, 2010;
originally announced November 2010.
