-
Learning to Defer to a Population: A Meta-Learning Approach
Authors:
Dharmesh Tailor,
Aditya Patra,
Rajeev Verma,
Putra Manggala,
Eric Nalisnick
Abstract:
The learning to defer (L2D) framework allows autonomous systems to be safe and robust by allocating difficult decisions to a human expert. All existing work on L2D assumes that each expert is well-identified, and if any expert were to change, the system should be re-trained. In this work, we alleviate this constraint, formulating an L2D system that can cope with never-before-seen experts at test-t…
▽ More
The learning to defer (L2D) framework allows autonomous systems to be safe and robust by allocating difficult decisions to a human expert. All existing work on L2D assumes that each expert is well-identified, and if any expert were to change, the system should be re-trained. In this work, we alleviate this constraint, formulating an L2D system that can cope with never-before-seen experts at test-time. We accomplish this by using meta-learning, considering both optimization- and model-based variants. Given a small context set to characterize the currently available expert, our framework can quickly adapt its deferral policy. For the model-based approach, we employ an attention mechanism that is able to look for points in the context set that are similar to a given test point, leading to an even more precise assessment of the expert's abilities. In the experiments, we validate our methods on image recognition, traffic sign detection, and skin lesion diagnosis benchmarks.
△ Less
Submitted 13 May, 2024; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles
Authors:
Rajeev Verma,
Daniel Barrejón,
Eric Nalisnick
Abstract:
We study the statistical properties of learning to defer (L2D) to multiple experts. In particular, we address the open problems of deriving a consistent surrogate loss, confidence calibration, and principled ensembling of experts. Firstly, we derive two consistent surrogates -- one based on a softmax parameterization, the other on a one-vs-all (OvA) parameterization -- that are analogous to the si…
▽ More
We study the statistical properties of learning to defer (L2D) to multiple experts. In particular, we address the open problems of deriving a consistent surrogate loss, confidence calibration, and principled ensembling of experts. Firstly, we derive two consistent surrogates -- one based on a softmax parameterization, the other on a one-vs-all (OvA) parameterization -- that are analogous to the single expert losses proposed by Mozannar and Sontag (2020) and Verma and Nalisnick (2022), respectively. We then study the frameworks' ability to estimate P( m_j = y | x ), the probability that the jth expert will correctly predict the label for x. Theory shows the softmax-based loss causes mis-calibration to propagate between the estimates while the OvA-based loss does not (though in practice, we find there are trade offs). Lastly, we propose a conformal inference technique that chooses a subset of experts to query when the system defers. We perform empirical validation on tasks for galaxy, skin lesion, and hate speech classification.
△ Less
Submitted 23 February, 2023; v1 submitted 30 October, 2022;
originally announced October 2022.
-
Reliability-Based Robust Design Optimization Method for Engineering Systems with Uncertainty Quantification
Authors:
Richa Verma,
Dinesh Kumar,
Kazuma Kobayashi,
Syed Alam
Abstract:
Robust optimization is a method for optimization under uncertainties in engineering systems and designs for applications ranging from aeronautics to nuclear. In a robust design process, parameter variability (or uncertainty) is incorporated into the engineering systems' optimization process to assure the systems' quality and reliability. This chapter focuses on a robust optimization approach for d…
▽ More
Robust optimization is a method for optimization under uncertainties in engineering systems and designs for applications ranging from aeronautics to nuclear. In a robust design process, parameter variability (or uncertainty) is incorporated into the engineering systems' optimization process to assure the systems' quality and reliability. This chapter focuses on a robust optimization approach for developing robust and reliable advanced systems and explains the framework for using uncertainty quantification and optimization techniques. For the uncertainty analysis, a polynomial chaos-based approach is combined with the optimization algorithms MOSA (Multi-Objective Simulated Annealing), and the process is discussed with a simplified test function. For the optimization process, gradient-free genetic algorithms are considered as the optimizer scans the whole design space, and the optimal values are not always dependent on the initial values.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
Leveraging Industry 4.0 -- Deep Learning, Surrogate Model and Transfer Learning with Uncertainty Quantification Incorporated into Digital Twin for Nuclear System
Authors:
M. Rahman,
Abid Khan,
Sayeed Anowar,
Md Al-Imran,
Richa Verma,
Dinesh Kumar,
Kazuma Kobayashi,
Syed Alam
Abstract:
Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments…
▽ More
Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogate models are excellent substitutes for fast computational analysis and the probabilistic prediction of the output parameter for new input parameters. The emergence of Internet of Things (IoT) and Machine Learning (ML) has made the concept of surrogate modeling even more viable. However, these surrogate models contain intrinsic uncertainties, originate from modeling defects, or both. These uncertainties, if not quantified and minimized, can produce a skewed result. Therefore, proper implementation of uncertainty quantification techniques is crucial during optimization, cost reduction, or safety enhancement processes analysis. This chapter begins with a brief overview of the concept of surrogate modeling, transfer learning, IoT and digital twins. After that, a detailed overview of uncertainties, uncertainty quantification frameworks, and specifics of uncertainty quantification methodologies for a surrogate model linked to a digital twin is presented. Finally, the use of uncertainty quantification approaches in the nuclear industry has been addressed.
△ Less
Submitted 30 September, 2022;
originally announced October 2022.
-
Machine Learning and Artificial Intelligence-Driven Multi-Scale Modeling for High Burnup Accident-Tolerant Fuels for Light Water-Based SMR Applications
Authors:
Md. Shamim Hassan,
Abid Hossain Khan,
Richa Verma,
Dinesh Kumar,
Kazuma Kobayashi,
Shoaib Usman,
Syed Alam
Abstract:
The concept of small modular reactor has changed the outlook for tackling future energy crises. This new reactor technology is very promising considering its lower investment requirements, modularity, design simplicity, and enhanced safety features. The application of artificial intelligence-driven multi-scale modeling (neutronics, thermal hydraulics, fuel performance, etc.) incorporating Digital…
▽ More
The concept of small modular reactor has changed the outlook for tackling future energy crises. This new reactor technology is very promising considering its lower investment requirements, modularity, design simplicity, and enhanced safety features. The application of artificial intelligence-driven multi-scale modeling (neutronics, thermal hydraulics, fuel performance, etc.) incorporating Digital Twin and associated uncertainties in the research of small modular reactors is a recent concept. In this work, a comprehensive study is conducted on the multiscale modeling of accident-tolerant fuels. The application of these fuels in the light water-based small modular reactors is explored. This chapter also focuses on the application of machine learning and artificial intelligence in the design optimization, control, and monitoring of small modular reactors. Finally, a brief assessment of the research gap on the application of artificial intelligence to the development of high burnup composite accident-tolerant fuels is provided. Necessary actions to fulfill these gaps are also discussed.
△ Less
Submitted 25 September, 2022;
originally announced September 2022.
-
Calibrated Learning to Defer with One-vs-All Classifiers
Authors:
Rajeev Verma,
Eric Nalisnick
Abstract:
The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag's (2020) multiclass framework is not calibrated with…
▽ More
The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag's (2020) multiclass framework is not calibrated with respect to expert correctness. Moreover, it is not even guaranteed to produce valid probabilities due to its parameterization being degenerate for this purpose. We propose an L2D system based on one-vs-all classifiers that is able to produce calibrated probabilities of expert correctness. Furthermore, our loss function is also a consistent surrogate for multiclass L2D, like Mozannar & Sontag's (2020). Our experiments verify that not only is our system calibrated, but this benefit comes at no cost to accuracy. Our model's accuracy is always comparable (and often superior) to Mozannar & Sontag's (2020) model's in tasks ranging from hate speech detection to galaxy classification to diagnosis of skin lesions.
△ Less
Submitted 18 June, 2022; v1 submitted 8 February, 2022;
originally announced February 2022.
-
SIBRE: Self Improvement Based REwards for Adaptive Feedback in Reinforcement Learning
Authors:
Somjit Nath,
Richa Verma,
Abhik Ray,
Harshad Khadilkar
Abstract:
We propose a generic reward shaping approach for improving the rate of convergence in reinforcement learning (RL), called Self Improvement Based REwards, or SIBRE. The approach is designed for use in conjunction with any existing RL algorithm, and consists of rewarding improvement over the agent's own past performance. We prove that SIBRE converges in expectation under the same conditions as the o…
▽ More
We propose a generic reward shaping approach for improving the rate of convergence in reinforcement learning (RL), called Self Improvement Based REwards, or SIBRE. The approach is designed for use in conjunction with any existing RL algorithm, and consists of rewarding improvement over the agent's own past performance. We prove that SIBRE converges in expectation under the same conditions as the original RL algorithm. The reshaped rewards help discriminate between policies when the original rewards are weakly discriminated or sparse. Experiments on several well-known benchmark environments with different RL algorithms show that SIBRE converges to the optimal policy faster and more stably. We also perform sensitivity analysis with respect to hyper-parameters, in comparison with baseline RL algorithms.
△ Less
Submitted 21 December, 2020; v1 submitted 21 April, 2020;
originally announced April 2020.
-
MRQy: An Open-Source Tool for Quality Control of MR Imaging Data
Authors:
Amir Reza Sadri,
Andrew Janowczyk,
Ren Zou,
Ruchika Verma,
Niha Beig,
Jacob Antunes,
Anant Madabhushi,
Pallavi Tiwari,
Satish E. Viswanath
Abstract:
We sought to develop a quantitative tool to quickly determine relative differences in MRI volumes both within and between large MR imaging cohorts (such as available in The Cancer Imaging Archive (TCIA)), in order to help determine the generalizability of radiomics and machine learning schemes to unseen datasets. The tool is intended to help quantify presence of (a) site- or scanner-specific varia…
▽ More
We sought to develop a quantitative tool to quickly determine relative differences in MRI volumes both within and between large MR imaging cohorts (such as available in The Cancer Imaging Archive (TCIA)), in order to help determine the generalizability of radiomics and machine learning schemes to unseen datasets. The tool is intended to help quantify presence of (a) site- or scanner-specific variations in image resolution, field-of-view, or image contrast, or (b) imaging artifacts such as noise, motion, inhomogeneity, ringing, or aliasing; which can adversely affect relative image quality between data cohorts. We present MRQy, a new open-source quality control tool to (a) interrogate MRI cohorts for site- or equipment-based differences, and (b) quantify the impact of MRI artifacts on relative image quality; to help determine how to correct for these variations prior to model development. MRQy extracts a series of quality measures (e.g. noise ratios, variation metrics, entropy and energy criteria) and MR image metadata (e.g. voxel resolution, image dimensions) for subsequent interrogation via a specialized HTML5 based front-end designed for real-time filtering and trend visualization. MRQy was used to evaluate (a) n=133 brain MRIs from TCIA (7 sites), and (b) n=104 rectal MRIs (3 local sites). MRQy measures revealed significant site-specific variations in both cohorts, indicating potential batch effects. Marked differences in specific MRQy measures were also able to identify outlier MRI datasets that needed to be corrected for common MR imaging artifacts. MRQy is designed to be a standalone, unsupervised tool that can be efficiently run on a standard desktop computer. It has been made freely accessible at \url{http://github.com/ccipd/MRQy} for wider community use and feedback.
△ Less
Submitted 17 August, 2020; v1 submitted 9 April, 2020;
originally announced April 2020.
-
Feature Engineering Combined with 1 D Convolutional Neural Network for Improved Mortality Prediction
Authors:
Saumil Maheshwari,
Rohit Verma,
Anupam Shukla,
Ritu Tiwari,
Rishu Garg
Abstract:
The intensive care units (ICUs) are responsible for generating a wealth of useful data in the form of Electronic Health Record (EHR). This data allows for the development of a prediction tool with perfect knowledge backing. We aimed to build a mortality prediction model on 2012 Physionet Challenge mortality prediction database of 4000 patients admitted in ICU. The challenges in the dataset, such a…
▽ More
The intensive care units (ICUs) are responsible for generating a wealth of useful data in the form of Electronic Health Record (EHR). This data allows for the development of a prediction tool with perfect knowledge backing. We aimed to build a mortality prediction model on 2012 Physionet Challenge mortality prediction database of 4000 patients admitted in ICU. The challenges in the dataset, such as high dimensionality, imbalanced distribution, and missing values were tackled with analytical methods and tools via feature engineering and new variable construction. The objective of the research is to utilize the relations among the clinical variables and construct new variables which would establish the effectiveness of 1-Dimensional Convolutional Neural Network (1- D CNN) with constructed features. Its performance with the traditional machine learning algorithms like XGBoost classifier, Support Vector Machine (SVM), K-Neighbours Classifier (K-NN), and Random Forest Classifier (RF) is compared for Area Under Curve (AUC). The investigation reveals the best AUC of 0.848 using 1-D CNN model.
△ Less
Submitted 27 July, 2020; v1 submitted 8 December, 2019;
originally announced December 2019.
-
Accelerating Training in Pommerman with Imitation and Reinforcement Learning
Authors:
Hardik Meisheri,
Omkar Shelke,
Richa Verma,
Harshad Khadilkar
Abstract:
The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimizat…
▽ More
The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimization (PPO) reinforcement learning algorithm. The basic PPO approach is modified for stable transition from the imitation learning phase through reward shaping, action filters based on heuristics, and curriculum learning. The proposed methodology is able to beat heuristic and pure reinforcement learning baselines with a combined 100,000 training games, significantly faster than other non-tree-search methods in literature. We present results against multiple agents provided by the developers of the simulation, including some that we have enhanced. We include a sensitivity analysis over different parameters, and highlight undesirable effects of some strategies that initially appear promising. Since Pommerman is a complex multi-agent competitive environment, the strategies developed here provide insights into several real-world problems with characteristics such as partial observability, decentralized execution (without communication), and very sparse and delayed rewards.
△ Less
Submitted 13 November, 2019; v1 submitted 12 November, 2019;
originally announced November 2019.
