-
Targeted transfer learning to improve performance in small medical physics datasets
Authors:
Miguel Romero,
Yannet Interian,
Timothy Solberg,
Gilmer Valdes
Abstract:
The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare the current state-of-the-art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the…
▽ More
The growing use of Machine Learning has produced significant advances in many fields. For image-based tasks, however, the use of deep learning remains challenging in small datasets. In this article, we review, evaluate and compare the current state-of-the-art techniques in training neural networks to elucidate which techniques work best for small datasets. We further propose a path forward for the improvement of model accuracy in medical imaging applications. We observed best results from one cycle training, discriminative learning rates with gradual freezing and parameter modification after transfer learning. We also established that when datasets are small, transfer learning plays an important role beyond parameter initialization by reusing previously learned features. Surprisingly we observed that there is little advantage in using pre-trained networks in images from another part of the body compared to Imagenet. On the contrary, if images from the same part of the body are available then transfer learning can produce a significant improvement in performance with as little as 50 images in the training data.
△ Less
Submitted 28 September, 2020; v1 submitted 13 December, 2019;
originally announced December 2019.
-
Expert-Augmented Machine Learning
Authors:
E. D. Gennatas,
J. H. Friedman,
L. H. Ungar,
R. Pirracchio,
E. Eaton,
L. Reichman,
Y. Interian,
C. B. Simone,
A. Auerbach,
E. Delgado,
M. J. Van der Laan,
T. D. Solberg,
G. Valdes
Abstract:
Machine Learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption by the level of trust that models afford users. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may…
▽ More
Machine Learning is proving invaluable across disciplines. However, its success is often limited by the quality and quantity of available data, while its adoption by the level of trust that models afford users. Human vs. machine performance is commonly compared empirically to decide whether a certain task should be performed by a computer or an expert. In reality, the optimal learning strategy may involve combining the complementary strengths of man and machine. Here we present Expert-Augmented Machine Learning (EAML), an automated method that guides the extraction of expert knowledge and its integration into machine-learned models. We use a large dataset of intensive care patient data to predict mortality and show that we can extract expert knowledge using an online platform, help reveal hidden confounders, improve generalizability on a different population and learn using less data. EAML presents a novel framework for high performance and dependable machine learning in critical applications.
△ Less
Submitted 5 January, 2021; v1 submitted 22 March, 2019;
originally announced March 2019.
-
Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees
Authors:
José Marcio Luna,
Eric Eaton,
Lyle H. Ungar,
Eric Diffenderfer,
Shane T. Jensen,
Efstathios D. Gennatas,
Mateo Wirth,
Charles B. Simone II,
Timothy D. Solberg,
Gilmer Valdes
Abstract:
Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these two approaches. This paper introduces a novel technique called tree-structured bo…
▽ More
Additive models, such as produced by gradient boosting, and full interaction models, such as classification and regression trees (CART), are widely used algorithms that have been investigated largely in isolation. We show that these models exist along a spectrum, revealing never-before-known connections between these two approaches. This paper introduces a novel technique called tree-structured boosting for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although tree-structured boosting is designed primarily to provide both the model interpretability and predictive performance needed for high-stake applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches.
△ Less
Submitted 17 November, 2017;
originally announced November 2017.
