-
Scaling Symbolic Methods using Gradients for Neural Model Explanation
Authors:
Subham Sekhar Sahoo,
Subhashini Venugopalan,
Li Li,
Rishabh Singh,
Patrick Riley
Abstract:
Symbolic techniques based on Satisfiability Modulo Theory (SMT) solvers have been proposed for analyzing and verifying neural network properties, but their usage has been fairly limited owing to their poor scalability with larger networks. In this work, we propose a technique for combining gradient-based methods with symbolic techniques to scale such analyses and demonstrate its application for mo…
▽ More
Symbolic techniques based on Satisfiability Modulo Theory (SMT) solvers have been proposed for analyzing and verifying neural network properties, but their usage has been fairly limited owing to their poor scalability with larger networks. In this work, we propose a technique for combining gradient-based methods with symbolic techniques to scale such analyses and demonstrate its application for model explanation. In particular, we apply this technique to identify minimal regions in an input that are most relevant for a neural network's prediction. Our approach uses gradient information (based on Integrated Gradients) to focus on a subset of neurons in the first layer, which allows our technique to scale to large networks. The corresponding SMT constraints encode the minimal input mask discovery problem such that after masking the input, the activations of the selected neurons are still above a threshold. After solving for the minimal masks, our approach scores the mask regions to generate a relative ordering of the features within the mask. This produces a saliency map which explains "where a model is looking" when making a prediction. We evaluate our technique on three datasets - MNIST, ImageNet, and Beer Reviews, and demonstrate both quantitatively and qualitatively that the regions generated by our approach are sparser and achieve higher saliency scores compared to the gradient-based methods alone. Code and examples are at - https://github.com/google-research/google-research/tree/master/smug_saliency
△ Less
Submitted 5 May, 2021; v1 submitted 29 June, 2020;
originally announced June 2020.
-
It's easy to fool yourself: Case studies on identifying bias and confounding in bio-medical datasets
Authors:
Subhashini Venugopalan,
Arunachalam Narayanaswamy,
Samuel Yang,
Anton Geraschenko,
Scott Lipnick,
Nina Makhortova,
James Hawrot,
Christine Marques,
Joao Pereira,
Michael Brenner,
Lee Rubin,
Brian Wainger,
Marc Berndl
Abstract:
Confounding variables are a well known source of nuisance in biomedical studies. They present an even greater challenge when we combine them with black-box machine learning techniques that operate on raw data. This work presents two case studies. In one, we discovered biases arising from systematic errors in the data generation process. In the other, we found a spurious source of signal unrelated…
▽ More
Confounding variables are a well known source of nuisance in biomedical studies. They present an even greater challenge when we combine them with black-box machine learning techniques that operate on raw data. This work presents two case studies. In one, we discovered biases arising from systematic errors in the data generation process. In the other, we found a spurious source of signal unrelated to the prediction task at hand. In both cases, our prediction models performed well but under careful examination hidden confounders and biases were revealed. These are cautionary tales on the limits of using machine learning techniques on raw data from scientific experiments.
△ Less
Submitted 6 April, 2020; v1 submitted 12 December, 2019;
originally announced December 2019.
-
Predicting optical coherence tomography-derived diabetic macular edema grades from fundus photographs using deep learning
Authors:
Avinash Varadarajan,
Pinal Bavishi,
Paisan Raumviboonsuk,
Peranut Chotcomwongse,
Subhashini Venugopalan,
Arunachalam Narayanaswamy,
Jorge Cuadros,
Kuniyoshi Kanai,
George Bresnick,
Mongkol Tadarati,
Sukhum Silpa-archa,
Jirawut Limwattanayingyong,
Variya Nganthavee,
Joe Ledsam,
Pearse A Keane,
Greg S Corrado,
Lily Peng,
Dale R Webster
Abstract:
Diabetic eye disease is one of the fastest growing causes of preventable blindness. With the advent of anti-VEGF (vascular endothelial growth factor) therapies, it has become increasingly important to detect center-involved diabetic macular edema (ci-DME). However, center-involved diabetic macular edema is diagnosed using optical coherence tomography (OCT), which is not generally available at scre…
▽ More
Diabetic eye disease is one of the fastest growing causes of preventable blindness. With the advent of anti-VEGF (vascular endothelial growth factor) therapies, it has become increasingly important to detect center-involved diabetic macular edema (ci-DME). However, center-involved diabetic macular edema is diagnosed using optical coherence tomography (OCT), which is not generally available at screening sites because of cost and workflow constraints. Instead, screening programs rely on the detection of hard exudates in color fundus photographs as a proxy for DME, often resulting in high false positive or false negative calls. To improve the accuracy of DME screening, we trained a deep learning model to use color fundus photographs to predict ci-DME. Our model had an ROC-AUC of 0.89 (95% CI: 0.87-0.91), which corresponds to a sensitivity of 85% at a specificity of 80%. In comparison, three retinal specialists had similar sensitivities (82-85%), but only half the specificity (45-50%, p<0.001 for each comparison with model). The positive predictive value (PPV) of the model was 61% (95% CI: 56-66%), approximately double the 36-38% by the retinal specialists. In addition to predicting ci-DME, our model was able to detect the presence of intraretinal fluid with an AUC of 0.81 (95% CI: 0.81-0.86) and subretinal fluid with an AUC of 0.88 (95% CI: 0.85-0.91). The ability of deep learning algorithms to make clinically relevant predictions that generally require sophisticated 3D-imaging equipment from simple 2D images has broad relevance to many other applications in medical imaging.
△ Less
Submitted 31 July, 2019; v1 submitted 18 October, 2018;
originally announced October 2018.
