-
The curious case of the test set AUROC
Authors:
Michael Roberts,
Alon Hazan,
Sören Dittmer,
James H. F. Rudd,
Carola-Bibiane Schönlieb
Abstract:
Whilst the size and complexity of ML models have rapidly and significantly increased over the past decade, the methods for assessing their performance have not kept pace. In particular, among the many potential performance metrics, the ML community stubbornly continues to use (a) the area under the receiver operating characteristic curve (AUROC) for a validation and test cohort (distinct from trai…
▽ More
Whilst the size and complexity of ML models have rapidly and significantly increased over the past decade, the methods for assessing their performance have not kept pace. In particular, among the many potential performance metrics, the ML community stubbornly continues to use (a) the area under the receiver operating characteristic curve (AUROC) for a validation and test cohort (distinct from training data) or (b) the sensitivity and specificity for the test data at an optimal threshold determined from the validation ROC. However, we argue that considering scores derived from the test ROC curve alone gives only a narrow insight into how a model performs and its ability to generalise.
△ Less
Submitted 19 December, 2023;
originally announced December 2023.
-
Ti$_3$C$_2$T$_x$ MXene Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks
Authors:
Adir Hazan,
Barak Ratzker,
Danzhen Zhang,
Aviad Katiyi,
Nachum Frage,
Maxim Sokol,
Yury Gogotsi,
Alina Karabchevsky
Abstract:
Neural networks are one of the first major milestones in developing artificial intelligence systems. The utilisation of integrated photonics in neural networks offers a promising alternative approach to microelectronic and hybrid optical-electronic implementations due to improvements in computational speed and low energy consumption in machine-learning tasks. However, at present, most of the neura…
▽ More
Neural networks are one of the first major milestones in developing artificial intelligence systems. The utilisation of integrated photonics in neural networks offers a promising alternative approach to microelectronic and hybrid optical-electronic implementations due to improvements in computational speed and low energy consumption in machine-learning tasks. However, at present, most of the neural network hardware systems are still electronic-based due to a lack of optical realisation of the nonlinear activation function. Here, we experimentally demonstrate two novel approaches for implementing an all-optical neural nonlinear activation function based on utilising unique light-matter interactions in 2D Ti$_3$C$_2$T$_x$ (MXene) in the infrared (IR) range in two configurations: 1) a saturable absorber made of MXene thin film, and 2) a silicon waveguide with MXene flakes overlayer. These configurations may serve as nonlinear units in photonic neural networks, while their nonlinear transfer function can be flexibly designed to optimise the performance of different neuromorphic tasks, depending on the operating wavelength. The proposed configurations are reconfigurable and can therefore be adjusted for various applications without the need to modify the physical structure. We confirm the capability and feasibility of the obtained results in machine-learning applications via an Modified National Institute of Standards and Technology (MNIST) handwritten digit classifications task, with near 99% accuracy. Our developed concept for an all-optical neuron is expected to constitute a major step towards the realization of all-optically implemented deep neural networks.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
A maximum entropy network reconstruction of macroeconomic models
Authors:
Aurélien Hazan
Abstract:
In this article the problem of reconstructing the pattern of connection between agents from partial empirical data in a macro-economic model is addressed, given a set of behavioral equations. This systemic point of view puts the focus on distributional and network effects, rather than time-dependence. Using the theory of complex networks we compare several models to reconstruct both the topology…
▽ More
In this article the problem of reconstructing the pattern of connection between agents from partial empirical data in a macro-economic model is addressed, given a set of behavioral equations. This systemic point of view puts the focus on distributional and network effects, rather than time-dependence. Using the theory of complex networks we compare several models to reconstruct both the topology and the flows of money of the different types of monetary transactions, while imposing a series of constraints related to national accounts, and to empirical network sparsity. Some properties of reconstructed networks are compared with their empirical counterpart.
△ Less
Submitted 7 December, 2018; v1 submitted 27 July, 2018;
originally announced July 2018.
-
Mammography Dual View Mass Correspondence
Authors:
Shaked Perek,
Alon Hazan,
Ella Barkan,
Ayelet Akselrod-Ballin
Abstract:
Standard breast cancer screening involves the acquisition of two mammography X-ray projections for each breast. Typically, a comparison of both views supports the challenging task of tumor detection and localization. We introduce a deep learning, patch-based Siamese network for lesion matching in dual-view mammography. Our locally-fitted approach generates a joint patch pair representation and com…
▽ More
Standard breast cancer screening involves the acquisition of two mammography X-ray projections for each breast. Typically, a comparison of both views supports the challenging task of tumor detection and localization. We introduce a deep learning, patch-based Siamese network for lesion matching in dual-view mammography. Our locally-fitted approach generates a joint patch pair representation and comparison with a shared configuration between the two views. We performed a comprehensive set of experiments with the network on standard datasets, among them the large Digital Database for Screening Mammography (DDSM). We analyzed the effect of transfer learning with the network between different types of datasets and compared the network-based matching to using Euclidean distance by template matching. Finally, we evaluated the contribution of the matching network in a full detection pipeline. Experimental results demonstrate the promise of improved detection accuracy using our approach.
△ Less
Submitted 2 July, 2018;
originally announced July 2018.
-
AdapterNet - learning input transformation for domain adaptation
Authors:
Alon Hazan,
Yoel Shoshan,
Daniel Khapun,
Roy Aladjem,
Vadim Ratner
Abstract:
Deep neural networks have demonstrated impressive performance in various machine learning tasks. However, they are notoriously sensitive to changes in data distribution. Often, even a slight change in the distribution can lead to drastic performance reduction. Artificially augmenting the data may help to some extent, but in most cases, fails to achieve model invariance to the data distribution. So…
▽ More
Deep neural networks have demonstrated impressive performance in various machine learning tasks. However, they are notoriously sensitive to changes in data distribution. Often, even a slight change in the distribution can lead to drastic performance reduction. Artificially augmenting the data may help to some extent, but in most cases, fails to achieve model invariance to the data distribution. Some examples where this sub-class of domain adaptation can be valuable are various imaging modalities such as thermal imaging, X-ray, ultrasound, and MRI, where changes in acquisition parameters or acquisition device manufacturer will result in a different representation of the same input. Our work shows that standard fine-tuning fails to adapt the model in certain important cases. We propose a novel method of adapting to a new data source, and demonstrate near perfect adaptation on a customized ImageNet benchmark. Moreover, our method does not require any samples from the original data set, it is completely explainable and can be tailored to the task.
△ Less
Submitted 15 November, 2018; v1 submitted 29 May, 2018;
originally announced May 2018.
-
Learning an attention model in an artificial visual system
Authors:
Alon Hazan,
Yuval Harel,
Ron Meir
Abstract:
The Human visual perception of the world is of a large fixed image that is highly detailed and sharp. However, receptor density in the retina is not uniform: a small central region called the fovea is very dense and exhibits high resolution, whereas a peripheral region around it has much lower spatial resolution. Thus, contrary to our perception, we are only able to observe a very small region aro…
▽ More
The Human visual perception of the world is of a large fixed image that is highly detailed and sharp. However, receptor density in the retina is not uniform: a small central region called the fovea is very dense and exhibits high resolution, whereas a peripheral region around it has much lower spatial resolution. Thus, contrary to our perception, we are only able to observe a very small region around the line of sight with high resolution. The perception of a complete and stable view is aided by an attention mechanism that directs the eyes to the numerous points of interest within the scene. The eyes move between these targets in quick, unconscious movements, known as "saccades". Once a target is centered at the fovea, the eyes fixate for a fraction of a second while the visual system extracts the necessary information. An artificial visual system was built based on a fully recurrent neural network set within a reinforcement learning protocol, and learned to attend to regions of interest while solving a classification task. The model is consistent with several experimentally observed phenomena, and suggests novel predictions.
△ Less
Submitted 24 January, 2017;
originally announced January 2017.
