-
A Compact Model of Interface-Type Memristors Linking Physical and Device Properties
Authors:
T. F. Tiotto,
A. S. Goossens,
A. E. Dima,
C. Yakopcic,
T. Banerjee,
J. P. Borst,
N. A. Taatgen
Abstract:
Memristors are an electronic device whose resistance depends on the voltage history that has been applied to its two terminals. Despite its clear advantage as a computational element, a suitable transport model is lacking for the special class of interface-based memristors. Here, we adapt the widely-used Yakopcic compact model by including transport equations relevant to interface-type memristors.…
▽ More
Memristors are an electronic device whose resistance depends on the voltage history that has been applied to its two terminals. Despite its clear advantage as a computational element, a suitable transport model is lacking for the special class of interface-based memristors. Here, we adapt the widely-used Yakopcic compact model by including transport equations relevant to interface-type memristors. This model is able to reproduce the qualitative behaviour measured upon Nb-doped SrTiO$_3$ memristive devices. Our analysis demonstrates a direct correlation between the devices' characteristic parameters and those of our model. The model can clearly identify the charge transport mechanism in different resistive states thus facilitating evaluation of the relevant parameters pertaining to resistive switching in interface-based memristors. One clear application of our study is its ability to inform the design and fabrication of related memristive devices.
△ Less
Submitted 4 October, 2022;
originally announced October 2022.
-
High Speed Cognitive Domain Ontologies for Asset Allocation Using Loihi Spiking Neurons
Authors:
Chris Yakopcic,
Nayim Rahman,
Tanvir Atahary,
Tarek M. Taha,
Alex Beigh,
Scott Douglass
Abstract:
Cognitive agents are typically utilized in autonomous systems for automated decision making. These systems interact at real time with their environment and are generally heavily power constrained. Thus, there is a strong need for a real time agent running on a low power platform. The agent examined is the Cognitively Enhanced Complex Event Processing (CECEP) architecture. This is an autonomous dec…
▽ More
Cognitive agents are typically utilized in autonomous systems for automated decision making. These systems interact at real time with their environment and are generally heavily power constrained. Thus, there is a strong need for a real time agent running on a low power platform. The agent examined is the Cognitively Enhanced Complex Event Processing (CECEP) architecture. This is an autonomous decision support tool that reasons like humans and enables enhanced agent-based decision-making. It has applications in a large variety of domains including autonomous systems, operations research, intelligence analysis, and data mining. One of the key components of CECEP is the mining of knowledge from a repository described as a Cognitive Domain Ontology (CDO). One problem that is often tasked to CDOs is asset allocation. Given the number of possible solutions in this allocation problem, determining the optimal solution via CDO can be very time consuming. In this work we show that a grid of isolated spiking neurons is capable of generating solutions to this problem very quickly, although some degree of approximation is required to achieve the speedup. However, the approximate spiking approach presented in this work was able to complete all allocation simulations with greater than 99.9% accuracy. To show the feasibility of low power implementation, this algorithm was executed using the Intel Loihi manycore neuromorphic processor. Given the vast increase in speed (greater than 1000 times in larger allocation problems), as well as the reduction in computational requirements, the presented algorithm is ideal for moving asset allocation to low power, portable, embedded hardware.
△ Less
Submitted 28 June, 2019;
originally announced June 2019.
-
Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural Network
Authors:
Md Zahangir Alom,
Chris Yakopcic,
Tarek M. Taha,
Vijayan K. Asari
Abstract:
The Deep Convolutional Neural Network (DCNN) is one of the most powerful and successful deep learning approaches. DCNNs have already provided superior performance in different modalities of medical imaging including breast cancer classification, segmentation, and detection. Breast cancer is one of the most common and dangerous cancers impacting women worldwide. In this paper, we have proposed a me…
▽ More
The Deep Convolutional Neural Network (DCNN) is one of the most powerful and successful deep learning approaches. DCNNs have already provided superior performance in different modalities of medical imaging including breast cancer classification, segmentation, and detection. Breast cancer is one of the most common and dangerous cancers impacting women worldwide. In this paper, we have proposed a method for breast cancer classification with the Inception Recurrent Residual Convolutional Neural Network (IRRCNN) model. The IRRCNN is a powerful DCNN model that combines the strength of the Inception Network (Inception-v4), the Residual Network (ResNet), and the Recurrent Convolutional Neural Network (RCNN). The IRRCNN shows superior performance against equivalent Inception Networks, Residual Networks, and RCNNs for object recognition tasks. In this paper, the IRRCNN approach is applied for breast cancer classification on two publicly available datasets including BreakHis and Breast Cancer Classification Challenge 2015. The experimental results are compared against the existing machine learning and deep learning-based approaches with respect to image-based, patch-based, image-level, and patient-level classification. The IRRCNN model provides superior classification performance in terms of sensitivity, Area Under the Curve (AUC), the ROC curve, and global accuracy compared to existing approaches for both datasets.
△ Less
Submitted 10 November, 2018;
originally announced November 2018.
-
Microscopic Nuclei Classification, Segmentation and Detection with improved Deep Convolutional Neural Network (DCNN) Approaches
Authors:
Md Zahangir Alom,
Chris Yakopcic,
Tarek M. Taha,
Vijayan K. Asari
Abstract:
Due to cellular heterogeneity, cell nuclei classification, segmentation, and detection from pathological images are challenging tasks. In the last few years, Deep Convolutional Neural Networks (DCNN) approaches have been shown state-of-the-art (SOTA) performance on histopathological imaging in different studies. In this work, we have proposed different advanced DCNN models and evaluated for nuclei…
▽ More
Due to cellular heterogeneity, cell nuclei classification, segmentation, and detection from pathological images are challenging tasks. In the last few years, Deep Convolutional Neural Networks (DCNN) approaches have been shown state-of-the-art (SOTA) performance on histopathological imaging in different studies. In this work, we have proposed different advanced DCNN models and evaluated for nuclei classification, segmentation, and detection. First, the Densely Connected Recurrent Convolutional Network (DCRN) model is used for nuclei classification. Second, Recurrent Residual U-Net (R2U-Net) is applied for nuclei segmentation. Third, the R2U-Net regression model which is named UD-Net is used for nuclei detection from pathological images. The experiments are conducted with different datasets including Routine Colon Cancer(RCC) classification and detection dataset, and Nuclei Segmentation Challenge 2018 dataset. The experimental results show that the proposed DCNN models provide superior performance compared to the existing approaches for nuclei classification, segmentation, and detection tasks. The results are evaluated with different performance metrics including precision, recall, Dice Coefficient (DC), Means Squared Errors (MSE), F1-score, and overall accuracy. We have achieved around 3.4% and 4.5% better F-1 score for nuclei classification and detection tasks compared to recently published DCNN based method. In addition, R2U-Net shows around 92.15% testing accuracy in term of DC. These improved methods will help for pathological practices for better quantitative analysis of nuclei in Whole Slide Images(WSI) which ultimately will help for better understanding of different types of cancer in clinical workflow.
△ Less
Submitted 8 November, 2018;
originally announced November 2018.
-
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches
Authors:
Md Zahangir Alom,
Tarek M. Taha,
Christopher Yakopcic,
Stefan Westberg,
Paheding Sidike,
Mst Shamima Nasrin,
Brian C Van Esesn,
Abdul A S. Awwal,
Vijayan K. Asari
Abstract:
Deep learning has demonstrated tremendous success in variety of application domains in the past few years. This new field of machine learning has been growing rapidly and applied in most of the application domains with some new modalities of applications, which helps to open new opportunity. There are different methods have been proposed on different category of learning approaches, which includes…
▽ More
Deep learning has demonstrated tremendous success in variety of application domains in the past few years. This new field of machine learning has been growing rapidly and applied in most of the application domains with some new modalities of applications, which helps to open new opportunity. There are different methods have been proposed on different category of learning approaches, which includes supervised, semi-supervised and un-supervised learning. The experimental results show state-of-the-art performance of deep learning over traditional machine learning approaches in the field of Image Processing, Computer Vision, Speech Recognition, Machine Translation, Art, Medical imaging, Medical information processing, Robotics and control, Bio-informatics, Natural Language Processing (NLP), Cyber security, and many more. This report presents a brief survey on development of DL approaches, including Deep Neural Network (DNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) including Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). In addition, we have included recent development of proposed advanced variant DL techniques based on the mentioned DL approaches. Furthermore, DL approaches have explored and evaluated in different application domains are also included in this survey. We have also comprised recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys have published on Deep Learning in Neural Networks [1, 38] and a survey on RL [234]. However, those papers have not discussed the individual advanced techniques for training large scale deep learning models and the recently developed method of generative models [1].
△ Less
Submitted 12 September, 2018; v1 submitted 3 March, 2018;
originally announced March 2018.
-
Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
Authors:
Md Zahangir Alom,
Mahmudul Hasan,
Chris Yakopcic,
Tarek M. Taha,
Vijayan K. Asari
Abstract:
Deep learning (DL) based semantic segmentation methods have been providing state-of-the-art performance in the last few years. More specifically, these techniques have been successfully applied to medical image classification, segmentation, and detection tasks. One deep learning technique, U-Net, has become one of the most popular for these applications. In this paper, we propose a Recurrent Convo…
▽ More
Deep learning (DL) based semantic segmentation methods have been providing state-of-the-art performance in the last few years. More specifically, these techniques have been successfully applied to medical image classification, segmentation, and detection tasks. One deep learning technique, U-Net, has become one of the most popular for these applications. In this paper, we propose a Recurrent Convolutional Neural Network (RCNN) based on U-Net as well as a Recurrent Residual Convolutional Neural Network (RRCNN) based on U-Net models, which are named RU-Net and R2U-Net respectively. The proposed models utilize the power of U-Net, Residual Network, as well as RCNN. There are several advantages of these proposed architectures for segmentation tasks. First, a residual unit helps when training deep architecture. Second, feature accumulation with recurrent residual convolutional layers ensures better feature representation for segmentation tasks. Third, it allows us to design better U-Net architecture with same number of network parameters with better performance for medical image segmentation. The proposed models are tested on three benchmark datasets such as blood vessel segmentation in retina images, skin cancer segmentation, and lung lesion segmentation. The experimental results show superior performance on segmentation tasks compared to equivalent models including U-Net and residual U-Net (ResU-Net).
△ Less
Submitted 29 May, 2018; v1 submitted 19 February, 2018;
originally announced February 2018.
-
Deep Versus Wide Convolutional Neural Networks for Object Recognition on Neuromorphic System
Authors:
Md Zahangir Alom,
Theodore Josue,
Md Nayim Rahman,
Will Mitchell,
Chris Yakopcic,
Tarek M. Taha
Abstract:
In the last decade, special purpose computing systems, such as Neuromorphic computing, have become very popular in the field of computer vision and machine learning for classification tasks. In 2015, IBM's released the TrueNorth Neuromorphic system, kick-starting a new era of Neuromorphic computing. Alternatively, Deep Learning approaches such as Deep Convolutional Neural Networks (DCNN) show almo…
▽ More
In the last decade, special purpose computing systems, such as Neuromorphic computing, have become very popular in the field of computer vision and machine learning for classification tasks. In 2015, IBM's released the TrueNorth Neuromorphic system, kick-starting a new era of Neuromorphic computing. Alternatively, Deep Learning approaches such as Deep Convolutional Neural Networks (DCNN) show almost human-level accuracies for detection and classification tasks. IBM's 2016 release of a deep learning framework for DCNNs, called Energy Efficient Deep Neuromorphic Networks (Eedn). Eedn shows promise for delivering high accuracies across a number of different benchmarks, while consuming very low power, using IBM's TrueNorth chip. However, there are many things that remained undiscovered using the Eedn framework for classification tasks on a Neuromorphic system. In this paper, we have empirically evaluated the performance of different DCNN architectures implemented within the Eedn framework. The goal of this work was discover the most efficient way to implement DCNN models for object classification tasks using the TrueNorth system. We performed our experiments using benchmark data sets such as MNIST, COIL 20, and COIL 100. The experimental results show very promising classification accuracies with very low power consumption on IBM's NS1e Neurosynaptic system. The results show that for datasets with large numbers of classes, wider networks perform better when compared to deep networks comprised of nearly the same core complexity on IBM's TrueNorth system.
△ Less
Submitted 7 February, 2018;
originally announced February 2018.
-
Improved Inception-Residual Convolutional Neural Network for Object Recognition
Authors:
Md Zahangir Alom,
Mahmudul Hasan,
Chris Yakopcic,
Tarek M. Taha,
Vijayan K. Asari
Abstract:
Machine learning and computer vision have driven many of the greatest advances in the modeling of Deep Convolutional Neural Networks (DCNNs). Nowadays, most of the research has been focused on improving recognition accuracy with better DCNN models and learning approaches. The recurrent convolutional approach is not applied very much, other than in a few DCNN architectures. On the other hand, Incep…
▽ More
Machine learning and computer vision have driven many of the greatest advances in the modeling of Deep Convolutional Neural Networks (DCNNs). Nowadays, most of the research has been focused on improving recognition accuracy with better DCNN models and learning approaches. The recurrent convolutional approach is not applied very much, other than in a few DCNN architectures. On the other hand, Inception-v4 and Residual networks have promptly become popular among computer the vision community. In this paper, we introduce a new DCNN model called the Inception Recurrent Residual Convolutional Neural Network (IRRCNN), which utilizes the power of the Recurrent Convolutional Neural Network (RCNN), the Inception network, and the Residual network. This approach improves the recognition accuracy of the Inception-residual network with same number of network parameters. In addition, this proposed architecture generalizes the Inception network, the RCNN, and the Residual network with significantly improved training accuracy. We have empirically evaluated the performance of the IRRCNN model on different benchmarks including CIFAR-10, CIFAR-100, TinyImageNet-200, and CU3D-100. The experimental results show higher recognition accuracy against most of the popular DCNN models including the RCNN. We have also investigated the performance of the IRRCNN approach against the Equivalent Inception Network (EIN) and the Equivalent Inception Residual Network (EIRN) counterpart on the CIFAR-100 dataset. We report around 4.53%, 4.49% and 3.56% improvement in classification accuracy compared with the RCNN, EIN, and EIRN on the CIFAR-100 dataset respectively. Furthermore, the experiment has been conducted on the TinyImageNet-200 and CU3D-100 datasets where the IRRCNN provides better testing accuracy compared to the Inception Recurrent CNN (IRCNN), the EIN, and the EIRN.
△ Less
Submitted 28 December, 2017;
originally announced December 2017.
-
Inception Recurrent Convolutional Neural Network for Object Recognition
Authors:
Md Zahangir Alom,
Mahmudul Hasan,
Chris Yakopcic,
Tarek M. Taha
Abstract:
Deep convolutional neural networks (DCNNs) are an influential tool for solving various problems in the machine learning and computer vision fields. In this paper, we introduce a new deep learning model called an Inception- Recurrent Convolutional Neural Network (IRCNN), which utilizes the power of an inception network combined with recurrent layers in DCNN architecture. We have empirically evaluat…
▽ More
Deep convolutional neural networks (DCNNs) are an influential tool for solving various problems in the machine learning and computer vision fields. In this paper, we introduce a new deep learning model called an Inception- Recurrent Convolutional Neural Network (IRCNN), which utilizes the power of an inception network combined with recurrent layers in DCNN architecture. We have empirically evaluated the recognition performance of the proposed IRCNN model using different benchmark datasets such as MNIST, CIFAR-10, CIFAR- 100, and SVHN. Experimental results show similar or higher recognition accuracy when compared to most of the popular DCNNs including the RCNN. Furthermore, we have investigated IRCNN performance against equivalent Inception Networks and Inception-Residual Networks using the CIFAR-100 dataset. We report about 3.5%, 3.47% and 2.54% improvement in classification accuracy when compared to the RCNN, equivalent Inception Networks, and Inception- Residual Networks on the augmented CIFAR- 100 dataset respectively.
△ Less
Submitted 25 April, 2017;
originally announced April 2017.
-
High Throughput Neural Network based Embedded Streaming Multicore Processors
Authors:
Raqibul Hasan,
Tarek M. Taha,
Chris Yakopcic,
David J. Mountain
Abstract:
With power consumption becoming a critical processor design issue, specialized architectures for low power processing are becoming popular. Several studies have shown that neural networks can be used for signal processing and pattern recognition applications. This study examines the design of memristor based multicore neural processors that would be used primarily to process data directly from sen…
▽ More
With power consumption becoming a critical processor design issue, specialized architectures for low power processing are becoming popular. Several studies have shown that neural networks can be used for signal processing and pattern recognition applications. This study examines the design of memristor based multicore neural processors that would be used primarily to process data directly from sensors. Additionally, we have examined the design of SRAM based neural processors for the same task. Full system evaluation of the multicore processors based on these specialized cores were performed taking I/O and routing circuits into consideration. The area and power benefits were compared with traditional multicore RISC processors. Our results show that the memristor based architectures can provide an energy efficiency between three and five orders of magnitude greater than that of RISC processors for the benchmarks examined.
△ Less
Submitted 14 June, 2016;
originally announced June 2016.
-
Hybrid Crossbar Architecture for a Memristor Based Memory
Authors:
Chris Yakopcic,
Tarek M. Taha
Abstract:
This paper describes a new memristor crossbar architecture that is proposed for use in a high density cache design. This design has less than 10% of the write energy consumption than a simple memristor crossbar. Also, it has up to 4 times the bit density of an STT-MRAM system and up to 11 times the bit density of an SRAM architecture. The proposed architecture is analyzed using a detailed SPICE an…
▽ More
This paper describes a new memristor crossbar architecture that is proposed for use in a high density cache design. This design has less than 10% of the write energy consumption than a simple memristor crossbar. Also, it has up to 4 times the bit density of an STT-MRAM system and up to 11 times the bit density of an SRAM architecture. The proposed architecture is analyzed using a detailed SPICE analysis that accounts for the resistance of the wires in the memristor structure. Additionally, the memristor model used in this work has been matched to specific device characterization data to provide accurate results in terms of energy, area, and timing.
△ Less
Submitted 9 April, 2013; v1 submitted 26 February, 2013;
originally announced February 2013.
