-
Implementation of Multilayer Perceptron Network with Highly Uniform Passive Memristive Crossbar Circuits
Authors:
F. Merrikh Bayat,
M. Prezioso,
B. Chakrabarti,
I. Kataeva,
D. Strukov
Abstract:
The progress in the field of neural computation hinges on the use of hardware more efficient than the conventional microprocessors. Recent works have shown that mixed-signal integrated memristive circuits, especially their passive ('0T1R') variety, may increase the neuromorphic network performance dramatically, leaving far behind their digital counterparts. The major obstacle, however, is relative…
▽ More
The progress in the field of neural computation hinges on the use of hardware more efficient than the conventional microprocessors. Recent works have shown that mixed-signal integrated memristive circuits, especially their passive ('0T1R') variety, may increase the neuromorphic network performance dramatically, leaving far behind their digital counterparts. The major obstacle, however, is relatively immature memristor technology so that only limited functionality has been demonstrated to date. Here we experimentally demonstrate operation of one-hidden layer perceptron classifier entirely in the mixed-signal integrated hardware, comprised of two passive 20x20 metal-oxide memristive crossbar arrays, board-integrated with discrete CMOS components. The demonstrated multilayer perceptron network, whose complexity is almost 10x higher as compared to previously reported functional neuromorphic classifiers based on passive memristive circuits, achieves classification fidelity within 3 percent of that obtained in simulations, when using ex-situ training approach. The successful demonstration was facilitated by improvements in fabrication technology of memristors, specifically by lowering variations in their I-V characteristics.
△ Less
Submitted 4 December, 2017;
originally announced December 2017.
-
Highly-Secure Physically Unclonable Cryptographic Primitives Using Nonlinear Conductance and Analog State Tuning in Memristive Crossbar Arrays
Authors:
Hussein Nili,
Gina C. Adam,
Mirko Prezioso,
Jeeson Kim,
Farnood Merrikh-Bayat,
Omid Kavehei,
Dmitri B. Strukov
Abstract:
The rapidly expanding hardware-intrinsic security primitives are aimed at addressing significant security challenges of a massively interconnected world in the age of information technology. The main idea of such primitives is to employ instance-specific process-induced variations in electronic hardware as a source of cryptographic data. Among the emergent technologies, memristive devices provide…
▽ More
The rapidly expanding hardware-intrinsic security primitives are aimed at addressing significant security challenges of a massively interconnected world in the age of information technology. The main idea of such primitives is to employ instance-specific process-induced variations in electronic hardware as a source of cryptographic data. Among the emergent technologies, memristive devices provide unique opportunities for security applications due to the underlying stochasticity in their operation. Herein, we report a prototype of a robust, dense, and reconfigurable physical unclonable function primitives based on the three-dimensional passive metal-oxide memristive crossbar circuits, by making positive use of process-induced variations in the devices' nonlinear I-Vs and their analog tuning. We first characterize security metrics for a basic building block of the security primitives based on a two layer stack with monolithically integrated 10x10 250-nm half-pitch memristive crossbar circuits. The experimental results show that the average uniformity and diffusivity, measured on a random sample of 6,000 64-bit responses, out of ~697,000 total, is close to ideal 50% with 5% standard deviation for both metrics. The uniqueness, which was evaluated on a smaller sample by readjusting conductances of crosspoint devices within the same crossbar, is also close to the ideal 50% +/- 1%, while the smallest bit error rate, i.e. reciprocal of reliability, measured over 30-day window under +/-20% power supply variations, was ~ 1.5% +/- 1%. We then utilize multiple instances of the basic block to demonstrate physically unclonable functional primitive with 10-bit hidden challenge generation that encodes more than 10^19 challenge response pairs and has comparable uniformity, diffusiveness, and bit error rate.
△ Less
Submitted 23 November, 2016;
originally announced November 2016.
-
Advancing Memristive Analog Neuromorphic Networks: Increasing Complexity, and Coping with Imperfect Hardware Components
Authors:
F. Merrikh Bayat,
M. Prezioso,
B. Chakrabarti,
I. Kataeva,
D. B. Strukov
Abstract:
We experimentally demonstrate classification of 4x4 binary images into 4 classes, using a 3-layer mixed-signal neuromorphic network ("MLP perceptron"), based on two passive 20x20 memristive crossbar arrays, board-integrated with discrete CMOS components. The network features 10 hidden-layer and 4 output-layer analog CMOS neurons and 428 metal-oxide memristors, i.e. is almost an order of magnitude…
▽ More
We experimentally demonstrate classification of 4x4 binary images into 4 classes, using a 3-layer mixed-signal neuromorphic network ("MLP perceptron"), based on two passive 20x20 memristive crossbar arrays, board-integrated with discrete CMOS components. The network features 10 hidden-layer and 4 output-layer analog CMOS neurons and 428 metal-oxide memristors, i.e. is almost an order of magnitude more complex than any previously reported functional memristor circuit. Moreover, the inference operation of this classifier is performed entirely in the integrated hardware. To deal with larger crossbar arrays, we have developed a semi-automatic approach to their forming and testing, and compared several memristor training schemes for coping with imperfect behavior of these devices, as well as with variability of analog CMOS neurons. The effectiveness of the proposed schemes for defect and variation tolerance was verified experimentally using the implemented network and, additionally, by modeling the operation of a larger network, with 300 hidden-layer neurons, on the MNIST benchmark. Finally, we propose a simple modification of the implemented memristor-based vector-by-matrix multiplier to allow its operation in a wider temperature range.
△ Less
Submitted 10 November, 2016;
originally announced November 2016.
-
Temperature-Insensitive Analog Vector-by-Matrix Multiplier Based on 55 nm NOR Flash Memory Cells
Authors:
X. Guo,
F. Merrikh Bayat,
M. Prezioso,
Y. Chen,
B. Nguyen,
N. Do,
D. B. Strukov
Abstract:
We have fabricated and successfully tested an analog vector-by-matrix multiplier, based on redesigned 10x12 arrays of 55 nm commercial NOR flash memory cells. The modified arrays enable high-precision individual analog tuning of each cell, with sub-1% accuracy, while keeping the highly optimized cells, with their long-term state retention, intact. The array has an area of 0.33 um^2 per cell, and i…
▽ More
We have fabricated and successfully tested an analog vector-by-matrix multiplier, based on redesigned 10x12 arrays of 55 nm commercial NOR flash memory cells. The modified arrays enable high-precision individual analog tuning of each cell, with sub-1% accuracy, while keeping the highly optimized cells, with their long-term state retention, intact. The array has an area of 0.33 um^2 per cell, and is at least one order of magnitude more dense than the reported prior implementations of nonvolatile analog memories. The demonstrated vector-by-vector multiplier, using gate coupling to additional periphery cells, has ~2% precision, limited by the aggregate effect of cell noise, retention, mismatch, process variations, tuning precision, and capacitive crosstalk. A differential version of the multiplier has allowed us to demonstrate sub-3% temperature drift of the output signal in the range between 25C and 85C.
△ Less
Submitted 10 November, 2016;
originally announced November 2016.
-
Sub-1-us, Sub-20-nJ Pattern Classification in a Mixed-Signal Circuit Based on Embedded 180-nm Floating-Gate Memory Cell Arrays
Authors:
F. Merrikh Bayat,
X. Guo,
M. Klachko,
M. Prezioso,
K. K. Likharev,
D. B. Strukov
Abstract:
We have designed, fabricated, and successfully tested a prototype mixed-signal, 28x28-binary-input, 10-output, 3-layer neuromorphic network ("MLP perceptron"). It is based on embedded nonvolatile floating-gate cell arrays redesigned from a commercial 180-nm NOR flash memory. The arrays allow precise (~1%) individual tuning of all memory cells, having long-term analog-level retention and low noise.…
▽ More
We have designed, fabricated, and successfully tested a prototype mixed-signal, 28x28-binary-input, 10-output, 3-layer neuromorphic network ("MLP perceptron"). It is based on embedded nonvolatile floating-gate cell arrays redesigned from a commercial 180-nm NOR flash memory. The arrays allow precise (~1%) individual tuning of all memory cells, having long-term analog-level retention and low noise. Each array performs a very fast and energy-efficient analog vector-by-matrix multiplication, which is the bottleneck for signal propagation in most neuromorphic networks. All functional components of the prototype circuit, including 2 synaptic arrays with 101,780 floating-gate synaptic cells, 74 analog neurons, and the peripheral circuitry for weight adjustment and I/O operations, have a total area below 1 mm^2. Its testing on the common MNIST benchmark set (at this stage, with a relatively low weight import precision) has shown a classification fidelity of 94.65%, close to the 96.2% obtained in simulation. The classification of one pattern takes less than 1 us time and ~20 nJ energy - both numbers much better than for digital implementations of the same task. Estimates show that this performance may be further improved using a better neuron design and a more advanced memory technology, leading to a >10^2 advantage in speed and a >10^4 advantage in energy efficiency over the state-of-the-art purely digital (GPU and custom) circuits, at classification of large, complex patterns.
△ Less
Submitted 10 October, 2016; v1 submitted 6 October, 2016;
originally announced October 2016.
-
Three-Dimensional Stateful Material Implication Logic
Authors:
Gina C. Adam,
Brian D. Hoskins,
Mirko Prezioso,
Dmitri B. Strukov
Abstract:
Monolithic three-dimensional integration of memory and logic circuits could dramatically improve performance and energy efficiency of computing systems. Some conventional and emerging memories are suitable for vertical integration, including highly scalable metal-oxide resistive switching devices (memristors), yet integration of logic circuits proves to be much more challenging. Here we demonstrat…
▽ More
Monolithic three-dimensional integration of memory and logic circuits could dramatically improve performance and energy efficiency of computing systems. Some conventional and emerging memories are suitable for vertical integration, including highly scalable metal-oxide resistive switching devices (memristors), yet integration of logic circuits proves to be much more challenging. Here we demonstrate memory and logic functionality in a monolithic three-dimensional circuit by adapting recently proposed memristor-based stateful material implication logic. Though such logic has been already implemented with a variety of memory devices, prohibitively large device variability in the most prospective memristor-based circuits has limited experimental demonstrations to simple gates and just a few cycles of operations. By developing a low-temperature, low-variability fabrication process, and modifying the original circuit to increase its robustness to device imperfections, we experimentally show, for the first time, reliable multi-cycle multi-gate material implication logic operation within a three-dimensional stack of monolithically integrated memristors. The direct data manipulation in three dimensions enables extremely compact and high-throughput logic-in-memory computing and, remarkably, presents a viable solution for the Feynman grand challenge of implementing an 8-bit adder at the nanoscale.
△ Less
Submitted 9 September, 2015;
originally announced September 2015.
-
Self-Adaptive Spike-Time-Dependent Plasticity of Metal-Oxide Memristors
Authors:
M. Prezioso,
F. Merrikh-Bayat,
B. Hoskins,
K. Likharev,
D. Strukov
Abstract:
Metal-oxide memristors have emerged as promising candidates for hardware implementation of artificial synapses - the key components of high-performance, analog neuromorphic networks - due to their excellent scaling prospects. Since some advanced cognitive tasks require spiking neuromorphic networks, which explicitly model individual neural pulses (spikes) in biological neural systems, it is crucia…
▽ More
Metal-oxide memristors have emerged as promising candidates for hardware implementation of artificial synapses - the key components of high-performance, analog neuromorphic networks - due to their excellent scaling prospects. Since some advanced cognitive tasks require spiking neuromorphic networks, which explicitly model individual neural pulses (spikes) in biological neural systems, it is crucial for memristive synapses to support the spike-time-dependent plasticity (STDP), which is believed to be the primary mechanism of Hebbian adaptation. A major challenge for the STDP implementation is that, in contrast to some simplistic models of the plasticity, the elementary change of a synaptic weight in an artificial hardware synapse depends not only on the pre-synaptic and post-synaptic signals, but also on the initial weight (memristor's conductance) value. Here we experimentally demonstrate, for the first time, STDP protocols that ensure self-adaptation of the average memristor conductance, making the plasticity stable, i.e. insensitive to the initial state of the devices. The experiments have been carried out with 200-nm Al2O3/TiO2-x memristors integrated into 12x12 crossbars. The experimentally observed self-adaptive STDP behavior has been complemented with numerical modeling of weight dynamics in a simple system with a leaky-integrate-and-fire neuron with a random spike-train input, using a compact model of memristor plasticity, fitted for quantitatively correct description of our memristors.
△ Less
Submitted 20 May, 2015;
originally announced May 2015.
-
Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
Authors:
Mirko Prezioso,
Farnood Merrikh-Bayat,
Brian Hoskins,
Gina Adam,
Konstantin K. Likharev,
Dmitri B. Strukov
Abstract:
Despite all the progress of semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. One of the most prospective candidates to provide comparable complexity, while operating much faster and with manageable power dissipation, are so-called…
▽ More
Despite all the progress of semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. One of the most prospective candidates to provide comparable complexity, while operating much faster and with manageable power dissipation, are so-called CrossNets based on hybrid CMOS/memristor circuits. In these circuits, the usual CMOS stack is augmented with one or several crossbar layers, with adjustable two-terminal memristors at each crosspoint. Recently, there was a significant progress in improvement of technology of fabrication of such memristive crossbars and their integration with CMOS circuits, including first demonstrations of their vertical integration. Separately, there have been several demonstrations of discrete memristors as artificial synapses for neuromorphic networks. Very recently such experiments were extended to crossbar arrays of phase-change memristive devices. The adjustment of such devices, however, requires an additional transistor at each crosspoint, and hence the prospects of their scaling are less impressive than those of metal-oxide memristors, whose nonlinear I-V curves enable transistor-free operation. Here we report the first experimental implementation of a transistor-free metal-oxide memristor crossbar with device variability lowered sufficiently to demonstrate a successful operation of a simple integrated neural network, a single layer-perceptron. The network could be taught in situ using a coarse-grain variety of the delta-rule algorithm to perform the perfect classification of 3x3-pixel black/white images into 3 classes. We believe that this demonstration is an important step towards the implementation of much larger and more complex memristive neuromorphic networks.
△ Less
Submitted 1 December, 2014;
originally announced December 2014.
