-
Roadmap to Neuromorphic Computing with Emerging Technologies
Authors:
Adnan Mehonic,
Daniele Ielmini,
Kaushik Roy,
Onur Mutlu,
Shahar Kvatinsky,
Teresa Serrano-Gotarredona,
Bernabe Linares-Barranco,
Sabina Spiga,
Sergey Savelev,
Alexander G Balanov,
Nitin Chawla,
Giuseppe Desoli,
Gerardo Malavena,
Christian Monzio Compagnoni,
Zhongrui Wang,
J Joshua Yang,
Ghazi Sarwat Syed,
Abu Sebastian,
Thomas Mikolajick,
Beatriz Noheda,
Stefan Slesazeck,
Bernard Dieny,
Tuo-Hung,
Hou,
Akhil Varri
, et al. (28 additional authors not shown)
Abstract:
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining t…
▽ More
The roadmap is organized into several thematic sections, outlining current computing challenges, discussing the neuromorphic computing approach, analyzing mature and currently utilized technologies, providing an overview of emerging technologies, addressing material challenges, exploring novel computing concepts, and finally examining the maturity level of emerging technologies while determining the next essential steps for their advancement.
△ Less
Submitted 5 July, 2024; v1 submitted 2 July, 2024;
originally announced July 2024.
-
Characterising Payload Entropy in Packet Flows
Authors:
Anthony Kenyon,
Lipika Deka,
David Elizondo
Abstract:
Accurate and timely detection of cyber threats is critical to keeping our online economy and data safe. A key technique in early detection is the classification of unusual patterns of network behaviour, often hidden as low-frequency events within complex time-series packet flows. One of the ways in which such anomalies can be detected is to analyse the information entropy of the payload within ind…
▽ More
Accurate and timely detection of cyber threats is critical to keeping our online economy and data safe. A key technique in early detection is the classification of unusual patterns of network behaviour, often hidden as low-frequency events within complex time-series packet flows. One of the ways in which such anomalies can be detected is to analyse the information entropy of the payload within individual packets, since changes in entropy can often indicate suspicious activity - such as whether session encryption has been compromised, or whether a plaintext channel has been co-opted as a covert channel. To decide whether activity is anomalous we need to compare real-time entropy values with baseline values, and while the analysis of entropy in packet data is not particularly new, to the best of our knowledge there are no published baselines for payload entropy across common network services. We offer two contributions: 1) We analyse several large packet datasets to establish baseline payload information entropy values for common network services, 2) We describe an efficient method for engineering entropy metrics when performing flow recovery from live or offline packet data, which can be expressed within feature subsets for subsequent analysis and machine learning applications.
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
Improved Flow Recovery from Packet Data
Authors:
Anthony Kenyon,
David Elizondo,
Lipika Deka
Abstract:
Typical event datasets such as those used in network intrusion detection comprise hundreds of thousands, sometimes millions, of discrete packet events. These datasets tend to be high dimensional, stateful, and time-series in nature, holding complex local and temporal feature associations. Packet data can be abstracted into lower dimensional summary data, such as packet flow records, where some of…
▽ More
Typical event datasets such as those used in network intrusion detection comprise hundreds of thousands, sometimes millions, of discrete packet events. These datasets tend to be high dimensional, stateful, and time-series in nature, holding complex local and temporal feature associations. Packet data can be abstracted into lower dimensional summary data, such as packet flow records, where some of the temporal complexities of packet data can be mitigated, and smaller well-engineered feature subsets can be created. This data can be invaluable as training data for machine learning and cyber threat detection techniques. Data can be collected in real-time, or from historical packet trace archives. In this paper we focus on how flow records and summary metadata can be extracted from packet data with high accuracy and robustness. We identify limitations in current methods, how they may impact datasets, and how these flaws may impact learning models. Finally, we propose methods to improve the state of the art and introduce proof of concept tools to support this work.
△ Less
Submitted 15 October, 2023;
originally announced October 2023.
-
Artificial Dendritic Computation: The case for dendrites in neuromorphic circuits
Authors:
Daniel John Mannion,
Anthony Joseph Kenyon
Abstract:
Bio-inspired computing has focused on neuron and synapses with great success. However, the connections between these, the dendrites, also play an important role. In this paper, we investigate the motivation for replicating dendritic computation and present a framework to guide future attempts in their construction. The framework identifies key properties of the dendrites and presents and example o…
▽ More
Bio-inspired computing has focused on neuron and synapses with great success. However, the connections between these, the dendrites, also play an important role. In this paper, we investigate the motivation for replicating dendritic computation and present a framework to guide future attempts in their construction. The framework identifies key properties of the dendrites and presents and example of dendritic computation in the task of sound localisation. We evaluate the impact of dendrites on an BiLSTM neural network's performance, finding that dendrite pre-processing reduce the size of network required for a threshold performance.
△ Less
Submitted 5 April, 2023; v1 submitted 3 April, 2023;
originally announced April 2023.
-
Memristive, Spintronic, and 2D-Materials-Based Devices to Improve and Complement Computing Hardware
Authors:
Dovydas Joksas,
AbdulAziz AlMutairi,
Oscar Lee,
Murat Cubukcu,
Antonio Lombardo,
Hidekazu Kurebayashi,
Anthony J. Kenyon,
Adnan Mehonic
Abstract:
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving…
▽ More
In a data-driven economy, virtually all industries benefit from advances in information technology -- powerful computing systems are critically important for rapid technological progress. However, this progress might be at risk of slowing down if we do not address the discrepancy between our current computing power demands and what the existing technologies can offer. Key limitations to improving energy efficiency are the excessive growth of data transfer costs associated with the von Neumann architecture and the fundamental limits of complementary metal-oxide-semiconductor (CMOS) technologies, such as transistors. In this perspective article, we discuss three technologies that will likely play an essential role in future computing systems: memristive electronics, spintronics, and electronics based on 2D materials. We present how these may transform conventional digital computers and contribute to the adoption of new paradigms, like neuromorphic computing.
△ Less
Submitted 1 July, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks
Authors:
Dovydas Joksas,
Erwei Wang,
Nikolaos Barmpatsalos,
Wing H. Ng,
Anthony J. Kenyon,
George A. Constantinides,
Adnan Mehonic
Abstract:
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy eff…
▽ More
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever-increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor-based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade-offs, such as those between power and reliability. In this work, we design nonideality-aware training of memristor-based neural networks capable of dealing with the most common device nonidealities. We demonstrate the feasibility of using high-resistance devices that exhibit high $I$-$V$ nonlinearity -- by analyzing experimental data and employing nonideality-aware training, we estimate that the energy efficiency of memristive vector-matrix multipliers is improved by three orders of magnitude ($0.715\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$ to $381\ \mathrm{TOPs}^{-1}\mathrm{W}^{-1}$) while maintaining similar accuracy. We show that associating the parameters of neural networks with individual memristors allows to bias these devices towards less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. We demonstrate the universality and robustness of our approach when dealing with a wide range of nonidealities.
△ Less
Submitted 5 May, 2022; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Substitutional tin acceptor states in black phosphorus
Authors:
Mark Wentink,
Julian Gaberle,
Martik Aghajanian,
Arash A. Mostofi,
Neil J. Curson,
Johannes Lischner,
Steven R. Schofield,
Alexander L. Shluger,
Anthony J. Kenyon
Abstract:
Nominally-pure black phosphorus (BP) is commonly found to be a p-type semiconductor, suggesting the ubiquitious presence of impurity species or intrinsic, charged defects. Moreover, scanning tunnelling microscopy (STM) images of black phosphorus reveal the presence of long-ranged double-lobed defect features superimposed onto the surface atomic lattice. We show that both the p-type doping of BP an…
▽ More
Nominally-pure black phosphorus (BP) is commonly found to be a p-type semiconductor, suggesting the ubiquitious presence of impurity species or intrinsic, charged defects. Moreover, scanning tunnelling microscopy (STM) images of black phosphorus reveal the presence of long-ranged double-lobed defect features superimposed onto the surface atomic lattice. We show that both the p-type doping of BP and the defect features observed in STM images can be attributed to substitutional tin impurities. We show that black phosphorus samples produced through two common synthesis pathways contain tin impurities, and we demonstrate that the ground state of substitutional tin impurities is negatively charged for a wide range of Fermi level positions within the BP bandgap. The localised negative charge of the tin impurities induces hydrogenic states in the bandgap and it is the 2p level that sits at the valence band edge that gives rise to the double-lobed features observed in STM images.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Brain-inspired computing: We need a master plan
Authors:
Adnan Mehonic,
Anthony J Kenyon
Abstract:
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with th…
▽ More
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with the funding, focus and support needed. We have done this in the past with digital technologies; we are in the process of doing it with quantum technologies; can we now do it for brain-inspired computing?
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
-
Nanoscale plasticity and neuromorphic dynamics in silicon suboxide RRAM
Authors:
Mark Buckwell,
Wing Hung Ng,
Daniel John Mannion,
Stephen Hudziak,
Adnan Mehonic,
Anthony Joseph Kenyon
Abstract:
Resistive random-access memories, also known as memristors, whose resistance can be modulated by the electrically driven formation and disruption of conductive filaments within an insulator, are promising candidates for neuromorphic applications due to their scalability, low-power operation and diverse functional behaviours. However, understanding the dynamics of individual filaments, and the surr…
▽ More
Resistive random-access memories, also known as memristors, whose resistance can be modulated by the electrically driven formation and disruption of conductive filaments within an insulator, are promising candidates for neuromorphic applications due to their scalability, low-power operation and diverse functional behaviours. However, understanding the dynamics of individual filaments, and the surrounding material, is challenging, owing to the typically very large cross-sectional areas of test devices relative to the nanometre scale of individual filaments. In the present work, conductive atomic force microscopy is used to study the evolution of conductivity at the nanoscale in a fully CMOS-compatible silicon suboxide thin film. Distinct filamentary plasticity and background conductivity enhancement are reported, suggesting that device behaviour might be best described by composite core (filament) and shell (background conductivity) dynamics. Furthermore, constant current measurements demonstrate an interplay between filament formation and rupture, resulting in current-controlled voltage spiking in nanoscale regions, with an estimated optimal energy consumption of 25 attojoules per spike. This is very promising for extremely low-power neuromorphic computation and suggests that the dynamic behaviour observed in larger devices should persist and improve as dimensions are scaled down.
△ Less
Submitted 28 October, 2020;
originally announced October 2020.
-
Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing
Authors:
Adnan Mehonic,
Abu Sebastian,
Bipin Rajendran,
Osvaldo Simeone,
Eleni Vasilaki,
Anthony J. Kenyon
Abstract:
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern reco…
▽ More
Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern recognition, speech and natural language processing, self-driving vehicles, intelligent self-diagnostics tools, autonomous robots, knowledgeable personal assistants, and monitoring. These successes have been mostly supported by three factors: availability of vast amounts of data, continuous growth in computing power, and algorithmic innovations. The approaching demise of Moore's law, and the consequent expected modest improvements in computing power that can be achieved by scaling, raise the question of whether the described progress will be slowed or halted due to hardware limitations. This paper reviews the case for a novel beyond CMOS hardware technology, memristors, as a potential solution for the implementation of power-efficient in-memory computing, deep learning accelerators, and spiking neural networks. Central themes are the reliance on non-von-Neumann computing architectures and the need for developing tailored learning and inference algorithms. To argue that lessons from biology can be useful in providing directions for further progress in artificial intelligence, we briefly discuss an example based reservoir computing. We conclude the review by speculating on the big picture view of future neuromorphic and brain-inspired computing systems.
△ Less
Submitted 30 April, 2020;
originally announced April 2020.
-
Committee machines -- a universal method to deal with non-idealities in memristor-based neural networks
Authors:
D. Joksas,
P. Freitas,
Z. Chai,
W. H. Ng,
M. Buckwell,
C. Li,
W. D. Zhang,
Q. Xia,
A. J. Kenyon,
A. Mehonic
Abstract:
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using…
▽ More
Artificial neural networks are notoriously power- and time-consuming when implemented on conventional von Neumann computing systems. Consequently, recent years have seen an emergence of research in machine learning hardware that strives to bring memory and computing closer together. A popular approach is to realise artificial neural networks in hardware by implementing their synaptic weights using memristive devices. However, various device- and system-level non-idealities usually prevent these physical implementations from achieving high inference accuracy. We suggest applying a well-known concept in computer science -- committee machines -- in the context of memristor-based neural networks. Using simulations and experimental data from three different types of memristive devices, we show that committee machines employing ensemble averaging can successfully increase inference accuracy in physically implemented neural networks that suffer from faulty devices, device-to-device variability, random telegraph noise and line resistance. Importantly, we demonstrate that the accuracy can be improved even without increasing the total number of memristors.
△ Less
Submitted 27 August, 2020; v1 submitted 14 September, 2019;
originally announced September 2019.
-
On the Profile of Multiplicities of Complete Subgraphs
Authors:
Uriel Feige,
Anne Kenyon,
Shimon Kogan
Abstract:
Let $G$ be a $2$-coloring of a complete graph on $n$ vertices, for sufficiently large $n$. We prove that $G$ contains at least $n^{(\frac{1}{4} - o(1))\log n}$ monochromatic complete subgraphs of size $r$, where \[ 0.3\log n < r < 0.7\log n. \] The previously known lower bound on the total number of monochromatic complete subgraphs, due to Székely was $n^{0.1576\log n}$. We also prove that $G$ con…
▽ More
Let $G$ be a $2$-coloring of a complete graph on $n$ vertices, for sufficiently large $n$. We prove that $G$ contains at least $n^{(\frac{1}{4} - o(1))\log n}$ monochromatic complete subgraphs of size $r$, where \[ 0.3\log n < r < 0.7\log n. \] The previously known lower bound on the total number of monochromatic complete subgraphs, due to Székely was $n^{0.1576\log n}$. We also prove that $G$ contains at least $n^{\frac{1}{7} \log n} $ monochromatic complete subgraphs of size $\frac{1}{2}\log n$.
If furthermore one assumes that the largest monochromatic complete subgraph in $G$ is of size $(\frac{1}{2} + o(1))\log n$ (it is a well known open question whether such graphs exist), then for every constant $0 \le c \le \frac{1}{2}$ we determine (up to low order terms) the number of monochromatic complete subgraphs of size $c \log n$. We do so by proving a lower bound that matches (up to low order terms) a previous upper bound of Székely. For example, the number of monochromatic complete subgraphs of size $\frac{1}{2} \log n$ is $n^{\frac{1}{8}(4 - \log e \pm o(1))\log n} \simeq n^{0.32 \log n}$.
△ Less
Submitted 7 January, 2019; v1 submitted 28 March, 2017;
originally announced March 2017.
-
Autonomous frequency stabilization of two extended cavity diode lasers at the potassium wavelength on a sounding rocket
Authors:
Aline N. Dinkelaker,
Max Schiemangk,
Vladimir Schkolnik,
Andrew Kenyon,
Kai Lampmann,
André Wenzlawski,
Patrick Windpassinger,
Ortwin Hellmig,
Thijs Wendrich,
Ernst M. Rasel,
Michele Giunta,
Christian Deutsch,
Christian Kürbis,
Robert Smol,
Andreas Wicht,
Markus Krutzik,
Achim Peters
Abstract:
We have developed, assembled, and flight-proven a stable, compact, and autonomous extended cavity diode laser (ECDL) system designed for atomic physics experiments in space. To that end, two micro-integrated ECDLs at 766.7 nm were frequency stabilized during a sounding rocket flight by means of frequency modulation spectroscopy (FMS) of 39^K and offset locking techniques based on the beat note of…
▽ More
We have developed, assembled, and flight-proven a stable, compact, and autonomous extended cavity diode laser (ECDL) system designed for atomic physics experiments in space. To that end, two micro-integrated ECDLs at 766.7 nm were frequency stabilized during a sounding rocket flight by means of frequency modulation spectroscopy (FMS) of 39^K and offset locking techniques based on the beat note of the two ECDLs. The frequency stabilization as well as additional hard- and software to test hot redundancy mechanisms were implemented as part of a state-machine, which controlled the experiment completely autonomously throughout the entire flight mission.
△ Less
Submitted 28 October, 2016;
originally announced October 2016.
-
Tiling with Small Tiles
Authors:
Anne Kenyon,
Martin Tassy
Abstract:
We look at sets of tiles that can tile any region of size greater than 1 on the square grid. This is not the typical tiling question, but relates closely to it and therefore can help solve other tiling problems -- we give an example of this. We also present a result to a more classic tiling question with dominoes and L-shape tiles.
We look at sets of tiles that can tile any region of size greater than 1 on the square grid. This is not the typical tiling question, but relates closely to it and therefore can help solve other tiling problems -- we give an example of this. We also present a result to a more classic tiling question with dominoes and L-shape tiles.
△ Less
Submitted 10 November, 2015;
originally announced November 2015.
