-
Machine learning using magnetic stochastic synapses
Authors:
Matthew O. A. Ellis,
Alex Welbourne,
Stephan J. Kyle,
Paul W. Fry,
Dan A. Allwood,
Thomas J. Hayward,
Eleni Vasilaki
Abstract:
The impressive performance of artificial neural networks has come at the cost of high energy usage and CO$_2$ emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally det…
▽ More
The impressive performance of artificial neural networks has come at the cost of high energy usage and CO$_2$ emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally detrimental stochastic effects in magnetic domain-wall motion in nanowires. We demonstrate functional binary stochastic synapses alongside a gradient learning rule that allows their training with applicability to a range of stochastic systems. The rule, utilising the mean and variance of the neuronal output distribution, finds a trade-off between synaptic stochasticity and energy efficiency depending on the number of measurements of each synapse. For single measurements, the rule results in binary synapses with minimal stochasticity, sacrificing potential performance for robustness. For multiple measurements, synaptic distributions are broad, approximating better-performing continuous synapses. This observation allows us to choose design principles depending on the desired performance and the device's operational speed and energy cost. We verify performance on physical hardware, showing it is comparable to a standard neural network.
△ Less
Submitted 3 March, 2023;
originally announced March 2023.
-
Neuromorphic Overparameterisation and Few-Shot Learning in Multilayer Physical Neural Networks
Authors:
Kilian D. Stenning,
Jack C. Gartside,
Luca Manneschi,
Christopher T. S. Cheung,
Tony Chen,
Alex Vanstone,
Jake Love,
Holly H. Holder,
Francesco Caravelli,
Hidekazu Kurebayashi,
Karin Everschor-Sitte,
Eleni Vasilaki,
Will R. Branford
Abstract:
Physical neuromorphic computing, exploiting the complex dynamics of physical systems, has seen rapid advancements in sophistication and performance. Physical reservoir computing, a subset of neuromorphic computing, faces limitations due to its reliance on single systems. This constrains output dimensionality and dynamic range, limiting performance to a narrow range of tasks. Here, we engineer a su…
▽ More
Physical neuromorphic computing, exploiting the complex dynamics of physical systems, has seen rapid advancements in sophistication and performance. Physical reservoir computing, a subset of neuromorphic computing, faces limitations due to its reliance on single systems. This constrains output dimensionality and dynamic range, limiting performance to a narrow range of tasks. Here, we engineer a suite of nanomagnetic array physical reservoirs and interconnect them in parallel and series to create a multilayer neural network architecture. The output of one reservoir is recorded, scaled and virtually fed as input to the next reservoir. This networked approach increases output dimensionality, internal dynamics and computational performance. We demonstrate that a physical neuromorphic system can achieve an overparameterised state, facilitating meta-learning on small training sets and yielding strong performance across a wide range of tasks. Our approach's efficacy is further demonstrated through few-shot learning, where the system rapidly adapts to new tasks.
△ Less
Submitted 20 August, 2024; v1 submitted 11 November, 2022;
originally announced November 2022.
-
Reconfigurable Reservoir Computing in a Magnetic Metamaterial
Authors:
Ian Vidamour,
Charles Swindells,
Guru Venkat,
Luca Manneschi,
Paul Fry,
Alexander Welbourne,
Richard Rowan-Robinson,
Dirk Backes,
Francisco Maccherozzi,
Sarnjeet Dhesi,
Eleni Vasilaki,
Daniel Allwood,
Thomas Hayward
Abstract:
In-materia reservoir computing (RC) leverages the intrinsic physical responses of functional materials to perform complex computational tasks. Magnetic metamaterials are exciting candidates for RC due to their huge state space, nonlinear emergent dynamics, and non-volatile memory. However, to be suitable for a broad range of tasks, the material system is required to exhibit a broad range of proper…
▽ More
In-materia reservoir computing (RC) leverages the intrinsic physical responses of functional materials to perform complex computational tasks. Magnetic metamaterials are exciting candidates for RC due to their huge state space, nonlinear emergent dynamics, and non-volatile memory. However, to be suitable for a broad range of tasks, the material system is required to exhibit a broad range of properties, and isolating these behaviours experimentally can often prove difficult. By using an electrically accessible device consisting of an array of interconnected magnetic nanorings -- a system shown to exhibit complex emergent dynamics -- here we show how reconfiguring the reservoir architecture allows exploitation of different aspects the system's dynamical behaviours. This is evidenced through state-of-the-art performance in diverse benchmark tasks with very different computational requirements, highlighting the additional computational configurability that can be obtained by altering the input/output architecture around the material system.
△ Less
Submitted 27 August, 2023; v1 submitted 9 June, 2022;
originally announced June 2022.
-
Quantifying the Computational Capability of a Nanomagnetic Reservoir Computing Platform with Emergent Magnetization Dynamics
Authors:
Ian T Vidamour,
Matthew O A Ellis,
David Griffin,
Guru Venkat,
Charles Swindells,
Richard W S Dawidek,
Thomas J Broomhall,
Nina-Juliane Steinke,
Joshaniel F K Cooper,
Francisco Maccherozzi,
Sarnjeet S Dhesi,
Susan Stepney,
Eleni Vasilaki,
Dan A Allwood,
Thomas J Hayward
Abstract:
Devices based on arrays of interconnected magnetic nano-rings with emergent magnetization dynamics have recently been proposed for use in reservoir computing applications, but for them to be computationally useful it must be possible to optimise their dynamical responses. Here, we use a phenomenological model to demonstrate that such reservoirs can be optimised for classification tasks by tuning h…
▽ More
Devices based on arrays of interconnected magnetic nano-rings with emergent magnetization dynamics have recently been proposed for use in reservoir computing applications, but for them to be computationally useful it must be possible to optimise their dynamical responses. Here, we use a phenomenological model to demonstrate that such reservoirs can be optimised for classification tasks by tuning hyperparameters that control the scaling and input rate of data into the system using rotating magnetic fields. We use task-independent metrics to assess the rings' computational capabilities at each set of these hyperparameters and show how these metrics correlate directly to performance in spoken and written digit recognition tasks. We then show that these metrics, and performance in tasks, can be further improved by expanding the reservoir's output to include multiple, concurrent measures of the ring arrays magnetic states.
△ Less
Submitted 31 January, 2022; v1 submitted 29 November, 2021;
originally announced November 2021.
