-
A Modular Workflow for Performance Benchmarking of Neuronal Network Simulations
Authors:
Jasper Albers,
Jari Pronold,
Anno Christopher Kurth,
Stine Brekke Vennemo,
Kaveh Haghighi Mood,
Alexander Patronis,
Dennis Terhorst,
Jakob Jordan,
Susanne Kunkel,
Tom Tetzlaff,
Markus Diesmann,
Johanna Senk
Abstract:
Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale models that study interactions between multiple brain areas with intricate connecti…
▽ More
Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale models that study interactions between multiple brain areas with intricate connectivity and investigate phenomena on long time scales such as system-level learning require progress in simulation speed. The corresponding development of state-of-the-art simulation engines relies on information provided by benchmark simulations which assess the time-to-solution for scientifically relevant, complementary network models using various combinations of hardware and software revisions. However, maintaining comparability of benchmark results is difficult due to a lack of standardized specifications for measuring the scaling performance of simulators on high-performance computing (HPC) systems. Motivated by the challenging complexity of benchmarking, we define a generic workflow that decomposes the endeavor into unique segments consisting of separate modules. As a reference implementation for the conceptual workflow, we develop beNNch: an open-source software framework for the configuration, execution, and analysis of benchmarks for neuronal network simulations. The framework records benchmarking data and metadata in a unified way to foster reproducibility. For illustration, we measure the performance of various versions of the NEST simulator across network models with different levels of complexity on a contemporary HPC system, demonstrating how performance bottlenecks can be identified, ultimately guiding the development toward more efficient simulation technology.
△ Less
Submitted 16 December, 2021;
originally announced December 2021.
-
Routing brain traffic through the von Neumann bottleneck: Efficient cache usage in spiking neural network simulation code on general purpose computers
Authors:
Jari Pronold,
Jakob Jordan,
Brian J. N. Wylie,
Itaru Kitayama,
Markus Diesmann,
Susanne Kunkel
Abstract:
Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. Wh…
▽ More
Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. When considering a random graph, each node's edges are distributed across thousands of parallel processes. The activity in neuronal networks is also sparse. Each neuron occasionally transmits a brief signal, called spike, via its outgoing synapses to the corresponding target neurons. This spatial and temporal sparsity represents an inherent bottleneck for simulations on conventional computers: Fundamentally irregular memory-access patterns cause poor cache utilization. Using an established neuronal network simulation code as a reference implementation, we investigate how common techniques to recover cache performance such as software-induced prefetching and software pipelining can benefit a real-world application. The algorithmic changes reduce simulation time by up to 50%. The study exemplifies that many-core systems assigned with an intrinsically parallel computational problem can overcome the von Neumann bottleneck of conventional computer architectures.
△ Less
Submitted 11 March, 2022; v1 submitted 27 September, 2021;
originally announced September 2021.
-
Routing brain traffic through the von Neumann bottleneck: Parallel sorting and refactoring
Authors:
Jari Pronold,
Jakob Jordan,
Brian J. N. Wylie,
Itaru Kitayama,
Markus Diesmann,
Susanne Kunkel
Abstract:
Generic simulation code for spiking neuronal networks spends the major part of time in the phase where spikes have arrived at a compute node and need to be delivered to their target neurons. These spikes were emitted over the last interval between communication steps by source neurons distributed across many compute nodes and are inherently irregular with respect to their targets. For finding the…
▽ More
Generic simulation code for spiking neuronal networks spends the major part of time in the phase where spikes have arrived at a compute node and need to be delivered to their target neurons. These spikes were emitted over the last interval between communication steps by source neurons distributed across many compute nodes and are inherently irregular with respect to their targets. For finding the targets, the spikes need to be dispatched to a three-dimensional data structure with decisions on target thread and synapse type to be made on the way. With growing network size a compute node receives spikes from an increasing number of different source neurons until in the limit each synapse on the compute node has a unique source. Here we show analytically how this sparsity emerges over the practically relevant range of network sizes from a hundred thousand to a billion neurons. By profiling a production code we investigate opportunities for algorithmic changes to avoid indirections and branching. Every thread hosts an equal share of the neurons on a compute node. In the original algorithm all threads search through all spikes to pick out the relevant ones. With increasing network size the fraction of hits remains invariant but the absolute number of rejections grows. An alternative algorithm equally divides the spikes among the threads and sorts them in parallel according to target thread and synapse type. After this every thread completes delivery solely of the section of spikes for its own neurons. The new algorithm halves the number of instructions in spike delivery which leads to a reduction of simulation time of up to 40 %. Thus, spike delivery is a fully parallelizable process with a single synchronization point and thereby well suited for many-core systems. Our analysis indicates that further progress requires a reduction of the latency instructions experience in accessing memory.
△ Less
Submitted 10 March, 2022; v1 submitted 23 September, 2021;
originally announced September 2021.
-
Usage and Scaling of an Open-Source Spiking Multi-Area Model of Monkey Cortex
Authors:
Sacha Jennifer van Albada,
Jari Pronold,
Alexander van Meegen,
Markus Diesmann
Abstract:
We are entering an age of `big' computational neuroscience, in which neural network models are increasing in size and in numbers of underlying data sets. Consolidating the zoo of models into large-scale models simultaneously consistent with a wide range of data is only possible through the effort of large teams, which can be spread across multiple research institutions. To ensure that computationa…
▽ More
We are entering an age of `big' computational neuroscience, in which neural network models are increasing in size and in numbers of underlying data sets. Consolidating the zoo of models into large-scale models simultaneously consistent with a wide range of data is only possible through the effort of large teams, which can be spread across multiple research institutions. To ensure that computational neuroscientists can build on each other's work, it is important to make models publicly available as well-documented code. This chapter describes such an open-source model, which relates the connectivity structure of all vision-related cortical areas of the macaque monkey with their resting-state dynamics. We give a brief overview of how to use the executable model specification, which employs NEST as simulation engine, and show its runtime scaling. The solutions found serve as an example for organizing the workflow of future models from the raw experimental data to the visualization of the results, expose the challenges, and give guidance for the construction of ICT infrastructure for neuroscience.
△ Less
Submitted 23 November, 2020;
originally announced November 2020.
-
Study of external electron injection and trapping in the blow-out regime
Authors:
Jari Pronold,
Johannes Thomas,
Alexander Pukhov
Abstract:
In this work we study electron side-injection and trapping in the blow-out regime in deep plasma channels. We analyze the maximum angle of injection, for which at least 90\% of the injected electrons are trapped. We discuss the dependence of this angle on the electrons' initial energy and their injection positions. In the scope of a semi-analytical blow-out model we show that the injection positio…
▽ More
In this work we study electron side-injection and trapping in the blow-out regime in deep plasma channels. We analyze the maximum angle of injection, for which at least 90\% of the injected electrons are trapped. We discuss the dependence of this angle on the electrons' initial energy and their injection positions. In the scope of a semi-analytical blow-out model we show that the injection position is a less critical factor for trapping if electrons are injected into deep plasma channels. PIC simulations and analytical approximations support our results from the semi-analytical model.
△ Less
Submitted 15 June, 2018;
originally announced June 2018.
-
A non-linear theory for the bubble regime of plasma wake fields in tailored plasma channels
Authors:
Johannes Thomas,
Igor Yu. Kostyukov,
Jari Pronold,
Anton Golovanov,
Alexander Pukhov
Abstract:
We introduce a first full analytical bubble and blow-out model for a radially inhomogeneous plasma in a quasi-static approximation. For both cases we calculate the accelerating and the focusing fields. In our model we also assume a thin electron layer that surrounds the wake field and calculate the field configuration within. Our theory holds for arbitrary radial density profiles and reduces to kn…
▽ More
We introduce a first full analytical bubble and blow-out model for a radially inhomogeneous plasma in a quasi-static approximation. For both cases we calculate the accelerating and the focusing fields. In our model we also assume a thin electron layer that surrounds the wake field and calculate the field configuration within. Our theory holds for arbitrary radial density profiles and reduces to known models in the limit of a homogeneous plasma. From a previous study of hollow plasma channels with smooth boundaries for laser-driven electron acceleration in the bubble regime we know that pancake-like laser pulses lead to highest electron energies [Pukhov et al, PRL 113, 245003 (2014)]. As it was shown, the bubble fields can be adjusted to balance the laser depletion and dephasing lengths by varying the plasma density profile inside a deep channel. Now we show why the radial fields in the vacuum part of a channel become defocussing.
△ Less
Submitted 30 October, 2015;
originally announced October 2015.
