-
Load Balancing in Strongly Inhomogeneous Simulations -- a Vlasiator Case Study
Authors:
Leo Kotipalo,
Markus Battarbee,
Yann Pfau-Kempf,
Vertti Tarvus,
Minna Palmroth
Abstract:
Parallelization is a necessity for large-scale simulations due to the amount of data processed. In this article we investigate different load balancing methods using Vlasiator, a global magnetospheric simulation as our case study.
The theoretical basis for load balancing is the (hyper)graph partitioning problem, modeling simulation units as vertices and their data dependencies as edges. As it is…
▽ More
Parallelization is a necessity for large-scale simulations due to the amount of data processed. In this article we investigate different load balancing methods using Vlasiator, a global magnetospheric simulation as our case study.
The theoretical basis for load balancing is the (hyper)graph partitioning problem, modeling simulation units as vertices and their data dependencies as edges. As it is an NP-hard problem, heuristics are necessary for dynamic runtime balancing.
We consider first hypergraph partitioning via an algorithm called parallel hypergraph partitioner (PHG); this is done by partitioning a simplified grid and then attempting to optimize the solution on the finer grid. The second and third are the geometric methods of recursive coordinate bisection (RCB) and recursive inertial bisection (RIB).
Finally we consider the method of Hilbert space filling curves (HSFC). The algorithm projects simulation cells along a Hilbert curve and makes cuts along the curve. This works well due to the excellent locality of Hilbert curves, and can be optimized further by choice of curve. We introduce and investigate six three-dimensional Hilbert curves in total.
Our findings on runs of two different scales indicate the HSFC method provides optimal load balance, followed by RIB and PHG methods and finally by RCB. Of the Hilbert curves evaluated, the Beta curve outperformed the most commonly used curve by a few percent.
△ Less
Submitted 27 May, 2025;
originally announced May 2025.
-
Coalescing MPI communication in 6D Vlasov simulations: solving ghost domains in Vlasiator
Authors:
Markus Battarbee,
Urs Ganse,
Yann Pfau-Kempf,
Markku Alho,
Konstantinos Papadakis,
Minna Palmroth
Abstract:
High-performance computing is used for diverse simulations, some of which parallelize over the Message Passing Interface (MPI) with ease, whilst others may have challenges related to uniform balancing of computational load and communication between simulation domains. We introduce an alternative approach to solving advection equations, specifically in an application to solving the six-dimensional…
▽ More
High-performance computing is used for diverse simulations, some of which parallelize over the Message Passing Interface (MPI) with ease, whilst others may have challenges related to uniform balancing of computational load and communication between simulation domains. We introduce an alternative approach to solving advection equations, specifically in an application to solving the six-dimensional Vlasov equation for modelling space plasmas. Communicating larger ghost domains around the partition assigned to each MPI task and computing on these ghost cells allows for coalescing several discrete communication calls into one. This approach needs more overall data communication and computation, but provides interesting new avenues for the balancing of computational load between MPI tasks. We discuss this trade-off, how it may assist in developing other algorithmic improvements, and how the transition to heterogeneous CPU-GPU architectures may impact its usefulness.
△ Less
Submitted 30 April, 2025;
originally announced April 2025.
-
Sunward Flows in the Magnetosheath Associated with Magnetic Pressure Gradient and Magnetosheath Expansion
Authors:
H. Madanian,
Y. Pfau-Kempf,
R. Rice,
T. Liu,
T. Karlsson,
S. Raptis,
D. Turner,
J. Beedle
Abstract:
A density structure within the magnetic cloud of an interplanetary coronal mass ejection impacted Earth and caused significant perturbations in plasma boundaries. We describe the effects of this structure on the magnetosheath plasma downstream of the bow shock using spacecraft observations. During this event, the bow shock breathing motion is evident due to the changes in the upstream dynamic pres…
▽ More
A density structure within the magnetic cloud of an interplanetary coronal mass ejection impacted Earth and caused significant perturbations in plasma boundaries. We describe the effects of this structure on the magnetosheath plasma downstream of the bow shock using spacecraft observations. During this event, the bow shock breathing motion is evident due to the changes in the upstream dynamic pressure. A magnetic enhancement forms in the inner magnetosheath and ahead of a plasma compression region. The structure has the characteristics of a fast magnetosonic shock wave, propagating earthward and perpendicular to the background magnetic field further accelerating the already heated magnetosheath plasma. Following these events, a sunward motion of the magnetosheath plasma is observed. Ion distributions show that both the high density core population as well as the high energy tail of the distribution have a sunward directed flow indicating that the sunward flows are caused by magnetic field line expansion in the very low $β$ magnetosheath plasma. Rarefaction effects and enhancement of the magnetic pressure in the magnetosheath result in magnetic pressure gradient forcing that drives the expansion of magnetosheath magnetic field lines. This picture is supported by a reasonable agreement between the estimated plasma accelerations and the magnetic pressure gradient force.
△ Less
Submitted 11 April, 2025;
originally announced April 2025.
-
Transient Upstream Mesoscale Structures: Drivers of Solar-Quiet Space Weather
Authors:
Primož Kajdič,
Xóchitl Blanco-Cano,
Lucile Turc,
Martin Archer,
Savvas Raptis,
Terry Z. Liu,
Yann Pfau-Kempf,
Adrian T. LaMoury,
Yufei Hao,
Philippe C. Escoubet,
Nojan Omidi,
David G. Sibeck,
Boyi Wang,
Hui Zhang,
Yu Lin
Abstract:
In recent years, it has become increasingly clear that space weather disturbances can be triggered by transient upstream mesoscale structures (TUMS), independently of the occurrence of large-scale solar wind (SW) structures, such as interplanetary coronal mass ejections and stream interaction regions. Different types of magnetospheric pulsations, transient perturbations of the geomagnetic field an…
▽ More
In recent years, it has become increasingly clear that space weather disturbances can be triggered by transient upstream mesoscale structures (TUMS), independently of the occurrence of large-scale solar wind (SW) structures, such as interplanetary coronal mass ejections and stream interaction regions. Different types of magnetospheric pulsations, transient perturbations of the geomagnetic field and auroral structures are often observed during times when SW monitors indicate quiet conditions, and have been found to be associated to TUMS. In this mini-review we describe the space weather phenomena that have been associated with four of the largest-scale and the most energetic TUMS, namely hot flow anomalies, foreshock bubbles, travelling foreshocks and foreshock compressional boundaries. The space weather phenomena associated with TUMS tend to be more localized and less intense compared to geomagnetic storms. However, the quiet time space weather may occur more often since, especially during solar minima, quiet SW periods prevail over the perturbed times.
△ Less
Submitted 11 November, 2024;
originally announced November 2024.
-
Simulation Models for Exploring Magnetic Reconnection
Authors:
Michael Shay,
Subash Adhikari,
Naoki Beesho,
Joachim Birn,
Jorg Buechner,
Paul Cassak,
Li-Jen Chen,
Yuxi Chen,
Giulia Cozzani,
Jim Drake,
Fan Guo,
Michael Hesse,
Neeraj Jain,
Yann Pfau-Kempf,
Yu Lin,
Yi-Hsin Liu,
Mitsuo Oka,
Yuri A. Omelchenko,
Minna Palmroth,
Oreste Pezzi,
Patricia H. Reiff,
Marc Swisdak,
Frank Toffoletto,
Gabor Toth,
Richard A. Wolf
Abstract:
Simulations have played a critical role in the advancement of our knowledge of magnetic reconnection. However, due to the inherently multiscale nature of reconnection, it is impossible to simulate all physics at all scales. For this reason, a wide range of simulation methods have been crafted to study particular aspects and consequences of magnetic reconnection. This chapter reviews many of these…
▽ More
Simulations have played a critical role in the advancement of our knowledge of magnetic reconnection. However, due to the inherently multiscale nature of reconnection, it is impossible to simulate all physics at all scales. For this reason, a wide range of simulation methods have been crafted to study particular aspects and consequences of magnetic reconnection. This chapter reviews many of these methods, laying out critical assumptions, numerical techniques, and giving examples of scientific results. Plasma models described include magnetohydrodynamics (MHD), Hall MHD, Hybrid, kinetic particle-in-cell (PIC), kinetic Vlasov, Fluid models with embedded PIC, Fluid models with direct feedback from energetic populations, and the Rice Convection Model (RCM).
△ Less
Submitted 9 June, 2024;
originally announced June 2024.
-
Porting the grid-based 3D+3V hybrid-Vlasov kinetic plasma simulation Vlasiator to heterogeneous GPU architectures
Authors:
Markus Battarbee,
Konstantinos Papadakis,
Urs Ganse,
Jaro Hokkanen,
Leo Kotipalo,
Yann Pfau-Kempf,
Markku Alho,
Minna Palmroth
Abstract:
Vlasiator is a space plasma simulation code which models near-Earth ion-kinetic dynamics in three spatial and three velocity dimensions. It is highly parallelized, modeling the Vlasov equation directly through the distribution function, discretized on a Cartesian grid, instead of the more common particle-in-cell approach. Modeling near-Earth space, plasma properties span several orders of magnitud…
▽ More
Vlasiator is a space plasma simulation code which models near-Earth ion-kinetic dynamics in three spatial and three velocity dimensions. It is highly parallelized, modeling the Vlasov equation directly through the distribution function, discretized on a Cartesian grid, instead of the more common particle-in-cell approach. Modeling near-Earth space, plasma properties span several orders of magnitude in temperature, density, and magnetic field strength. In order to fit the required six-dimensional grids in memory, Vlasiator utilizes a sparse block-based velocity mesh, where chunks of velocity space are added or deleted based on the advection requirements of the Vlasov solver. In addition, the spatial mesh is adaptively refined through cell-based octree refinement. In this paper, we describe the design choices of porting Vlasiator to heterogeneous CPU/GPU architectures. We detail the memory management, algorithmic changes, and kernel construction as well as our unified codebase approach, resulting in portability to both NVIDIA and AMD hardware (CUDA and HIP languages, respectively). In particular, we showcase a highly parallel block adjustment approach allowing efficient re-ordering of a sparse velocity mesh. We detail pitfalls we have overcome and lay out a plan for optimization to facilitate future exascale simulations using multi-node GPU supercomputing.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Hybrid-Vlasov simulation of soft X-ray emissions at the Earth's dayside magnetospheric boundaries
Authors:
Maxime Grandin,
Hyunju K. Connor,
Sanni Hoilijoki,
Markus Battarbee,
Yann Pfau-Kempf,
Urs Ganse,
Konstantinos Papadakis,
Minna Palmroth
Abstract:
Solar wind charge exchange produces emissions in the soft X-ray energy range which can enable the study of near-Earth space regions such as the magnetopause, the magnetosheath and the polar cusps by remote sensing techniques. The Solar wind-Magnetosphere-Ionosphere Link Explorer (SMILE) and Lunar Environment heliospheric X-ray Imager (LEXI) missions aim to obtain soft X-ray images of near-Earth sp…
▽ More
Solar wind charge exchange produces emissions in the soft X-ray energy range which can enable the study of near-Earth space regions such as the magnetopause, the magnetosheath and the polar cusps by remote sensing techniques. The Solar wind-Magnetosphere-Ionosphere Link Explorer (SMILE) and Lunar Environment heliospheric X-ray Imager (LEXI) missions aim to obtain soft X-ray images of near-Earth space thanks to their Soft X-ray Imager (SXI) instruments. While earlier modeling works have already simulated soft X-ray images as might be obtained by SMILE SXI during its mission, the numerical models used so far are all based on the magnetohydrodynamics description of the space plasma. To investigate the possible signatures of ion-kinetic-scale processes in soft X-ray images, we use for the first time a global hybrid-Vlasov simulation of the geospace from the Vlasiator model. The simulation is driven by fast and tenuous solar wind conditions and purely southward interplanetary magnetic field. We first produce global X-ray images of the dayside near-Earth space by placing a virtual imaging satellite at two different locations, providing meridional and equatorial views. We then analyze regional features present in the images and show that they correspond to signatures in soft X-ray emissions of mirror-mode wave structures in the magnetosheath and flux transfer events (FTEs) at the magnetopause. Our results suggest that, although the time scales associated with the motion of those transient phenomena will likely be significantly smaller than the integration time of the SMILE and LEXI imagers, mirror-mode structures and FTEs can cumulatively produce detectable signatures in the soft X-ray images. [...]
△ Less
Submitted 20 April, 2023; v1 submitted 30 January, 2023;
originally announced January 2023.
-
First 3D hybrid-Vlasov global simulation of auroral proton precipitation and comparison with satellite observations
Authors:
Maxime Grandin,
Thijs Luttikhuis,
Markus Battarbee,
Giulia Cozzani,
Hongyang Zhou,
Lucile Turc,
Yann Pfau-Kempf,
Harriet George,
Konstantinos Horaites,
Evgeny Gordeev,
Urs Ganse,
Konstantinos Papadakis,
Markku Alho,
Fasil Tesema,
Jonas Suni,
Maxime Dubart,
Vertti Tarvus,
Minna Palmroth
Abstract:
The precipitation of charged particles from the magnetosphere into the ionosphere is one of the crucial coupling mechanisms between these two regions of geospace and is associated with multiple space weather effects, such as global navigation satellite system signal disruption and geomagnetically induced currents at ground level. While precipitating particle fluxes have been measured by numerous s…
▽ More
The precipitation of charged particles from the magnetosphere into the ionosphere is one of the crucial coupling mechanisms between these two regions of geospace and is associated with multiple space weather effects, such as global navigation satellite system signal disruption and geomagnetically induced currents at ground level. While precipitating particle fluxes have been measured by numerous spacecraft missions over the past decades, it often remains difficult to obtain global precipitation patterns with a good time resolution during a substorm. Numerical simulations can help to bridge this gap and improve the understanding of mechanisms leading to particle precipitation at high latitudes through the global view they offer on the near-Earth space system. We present the first results on auroral (0.5-50 keV) proton precipitation within a 3-dimensional simulation of the Vlasiator hybrid-Vlasov model. The run is driven by southward interplanetary magnetic field conditions with constant solar wind parameters. We find that, on the dayside, cusp proton precipitation exhibits the expected energy-latitude dispersion and takes place in the form of successive bursts associated with the transit of flux transfer events formed through dayside magnetopause reconnection. On the nightside, the precipitation takes place within the expected range of geomagnetic latitudes, and it appears clearly that the precipitating particle injection is taking place within a narrow magnetic local time span, associated with fast Earthward plasma flows in the near-Earth magnetotail. Finally, the simulated precipitating fluxes are compared to observations from Defense Meteorological Satellite Program spacecraft during driving conditions similar to those in the simulation and are found to be in good agreement with the measurements.
△ Less
Submitted 24 May, 2023; v1 submitted 16 January, 2023;
originally announced January 2023.
-
ULF wave transmission across collisionless shocks: 2.5D local hybrid simulations
Authors:
Primoz Kajdic,
Yann Pfau-Kempf,
Lucile Turc,
Andrew P Dimmock,
Minna Palmroth,
Kazue Takahashi,
Eemilia Kilpua,
Jan Soucek,
Naoko Takahashi,
Luis Preisser,
Xochitl Blanco-Cano,
Domenico Trotta,
David Burgess
Abstract:
We study the interaction of upstream ultra-low frequency (ULF) waves with collisionless shocks by analyzing the outputs of eleven 2D local hybrid simulation runs. Our simulated shocks have Alfvénic Mach numbers between 4.29-7.42 and their $θ_{BN}$ angles are 15$^\circ$, 30$^\circ$, 45$^\circ$ and 50$^\circ$. The ULF wave foreshocks develop upstream of all of them. The wavelength and the amplitude…
▽ More
We study the interaction of upstream ultra-low frequency (ULF) waves with collisionless shocks by analyzing the outputs of eleven 2D local hybrid simulation runs. Our simulated shocks have Alfvénic Mach numbers between 4.29-7.42 and their $θ_{BN}$ angles are 15$^\circ$, 30$^\circ$, 45$^\circ$ and 50$^\circ$. The ULF wave foreshocks develop upstream of all of them. The wavelength and the amplitude of the upstream waves exhibit a complex dependence on the shock's M$_A$ and $θ_{BN}$. The wavelength positively correlates with both parameters, with the dependence on $θ_{BN}$ being much stronger. The amplitude of the ULF waves is proportional to the product of the reflected beam velocity and density, which also depend on M$_A$ and $θ_{BN}$. The interaction of the ULF waves with the shock causes large-scale (several tens of upstream ion inertial lengths) shock rippling. The properties of the shock ripples are related to the ULF wave properties, namely thier wavelength and amplitude. In turn, the ripples have a large impact on the ULF wave transmission across the shock because they change local shock properties ($θ_{BN}$, strength), so that different sections of the same ULF wave front encounter shock with different characteristics. Downstream fluctuations do not resemble the upstream waves in terms the wavefront extension, orientation or their wavelength. However some features are conserved in the Fourier spectra of downstream compressive waves that present a bump or flattening at wavelengths approximately corresponding to those of the upstream ULF waves. In the transverse downstream spectra these features are weaker.
△ Less
Submitted 25 January, 2022;
originally announced January 2022.
-
FORESAIL-1 cubesat mission to measure radiation belt losses and demonstrate de-orbiting
Authors:
FORESAIL collaboration,
M. Palmroth,
J. Praks,
R. Vainio,
P. Janhunen,
E. K. J. Kilpua,
N. Yu. Ganushkina,
A. Afanasiev,
M. Ala-Lahti,
A. Alho,
T. Asikainen,
E. Asvestari,
M. Battarbee,
A. Binios,
A. Bosser,
T. Brito,
J. Envall,
U. Ganse,
H. George,
J. Gieseler,
S. Good,
M. Grandin,
S. Haslam,
H. -P. Hedman,
H. Hietala
, et al. (35 additional authors not shown)
Abstract:
Today, the near-Earth space is facing a paradigm change as the number of new spacecraft is literally sky-rocketing. Increasing numbers of small satellites threaten the sustainable use of space, as without removal, space debris will eventually make certain critical orbits unusable. A central factor affecting small spacecraft health and leading to debris is the radiation environment, which is unpred…
▽ More
Today, the near-Earth space is facing a paradigm change as the number of new spacecraft is literally sky-rocketing. Increasing numbers of small satellites threaten the sustainable use of space, as without removal, space debris will eventually make certain critical orbits unusable. A central factor affecting small spacecraft health and leading to debris is the radiation environment, which is unpredictable due to an incomplete understanding of the near-Earth radiation environment itself and its variability driven by the solar wind and outer magnetosphere. This paper presents the FORESAIL-1 nanosatellite mission, having two scientific and one technological objectives. The first scientific objective is to measure the energy and flux of energetic particle loss to the atmosphere with a representative energy and pitch angle resolution over a wide range of magnetic local times. To pave the way to novel model - in situ data comparisons, we also show preliminary results on precipitating electron fluxes obtained with the new global hybrid-Vlasov simulation Vlasiator. The second scientific objective of the FORESAIL-1 mission is to measure energetic neutral atoms (ENAs) of solar origin. The solar ENA flux has the potential to contribute importantly to the knowledge of solar eruption energy budget estimations. The technological objective is to demonstrate a satellite de-orbiting technology, and for the first time, make an orbit manoeuvre with a propellantless nanosatellite. FORESAIL-1 will demonstrate the potential for nanosatellites to make important scientific contributions as well as promote the sustainable utilisation of space by using a cost-efficient de-orbiting technology.
△ Less
Submitted 23 May, 2019;
originally announced May 2019.
-
Vlasov methods in space physics and astrophysics
Authors:
Minna Palmroth,
Urs Ganse,
Yann Pfau-Kempf,
Markus Battarbee,
Lucile Turc,
Thiago Brito,
Maxime Grandin,
Sanni Hoilijoki,
Arto Sandroos,
Sebastian von Alfthan
Abstract:
This paper reviews Vlasov-based numerical methods used to model plasma in space physics and astrophysics. Plasma consists of collectively behaving charged particles that form the major part of baryonic matter in the Universe. Many concepts ranging from our own planetary environment to the Solar system and beyond can be understood in terms of kinetic plasma physics, represented by the Vlasov equati…
▽ More
This paper reviews Vlasov-based numerical methods used to model plasma in space physics and astrophysics. Plasma consists of collectively behaving charged particles that form the major part of baryonic matter in the Universe. Many concepts ranging from our own planetary environment to the Solar system and beyond can be understood in terms of kinetic plasma physics, represented by the Vlasov equation. We introduce the physical basis for the Vlasov system, and then outline the associated numerical methods that are typically used. A particular application of the Vlasov system is Vlasiator, the world's first global hybrid-Vlasov simulation for the Earth's magnetic domain, the magnetosphere. We introduce the design strategies for Vlasiator and outline its numerical concepts ranging from solvers to coupling schemes. We review Vlasiator's parallelisation methods and introduce the used high-performance computing (HPC) techniques. A short review of verification, validation and physical results is included. The purpose of the paper is to present the Vlasov system and introduce an example implementation, and to illustrate that even with massive computational challenges, an accurate description of physics can be rewarding in itself and significantly advance our understanding. Upcoming supercomputing resources are making similar efforts feasible in other fields as well, making our design options relevant for others facing similar challenges.
△ Less
Submitted 17 August, 2018;
originally announced August 2018.
