-
Design and Development of a Java Parallel I/O Library
Authors:
Muhammad Sohaib Ayub,
Muhammad Adnan,
Muhammad Yasir Shafi
Abstract:
Parallel I/O refers to the ability of scientific programs to concurrently read/write from/to a single file from multiple processes executing on distributed memory platforms like compute clusters. In the HPC world, I/O becomes a significant bottleneck for many real-world scientific applications. In the last two decades, there has been significant research in improving the performance of I/O operati…
▽ More
Parallel I/O refers to the ability of scientific programs to concurrently read/write from/to a single file from multiple processes executing on distributed memory platforms like compute clusters. In the HPC world, I/O becomes a significant bottleneck for many real-world scientific applications. In the last two decades, there has been significant research in improving the performance of I/O operations in scientific computing for traditional languages including C, C++, and Fortran. As a result of this, several mature and high-performance libraries including ROMIO (implementation of MPI-IO), parallel HDF5, Parallel I/O (PIO), and parallel netCDF are available today that provide efficient I/O for scientific applications. However, there is very little research done to evaluate and improve I/O performance of Java-based HPC applications. The main hindrance in the development of efficient parallel I/O Java libraries is the lack of a standard API (something equivalent to MPI-IO). Some adhoc solutions have been developed and used in proprietary applications, but there is no general-purpose solution that can be used by performance hungry applications. As part of this project, we plan to develop a Java-based parallel I/O API inspired by the MPI-IO bindings (MPI 2.0 standard document) for C, C++, and Fortran. Once the Java equivalent API of MPI-IO has been developed, we will develop a reference implementation on top of existing Java messaging libraries. Later, we will evaluate and compare performance of our reference Java Parallel I/O library with C/C++ counterparts using benchmarks and real-world applications.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Accelerating Physics Simulations with TPUs: An Inundation Modeling Example
Authors:
Damien Pierce,
R. Lily Hu,
Yusef Shafi,
Anudhyan Boral,
Vladimir Anisimov,
Sella Nevo,
Yi-fan Chen
Abstract:
Recent advancements in hardware accelerators such as Tensor Processing Units (TPUs) speed up computation time relative to Central Processing Units (CPUs) not only for machine learning but, as demonstrated here, also for scientific modeling and computer simulations. To study TPU hardware for distributed scientific computing, we solve partial differential equations (PDEs) for the physics simulation…
▽ More
Recent advancements in hardware accelerators such as Tensor Processing Units (TPUs) speed up computation time relative to Central Processing Units (CPUs) not only for machine learning but, as demonstrated here, also for scientific modeling and computer simulations. To study TPU hardware for distributed scientific computing, we solve partial differential equations (PDEs) for the physics simulation of fluids to model riverine floods. We demonstrate that TPUs achieve a two orders of magnitude speedup over CPUs. Running physics simulations on TPUs is publicly accessible via the Google Cloud Platform, and we release a Python interactive notebook version of the simulation.
△ Less
Submitted 21 April, 2022;
originally announced April 2022.
-
End-to-End Jet Classification of Boosted Top Quarks with the CMS Open Data
Authors:
Michael Andrews,
Bjorn Burkle,
Yi-fan Chen,
Davide DiCroce,
Sergei Gleyzer,
Ulrich Heintz,
Meenakshi Narain,
Manfred Paulini,
Nikolas Pervan,
Yusef Shafi,
Wei Sun,
Emanuele Usai,
Kun Yang
Abstract:
We describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon. The end-to-end deep learning technique combines deep learning algorithms and low-level detector representation of the high-energy collision event. In this study, we use low-level detector informat…
▽ More
We describe a novel application of the end-to-end deep learning technique to the task of discriminating top quark-initiated jets from those originating from the hadronization of a light quark or a gluon. The end-to-end deep learning technique combines deep learning algorithms and low-level detector representation of the high-energy collision event. In this study, we use low-level detector information from the simulated CMS Open Data samples to construct the top jet classifiers. To optimize classifier performance we progressively add low-level information from the CMS tracking detector, including pixel detector reconstructed hits and impact parameters, and demonstrate the value of additional tracking information even when no new spatial structures are added. Relying only on calorimeter energy deposits and reconstructed pixel detector hits, the end-to-end classifier achieves an AUC score of 0.975$\pm$0.002 for the task of classifying boosted top quark jets. After adding derived track quantities, the classifier AUC score increases to 0.9824$\pm$0.0013, serving as the first performance benchmark for these CMS Open Data samples. We additionally provide a timing performance comparison of different processor unit architectures for training the network.
△ Less
Submitted 21 January, 2022; v1 submitted 19 April, 2021;
originally announced April 2021.
-
Simulation Pipeline for Traffic Evacuation in Urban Areas and Emergency Traffic Management Policy Improvements through Case Studies
Authors:
Yu Chen,
S. Yusef Shafi,
Yi-fan Chen
Abstract:
Traffic evacuation plays a critical role in saving lives in devastating disasters such as hurricanes, wildfires, floods, earthquakes, etc. An ability to evaluate evacuation plans in advance for these rare events, including identifying traffic flow bottlenecks, improving traffic management policies, and understanding the robustness of the traffic management policy are critical for emergency managem…
▽ More
Traffic evacuation plays a critical role in saving lives in devastating disasters such as hurricanes, wildfires, floods, earthquakes, etc. An ability to evaluate evacuation plans in advance for these rare events, including identifying traffic flow bottlenecks, improving traffic management policies, and understanding the robustness of the traffic management policy are critical for emergency management. Given the rareness of such events and the corresponding lack of real data, traffic simulation provides a flexible and versatile approach for such scenarios, and furthermore allows dynamic interaction with the simulated evacuation. In this paper, we build a traffic simulation pipeline to explore the above problems, covering many aspects of evacuation, including map creation, demand generation, vehicle behavior, bottleneck identification, traffic management policy improvement, and results analysis. We apply the pipeline to two case studies in California. The first is Paradise, which was destroyed by a large wildfire in 2018 and experienced catastrophic traffic jams during the evacuation. The second is Mill Valley, which has high risk of wildfire and potential traffic issues since the city is situated in a narrow valley.
△ Less
Submitted 22 August, 2020; v1 submitted 13 February, 2020;
originally announced February 2020.
-
Inundation Modeling in Data Scarce Regions
Authors:
Zvika Ben-Haim,
Vladimir Anisimov,
Aaron Yonas,
Varun Gulshan,
Yusef Shafi,
Stephan Hoyer,
Sella Nevo
Abstract:
Flood forecasts are crucial for effective individual and governmental protective action. The vast majority of flood-related casualties occur in developing countries, where providing spatially accurate forecasts is a challenge due to scarcity of data and lack of funding. This paper describes an operational system providing flood extent forecast maps covering several flood-prone regions in India, wi…
▽ More
Flood forecasts are crucial for effective individual and governmental protective action. The vast majority of flood-related casualties occur in developing countries, where providing spatially accurate forecasts is a challenge due to scarcity of data and lack of funding. This paper describes an operational system providing flood extent forecast maps covering several flood-prone regions in India, with the goal of being sufficiently scalable and cost-efficient to facilitate the establishment of effective flood forecasting systems globally.
△ Less
Submitted 30 October, 2019; v1 submitted 11 October, 2019;
originally announced October 2019.
-
Tensor Processing Units for Financial Monte Carlo
Authors:
Francois Belletti,
Davis King,
Kun Yang,
Roland Nelet,
Yusef Shafi,
Yi-Fan Chen,
John Anderson
Abstract:
Monte Carlo methods are critical to many routines in quantitative finance such as derivatives pricing, hedging and risk metrics. Unfortunately, Monte Carlo methods are very computationally expensive when it comes to running simulations in high-dimensional state spaces where they are still a method of choice in the financial industry. Recently, Tensor Processing Units (TPUs) have provided considera…
▽ More
Monte Carlo methods are critical to many routines in quantitative finance such as derivatives pricing, hedging and risk metrics. Unfortunately, Monte Carlo methods are very computationally expensive when it comes to running simulations in high-dimensional state spaces where they are still a method of choice in the financial industry. Recently, Tensor Processing Units (TPUs) have provided considerable speedups and decreased the cost of running Stochastic Gradient Descent (SGD) in Deep Learning. After highlighting computational similarities between training neural networks with SGD and simulating stochastic processes, we ask in the present paper whether TPUs are accurate, fast and simple enough to use for financial Monte Carlo. Through a theoretical reminder of the key properties of such methods and thorough empirical experiments we examine the fitness of TPUs for option pricing, hedging and risk metrics computation. In particular we demonstrate that, in spite of the use of mixed precision, TPUs still provide accurate estimators which are fast to compute when compared to GPUs. We also show that the Tensorflow programming model for TPUs is elegant, expressive and simplifies automated differentiation.
△ Less
Submitted 27 January, 2020; v1 submitted 6 June, 2019;
originally announced June 2019.
-
Output Synchronization of Nonlinear Systems under Input Disturbances
Authors:
He Bai,
S. Yusef Shafi
Abstract:
We study synchronization of nonlinear systems that satisfy an incremental passivity property. We consider the case where the control input is subject to a class of disturbances, including constant and sinusoidal disturbances with unknown phases and magnitudes and known frequencies. We design a distributed control law that recovers the synchronization of the nonlinear systems in the presence of the…
▽ More
We study synchronization of nonlinear systems that satisfy an incremental passivity property. We consider the case where the control input is subject to a class of disturbances, including constant and sinusoidal disturbances with unknown phases and magnitudes and known frequencies. We design a distributed control law that recovers the synchronization of the nonlinear systems in the presence of the disturbances. Simulation results of Goodwin oscillators illustrate the effectiveness of the control law. Finally, we highlight the connection of the proposed control law to the dynamic average consensus estimator developed in [1].
△ Less
Submitted 22 December, 2013;
originally announced December 2013.
-
An Adaptive Algorithm for Synchronization in Diffusively Coupled Systems
Authors:
S. Yusef Shafi,
Murat Arcak
Abstract:
We present an adaptive algorithm that guarantees synchronization in diffusively coupled systems. We first consider compartmental systems of ODEs, where each compartment represents a spatial domain of components interconnected through diffusion terms with like components in different compartments. Each set of like components may have its own weighted undirected graph describing the topology of the…
▽ More
We present an adaptive algorithm that guarantees synchronization in diffusively coupled systems. We first consider compartmental systems of ODEs, where each compartment represents a spatial domain of components interconnected through diffusion terms with like components in different compartments. Each set of like components may have its own weighted undirected graph describing the topology of the interconnection between compartments. The link weights are updated adaptively according to the magnitude of the difference between neighboring agents connected by the link. We next consider reaction-diffusion PDEs with Neumann boundary conditions, and derive an analogous algorithm guaranteeing spatial homogenization of solutions. We provide a numerical example demonstrating the results.
△ Less
Submitted 6 September, 2013;
originally announced September 2013.
-
Intergalactic Magnetic Fields and Gamma Ray Observations of Extreme TeV Blazars
Authors:
Timothy C. Arlen,
Vladimir V. Vassiliev,
Thomas Weisgarber,
Scott P. Wakely,
S. Yusef Shafi
Abstract:
The intergalactic magnetic field (IGMF) in cosmic voids can be indirectly probed through its effect on electromagnetic cascades initiated by a source of TeV gamma-rays, such as active galactic nuclei (AGN). AGN that are sufficiently luminous at TeV energies, "extreme TeV blazars" can produce detectable levels of secondary radiation from inverse Compton (IC) scattering of the electrons in the casca…
▽ More
The intergalactic magnetic field (IGMF) in cosmic voids can be indirectly probed through its effect on electromagnetic cascades initiated by a source of TeV gamma-rays, such as active galactic nuclei (AGN). AGN that are sufficiently luminous at TeV energies, "extreme TeV blazars" can produce detectable levels of secondary radiation from inverse Compton (IC) scattering of the electrons in the cascade, provided that the IGMF is not too large. We review recent work in the literature which utilizes this idea to derive constraints on the IGMF for three TeV-detected blazars-1ES 0229+200, 1ES 1218+304, and RGB J0710+591, and we also investigate four other hard-spectrum TeV blazars in the same framework. Through a recently developed detailed 3D particle-tracking Monte Carlo code, incorporating all major effects of QED and cosmological expansion, we research effects of major uncertainties such as the spectral properties of the source, uncertainty in the UV - far IR extragalactic background light (EBL), undersampled Very High Energy (VHE; energy > 100 GeV) coverage, past history of gamma-ray emission, source vs. observer geometry, and jet AGN Doppler factor. The implications of these effects on the recently reported lower limits of the IGMF are thoroughly examined to conclude that presently available data are compatible with a zero IGMF hypothesis.
△ Less
Submitted 25 July, 2013; v1 submitted 10 October, 2012;
originally announced October 2012.
-
Guaranteeing Spatial Uniformity in Diffusively-Coupled Systems
Authors:
S. Yusef Shafi
Abstract:
We present a condition that guarantees spatially uniformity in the solution trajectories of a diffusively-coupled compartmental ODE model, where each compartment represents a spatial domain of components interconnected through diffusion terms with like components in different compartments. Each set of like components has its own weighted undirected graph describing the topology of the interconnect…
▽ More
We present a condition that guarantees spatially uniformity in the solution trajectories of a diffusively-coupled compartmental ODE model, where each compartment represents a spatial domain of components interconnected through diffusion terms with like components in different compartments. Each set of like components has its own weighted undirected graph describing the topology of the interconnection between compartments. The condition makes use of the Jacobian matrix to describe the dynamics of each compartment as well as the Laplacian eigenvalues of each of the graphs. We discuss linear matrix inequalities that can be used to verify the condition guaranteeing spatial uniformity, and apply the result to a coupled oscillator network. Next we turn to reaction-diffusion PDEs with Neumann boundary conditions, and derive an analogous condition guaranteeing spatial uniformity of solutions. The paper contributes a relaxed condition to check spatial uniformity that allows individual components to have their own specific diffusion terms and interconnection structures.
△ Less
Submitted 21 August, 2012;
originally announced August 2012.
