-
FastSample: Accelerating Distributed Graph Neural Network Training for Billion-Scale Graphs
Authors:
Hesham Mostafa,
Adam Grabowski,
Md Asadullah Turja,
Juan Cervino,
Alejandro Ribeiro,
Nageen Himayat
Abstract:
Training Graph Neural Networks(GNNs) on a large monolithic graph presents unique challenges as the graph cannot fit within a single machine and it cannot be decomposed into smaller disconnected components. Distributed sampling-based training distributes the graph across multiple machines and trains the GNN on small parts of the graph that are randomly sampled every training iteration. We show that…
▽ More
Training Graph Neural Networks(GNNs) on a large monolithic graph presents unique challenges as the graph cannot fit within a single machine and it cannot be decomposed into smaller disconnected components. Distributed sampling-based training distributes the graph across multiple machines and trains the GNN on small parts of the graph that are randomly sampled every training iteration. We show that in a distributed environment, the sampling overhead is a significant component of the training time for large-scale graphs. We propose FastSample which is composed of two synergistic techniques that greatly reduce the distributed sampling time: 1)a new graph partitioning method that eliminates most of the communication rounds in distributed sampling , 2)a novel highly optimized sampling kernel that reduces memory movement during sampling. We test FastSample on large-scale graph benchmarks and show that FastSample speeds up distributed sampling-based GNN training by up to 2x with no loss in accuracy.
△ Less
Submitted 29 November, 2023;
originally announced November 2023.
-
End-to-End Learning for VCSEL-based Optical Interconnects: State-of-the-Art, Challenges, and Opportunities
Authors:
Muralikrishnan Srinivasan,
Jinxiang Song,
Alexander Grabowski,
Krzysztof Szczerba,
Holger K. Iversen,
Mikkel N. Schmidt,
Darko Zibar,
Jochen Schröder,
Anders Larsson,
Christian Häger,
Henk Wymeersch
Abstract:
Optical interconnects (OIs) based on vertical-cavity surface-emitting lasers (VCSELs) are the main workhorse within data centers, supercomputers, and even vehicles, providing low-cost, high-rate connectivity. VCSELs must operate under extremely harsh and time-varying conditions, thus requiring adaptive and flexible designs of the communication chain. Such designs can be built based on mathematical…
▽ More
Optical interconnects (OIs) based on vertical-cavity surface-emitting lasers (VCSELs) are the main workhorse within data centers, supercomputers, and even vehicles, providing low-cost, high-rate connectivity. VCSELs must operate under extremely harsh and time-varying conditions, thus requiring adaptive and flexible designs of the communication chain. Such designs can be built based on mathematical models (model-based design) or learned from data (machine learning (ML) based design). Various ML techniques have recently come to the forefront, replacing individual components in the transmitters and receivers with deep neural networks. Beyond such component-wise learning, end-to-end (E2E) autoencoder approaches can reach the ultimate performance through co-optimizing entire parameterized transmitters and receivers. This tutorial paper aims to provide an overview of ML for VCSEL-based OIs, with a focus on E2E approaches, dealing specifically with the unique challenges facing VCSELs, such as the wide temperature variations and complex models.
△ Less
Submitted 25 November, 2022;
originally announced November 2022.
-
Influence of temporal aspects and age-correlations on the process of opinion formation based on Polish contact survey
Authors:
Andrzej Grabowski,
Andrzej Jarynowski
Abstract:
On the basis of the experimental data concerning interactions between humans the process of Ising-based model of opinion formation in a social network was investigated. In the paper the data concerning human social activity, i.e. frequency and duration time of interpersonal interactions as well as age correlations - homophily are presented in comparison to base line homogeneous, static and uniform…
▽ More
On the basis of the experimental data concerning interactions between humans the process of Ising-based model of opinion formation in a social network was investigated. In the paper the data concerning human social activity, i.e. frequency and duration time of interpersonal interactions as well as age correlations - homophily are presented in comparison to base line homogeneous, static and uniform mixing. It is known from previous studies that number of contact and average age of nearest neighbors are highly correlated with age of an individual. Such real, assortative patterns usually speed up processes (like epidemic spread) on the networks, but here it only plays a role for small social temperature values (by reducing `freezing by heating' effect). A real structure of contacts affects processes in many various studies in different way, however here it causes stronger (dynamic) and smoother (durations) susceptibility on external field. Moreover, our research shows that the cross interactions between contact frequency and its duration impose the significant increase in critical temperature.
△ Less
Submitted 9 July, 2016;
originally announced July 2016.
-
On Duplication in Mathematical Repositories
Authors:
Adam Grabowski,
Christoph Schwarzweller
Abstract:
Building a repository of proof-checked mathematical knowledge is without any doubt a lot of work, and besides the actual formalization process there also is the task of maintaining the repository. Thus it seems obvious to keep a repsoitory as small as possible, in particular each piece of mathematical knowledge should be formalized only once. In this paper, however, we claim that it might be reaso…
▽ More
Building a repository of proof-checked mathematical knowledge is without any doubt a lot of work, and besides the actual formalization process there also is the task of maintaining the repository. Thus it seems obvious to keep a repsoitory as small as possible, in particular each piece of mathematical knowledge should be formalized only once. In this paper, however, we claim that it might be reasonable or even necessary to duplicate knowledge in a mathematical repository. We analyze different situations and reasons for doing so and provide a number of examples supporting our thesis.
△ Less
Submitted 6 May, 2010;
originally announced May 2010.
