-
Communication-Efficient Distributed Graph Clustering and Sparsification under Duplication Models
Authors:
Chun Jiang Zhu
Abstract:
In this paper, we consider the problem of clustering graph nodes and sparsifying graph edges over distributed graphs, when graph edges with possibly edge duplicates are observed at physically remote sites. Although edge duplicates across different sites appear to be beneficial at the first glance, in fact they could make the clustering and sparsification more complicated since potentially their pr…
▽ More
In this paper, we consider the problem of clustering graph nodes and sparsifying graph edges over distributed graphs, when graph edges with possibly edge duplicates are observed at physically remote sites. Although edge duplicates across different sites appear to be beneficial at the first glance, in fact they could make the clustering and sparsification more complicated since potentially their processing would need extra computations and communications. We propose the first communication-optimal algorithms for two well-established communication models namely the message passing and the blackboard models. Specifically, given a graph on $n$ nodes with edges observed at $s$ sites, our algorithms achieve communication costs $\tilde{O}(ns)$ and $\tilde{O}(n+s)$ ($\tilde{O}$ hides a polylogarithmic factor), which almost match their lower bounds, $Ω(ns)$ and $Ω(n+s)$, in the message passing and the blackboard models respectively. The communication costs are asymptotically the same as those under non-duplication models, under an assumption on edge distribution. Our algorithms can also guarantee clustering quality nearly as good as that of centralizing all edges and then applying any standard clustering algorithm. Moreover, we perform the first investigation of distributed constructions of graph spanners in the blackboard model. We provide almost matching communication lower and upper bounds for both multiplicative and additive spanners. For example, the communication lower bounds of constructing a $(2k-1)$-spanner in the blackboard with and without duplication models are $Ω(s+n^{1+1/k}\log s)$ and $Ω(s+n^{1+1/k}\max\{1,s^{-1/2-1/(2k)}\log s\})$ respectively, which almost match the upper bound $\tilde{O}(s+n^{1+1/k})$ for both models.
△ Less
Submitted 19 February, 2023;
originally announced February 2023.
-
Heterogeneous Graph Sparsification for Efficient Representation Learning
Authors:
Chandan Chunduru,
Chun Jiang Zhu,
Blake Gains,
Jinbo Bi
Abstract:
Graph sparsification is a powerful tool to approximate an arbitrary graph and has been used in machine learning over homogeneous graphs. In heterogeneous graphs such as knowledge graphs, however, sparsification has not been systematically exploited to improve efficiency of learning tasks. In this work, we initiate the study on heterogeneous graph sparsification and develop sampling-based algorithm…
▽ More
Graph sparsification is a powerful tool to approximate an arbitrary graph and has been used in machine learning over homogeneous graphs. In heterogeneous graphs such as knowledge graphs, however, sparsification has not been systematically exploited to improve efficiency of learning tasks. In this work, we initiate the study on heterogeneous graph sparsification and develop sampling-based algorithms for constructing sparsifiers that are provably sparse and preserve important information in the original graphs. We have performed extensive experiments to confirm that the proposed method can improve time and space complexities of representation learning while achieving comparable, or even better performance in subsequent graph learning tasks based on the learned embedding.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
A Fast Algorithm for Source-wise Round-trip Spanners
Authors:
Chun Jiang Zhu,
Song Han,
Kam-Yiu Lam
Abstract:
In this paper, we study the problem of fast constructions of source-wise round-trip spanners in weighted directed graphs. For a source vertex set $S\subseteq V$ in a graph $G(V,E)$, an $S$-sourcewise round-trip spanner of $G$ of stretch $k$ is a subgraph $H$ of $G$ such that for every pair of vertices $u,v\in S\times V$, their round-trip distance in $H$ is at most $k$ times of their round-trip dis…
▽ More
In this paper, we study the problem of fast constructions of source-wise round-trip spanners in weighted directed graphs. For a source vertex set $S\subseteq V$ in a graph $G(V,E)$, an $S$-sourcewise round-trip spanner of $G$ of stretch $k$ is a subgraph $H$ of $G$ such that for every pair of vertices $u,v\in S\times V$, their round-trip distance in $H$ is at most $k$ times of their round-trip distance in $G$. We show that for a graph $G(V,E)$ with $n$ vertices and $m$ edges, an $s$-sized source vertex set $S\subseteq V$ and an integer $k>1$, there exists an algorithm that in time $O(ms^{1/k}\log^5n)$ constructs an $S$-sourcewise round-trip spanner of stretch $O(k\log n)$ and $O(ns^{1/k}\log^2n)$ edges with high probability. Compared to the fast algorithms for constructing all-pairs round-trip spanners \cite{PRS+18,CLR+20}, our algorithm improve the running time and the number of edges in the spanner when $k$ is super-constant. Compared with the existing algorithm for constructing source-wise round-trip spanners \cite{ZL17}, our algorithm significantly improves their construction time $Ω(\min\{ms,n^ω\})$ (where $ω\in [2,2.373)$ and 2.373 is the matrix multiplication exponent) to nearly linear $O(ms^{1/k}\log^5n)$, at the expense of paying an extra $O(\log n)$ in the stretch. As an important building block of the algorithm, we develop a graph partitioning algorithm to partition $G$ into clusters of bounded radius and prove that for every $u,v\in S\times V$ at small round-trip distance, the probability of separating them in different clusters is small. The algorithm takes the size of $S$ as input and does not need the knowledge of $S$. With the algorithm and a reachability vertex size estimation algorithm, we show that the recursive algorithm for constructing standard round-trip spanners \cite{PRS+18} can be adapted to the source-wise setting.
△ Less
Submitted 19 February, 2023; v1 submitted 12 April, 2020;
originally announced April 2020.
-
Communication-Optimal Distributed Dynamic Graph Clustering
Authors:
Chun Jiang Zhu,
Tan Zhu,
Kam-Yiu Lam,
Song Han,
Jinbo Bi
Abstract:
We consider the problem of clustering graph nodes over large-scale dynamic graphs, such as citation networks, images and web networks, when graph updates such as node/edge insertions/deletions are observed distributively. We propose communication-efficient algorithms for two well-established communication models namely the message passing and the blackboard models. Given a graph with $n$ nodes tha…
▽ More
We consider the problem of clustering graph nodes over large-scale dynamic graphs, such as citation networks, images and web networks, when graph updates such as node/edge insertions/deletions are observed distributively. We propose communication-efficient algorithms for two well-established communication models namely the message passing and the blackboard models. Given a graph with $n$ nodes that is observed at $s$ remote sites over time $[1,t]$, the two proposed algorithms have communication costs $\tilde{O}(ns)$ and $\tilde{O}(n+s)$ ($\tilde{O}$ hides a polylogarithmic factor), almost matching their lower bounds, $Ω(ns)$ and $Ω(n+s)$, respectively, in the message passing and the blackboard models. More importantly, we prove that at each time point in $[1,t]$ our algorithms generate clustering quality nearly as good as that of centralizing all updates up to that time and then applying a standard centralized clustering algorithm. We conducted extensive experiments on both synthetic and real-life datasets which confirmed the communication efficiency of our approach over baseline algorithms while achieving comparable clustering results.
△ Less
Submitted 14 November, 2018;
originally announced November 2018.
