Title:Collapse Safe and Scalable Acyclic Graph Generation using Scale Free Distributions

Abstract:Most of the generators currently used to build acyclic graphs do so via random selection of edges. Unfortunately, the graphs thus generated inherently have a tendency to collapse, i.e., the resulting graphs always have almost the same and predictable structure. Specifically, we show that if $O(n \log n \log n)$ edges are added to a binary tree of $n$ nodes then with probability more than $O(1/(\log n)^{1/n})$ the depth of all but $O({\log \log n} ^{\log \log n})$ collapses to 1. Experiments show that the maximum depth is very small (less than 3) thereby reducing the DAG to almost a bipartite graph. Their irregularity, as measured by distribution of length of random walks from root to leaves, is also predictable and small.
In this work, we develop an approach for generating directed acyclic graphs (DAGs) that have "truly irregular" structure, non-trivial depth distribution, and scale to multi million nodes. With such attributes, these graphs would more appropriately represent real world graphs to be dealt with in near future. The generator provides the user knobs to control the structure and shape of the resulting DAG. We conjecture that structurally sound (collapse safe) scalable acyclic graphs require scale free distributions (just as they were crucial in R-MAT for generating real world interaction graphs). Experiments are presented for graphs of size $2^{23}$ and varying edge densities, and in and out degree distributions.