-
Testing for Repeated Motifs and Hierarchical Structure in Stochastic Blockmodels
Authors:
Al-Fahad Al-Qadhi,
Keith Levin,
Vincent Lyzinski
Abstract:
The rise in complexity of network data in neuroscience, social networks, and protein-protein interaction networks has been accompanied by several efforts to model and understand these data at different scales. A key multiscale network modeling technique posits hierarchical structure in the network, and by treating networks as multiple levels of subdivisions with shared statistical properties we ca…
▽ More
The rise in complexity of network data in neuroscience, social networks, and protein-protein interaction networks has been accompanied by several efforts to model and understand these data at different scales. A key multiscale network modeling technique posits hierarchical structure in the network, and by treating networks as multiple levels of subdivisions with shared statistical properties we can efficiently discover smaller subgraph primitives with manageable complexity. One such example of hierarchical modeling is the Hierarchical Stochastic Block Model, which seeks to model complex networks as being composed of community structures repeated across the network. Incorporating repeated structure allows for parameter tying across communities in the SBM, reducing the model complexity compared to the traditional blockmodel. In this work, we formulate a framework for testing for repeated motif hierarchical structure in the stochastic blockmodel framework. We describe a model which naturally expresses networks as a hierarchy of sub-networks with a set of motifs repeating across it, and we demonstrate the practical utility of the test through theoretical analysis and extensive simulation and real data.
△ Less
Submitted 2 March, 2025;
originally announced March 2025.
-
Peer effects in the linear-in-means model may be inestimable even when identified
Authors:
Alex Hayes,
Keith Levin
Abstract:
Linear-in-means models are widely used to investigate peer effects. Identifying peer effects in these models is challenging, but conditions for identification are well-known. However, even when peer effects are identified, they may not be estimable, due to an asymptotic colinearity issue: as sample size increases, peer effects become more and more linearly dependent. We show that asymptotic coline…
▽ More
Linear-in-means models are widely used to investigate peer effects. Identifying peer effects in these models is challenging, but conditions for identification are well-known. However, even when peer effects are identified, they may not be estimable, due to an asymptotic colinearity issue: as sample size increases, peer effects become more and more linearly dependent. We show that asymptotic colinearity occurs whenever nodal covariates are independent of the network and the minimum degree of the network is growing. Asymptotic colinearity can cause estimators to be inconsistent or to converge at slower than expected rates. We also demonstrate that dependence between nodal covariates and network structure can alleviate colinearity issues in random dot product graphs. These results suggest that linear-in-means models are less reliable for studying peer influence than previously believed.
△ Less
Submitted 14 October, 2024;
originally announced October 2024.
-
Estimating network-mediated causal effects via principal components network regression
Authors:
Alex Hayes,
Mark M. Fredrickson,
Keith Levin
Abstract:
We develop a method to decompose causal effects on a social network into an indirect effect mediated by the network, and a direct effect independent of the social network. To handle the complexity of network structures, we assume that latent social groups act as causal mediators. We develop principal components network regression models to differentiate the social effect from the non-social effect…
▽ More
We develop a method to decompose causal effects on a social network into an indirect effect mediated by the network, and a direct effect independent of the social network. To handle the complexity of network structures, we assume that latent social groups act as causal mediators. We develop principal components network regression models to differentiate the social effect from the non-social effect. Fitting the regression models is as simple as principal components analysis followed by ordinary least squares estimation. We prove asymptotic theory for regression coefficients from this procedure and show that it is widely applicable, allowing for a variety of distributions on the regression errors and network edges. We carefully characterize the counterfactual assumptions necessary to use the regression models for causal inference, and show that current approaches to causal network regression may result in over-control bias. The method is very general, so that it is applicable to many types of structured data beyond social networks, such as text, areal data, psychometrics, images and omics.
△ Less
Submitted 6 March, 2025; v1 submitted 22 December, 2022;
originally announced December 2022.
-
Predicting Responses from Weighted Networks with Node Covariates in an Application to Neuroimaging
Authors:
Daniel Kessler,
Keith Levin,
Elizaveta Levina
Abstract:
We consider the setting where many networks are observed on a common node set, and each observation comprises edge weights of a network, covariates observed at each node, and an overall response. The goal is to use the edge weights and node covariates to predict the response while identifying an interpretable set of predictive features. Our motivating application is neuroimaging, where edge weight…
▽ More
We consider the setting where many networks are observed on a common node set, and each observation comprises edge weights of a network, covariates observed at each node, and an overall response. The goal is to use the edge weights and node covariates to predict the response while identifying an interpretable set of predictive features. Our motivating application is neuroimaging, where edge weights encode functional connectivity measured between brain regions, node covariates encode task activations at each brain region, and the response is disease status or score on a behavioral task. We propose an approach that constructs feature groups based on assumed community structure (naturally occurring in neuroimaging applications). We propose two feature grouping schemes that incorporate both edge weights and node covariates, and we derive algorithms for optimization using an overlapping group LASSO penalty. Empirical results on synthetic data show that our method, relative to competing approaches, has similar or improved prediction error along with superior support recovery, enabling a more interpretable and potentially more accurate understanding of the underlying process. We also apply the method to neuroimaging data from the Human Connectome Project. Our approach is widely applicable in neuroimaging where interpretability is highly desired.
△ Less
Submitted 22 August, 2023; v1 submitted 31 October, 2022;
originally announced October 2022.
-
Vertex Nomination in Richly Attributed Networks
Authors:
Keith Levin,
Carey E. Priebe,
Vince Lyzinski
Abstract:
Vertex nomination is a lightly-supervised network information retrieval task in which vertices of interest in one graph are used to query a second graph to discover vertices of interest in the second graph. Similar to other information retrieval tasks, the output of a vertex nomination scheme is a ranked list of the vertices in the second graph, with the heretofore unknown vertices of interest ide…
▽ More
Vertex nomination is a lightly-supervised network information retrieval task in which vertices of interest in one graph are used to query a second graph to discover vertices of interest in the second graph. Similar to other information retrieval tasks, the output of a vertex nomination scheme is a ranked list of the vertices in the second graph, with the heretofore unknown vertices of interest ideally concentrating at the top of the list. Vertex nomination schemes provide a useful suite of tools for efficiently mining complex networks for pertinent information. In this paper, we explore, both theoretically and practically, the dual roles of content (i.e., edge and vertex attributes) and context (i.e., network topology) in vertex nomination. We provide necessary and sufficient conditions under which vertex nomination schemes that leverage both content and context outperform schemes that leverage only content or context separately. While the joint utility of both content and context has been demonstrated empirically in the literature, the framework presented in this paper provides a novel theoretical basis for understanding the potential complementary roles of network features and topology.
△ Less
Submitted 4 May, 2023; v1 submitted 29 April, 2020;
originally announced May 2020.
-
Limit theorems for out-of-sample extensions of the adjacency and Laplacian spectral embeddings
Authors:
Keith Levin,
Fred Roosta,
Minh Tang,
Michael W. Mahoney,
Carey E. Priebe
Abstract:
Graph embeddings, a class of dimensionality reduction techniques designed for relational data, have proven useful in exploring and modeling network structure. Most dimensionality reduction methods allow out-of-sample extensions, by which an embedding can be applied to observations not present in the training set. Applied to graphs, the out-of-sample extension problem concerns how to compute the em…
▽ More
Graph embeddings, a class of dimensionality reduction techniques designed for relational data, have proven useful in exploring and modeling network structure. Most dimensionality reduction methods allow out-of-sample extensions, by which an embedding can be applied to observations not present in the training set. Applied to graphs, the out-of-sample extension problem concerns how to compute the embedding of a vertex that is added to the graph after an embedding has already been computed. In this paper, we consider the out-of-sample extension problem for two graph embedding procedures: the adjacency spectral embedding and the Laplacian spectral embedding. In both cases, we prove that when the underlying graph is generated according to a latent space model called the random dot product graph, which includes the popular stochastic block model as a special case, an out-of-sample extension based on a least-squares objective obeys a central limit theorem about the true latent position of the out-of-sample vertex. In addition, we prove a concentration inequality for the out-of-sample extension of the adjacency spectral embedding based on a maximum-likelihood objective. Our results also yield a convenient framework in which to analyze trade-offs between estimation accuracy and computational expense, which we explore briefly.
△ Less
Submitted 29 September, 2019;
originally announced October 2019.
-
Bootstrapping Networks with Latent Space Structure
Authors:
Keith Levin,
Elizaveta Levina
Abstract:
A core problem in statistical network analysis is to develop network analogues of classical techniques. The problem of bootstrapping network data stands out as especially challenging, since typically one observes only a single network, rather than a sample. Here we propose two methods for obtaining bootstrap samples for networks drawn from latent space models. The first method generates bootstrap…
▽ More
A core problem in statistical network analysis is to develop network analogues of classical techniques. The problem of bootstrapping network data stands out as especially challenging, since typically one observes only a single network, rather than a sample. Here we propose two methods for obtaining bootstrap samples for networks drawn from latent space models. The first method generates bootstrap replicates of network statistics that can be represented as U-statistics in the latent positions, and avoids actually constructing new bootstrapped networks. The second method generates bootstrap replicates of whole networks, and thus can be used for bootstrapping any network function. Commonly studied network quantities that can be represented as U-statistics include many popular summaries, such as average degree and subgraph counts, but other equally popular summaries, such as the clustering coefficient, are not expressible as U-statistics and thus require the second bootstrap method. Under the assumption of a random dot product graph, a type of latent space network model, we show consistency of the proposed bootstrap methods. We give motivating examples throughout and demonstrate the effectiveness of our methods on synthetic data.
△ Less
Submitted 11 October, 2021; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Recovering shared structure from multiple networks with unknown edge distributions
Authors:
Keith Levin,
Asad Lodhia,
Elizaveta Levina
Abstract:
In increasingly many settings, data sets consist of multiple samples from a population of networks, with vertices aligned across these networks. For example, brain connectivity networks in neuroscience consist of measures of interaction between brain regions that have been aligned to a common template. We consider the setting where the observed networks have a shared expectation, but may differ in…
▽ More
In increasingly many settings, data sets consist of multiple samples from a population of networks, with vertices aligned across these networks. For example, brain connectivity networks in neuroscience consist of measures of interaction between brain regions that have been aligned to a common template. We consider the setting where the observed networks have a shared expectation, but may differ in the noise structure on their edges. Our approach exploits the shared mean structure to denoise edge-level measurements of the observed networks and estimate the underlying population-level parameters. We also explore the extent to which edge-level errors influence estimation and downstream inference. We establish a finite-sample concentration inequality for the low-rank eigenvalue truncation of a random weighted adjacency matrix that may be of independent interest. The proposed approach is illustrated on synthetic networks and on data from an fMRI study of schizophrenia.
△ Less
Submitted 8 May, 2021; v1 submitted 12 June, 2019;
originally announced June 2019.
-
Out-of-sample extension of graph adjacency spectral embedding
Authors:
Keith Levin,
Farbod Roosta-Khorasani,
Michael W. Mahoney,
Carey E. Priebe
Abstract:
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample. In this work, we consider the problem of obtaining an out-of-sample extension for the adjacency spectral embedding, a procedure for embedding the vertices of a graph into Euclidean space. We present two diff…
▽ More
Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample. In this work, we consider the problem of obtaining an out-of-sample extension for the adjacency spectral embedding, a procedure for embedding the vertices of a graph into Euclidean space. We present two different approaches to this problem, one based on a least-squares objective and the other based on a maximum-likelihood formulation. We show that if the graph of interest is drawn according to a certain latent position model called a random dot product graph, then both of these out-of-sample extensions estimate the true latent position of the out-of-sample vertex with the same error rate. Further, we prove a central limit theorem for the least-squares-based extension, showing that the estimate is asymptotically normal about the truth in the large-graph limit.
△ Less
Submitted 17 February, 2018;
originally announced February 2018.
-
Vertex nomination: The canonical sampling and the extended spectral nomination schemes
Authors:
Jordan Yoder,
Li Chen,
Henry Pao,
Eric Bridgeford,
Keith Levin,
Donniell Fishkind,
Carey Priebe,
Vince Lyzinski
Abstract:
Suppose that one particular block in a stochastic block model is of interest, but block labels are only observed for a few of the vertices in the network. Utilizing a graph realized from the model and the observed block labels, the vertex nomination task is to order the vertices with unobserved block labels into a ranked nomination list with the goal of having an abundance of interesting vertices…
▽ More
Suppose that one particular block in a stochastic block model is of interest, but block labels are only observed for a few of the vertices in the network. Utilizing a graph realized from the model and the observed block labels, the vertex nomination task is to order the vertices with unobserved block labels into a ranked nomination list with the goal of having an abundance of interesting vertices near the top of the list. There are vertex nomination schemes in the literature, including the optimally precise canonical nomination scheme~$\mathcal{L}^C$ and the consistent spectral partitioning nomination scheme~$\mathcal{L}^P$. While the canonical nomination scheme $\mathcal{L}^C$ is provably optimally precise, it is computationally intractable, being impractical to implement even on modestly sized graphs.
With this in mind, an approximation of the canonical scheme---denoted the {\it canonical sampling nomination scheme} $\mathcal{L}^{CS}$---is introduced; $\mathcal{L}^{CS}$ relies on a scalable, Markov chain Monte Carlo-based approximation of $\mathcal{L}^{C}$, and converges to $\mathcal{L}^{C}$ as the amount of sampling goes to infinity. The spectral partitioning nomination scheme is also extended to the {\it extended spectral partitioning nomination scheme}, $\mathcal{L}^{EP}$, which introduces a novel semisupervised clustering framework to improve upon the precision of $\mathcal{L}^P$. Real-data and simulation experiments are employed to illustrate the precision of these vertex nomination schemes, as well as their empirical computational complexity.
Keywords: vertex nomination, Markov chain Monte Carlo, spectral partitioning, Mclust
MSC[2010]: 60J22, 65C40, 62H30, 62H25
△ Less
Submitted 22 January, 2020; v1 submitted 14 February, 2018;
originally announced February 2018.
-
On consistent vertex nomination schemes
Authors:
Vince Lyzinski,
Keith Levin,
Carey E. Priebe
Abstract:
Given a vertex of interest in a network $G_1$, the vertex nomination problem seeks to find the corresponding vertex of interest (if it exists) in a second network $G_2$. A vertex nomination scheme produces a list of the vertices in $G_2$, ranked according to how likely they are judged to be the corresponding vertex of interest in $G_2$. The vertex nomination problem and related information retriev…
▽ More
Given a vertex of interest in a network $G_1$, the vertex nomination problem seeks to find the corresponding vertex of interest (if it exists) in a second network $G_2$. A vertex nomination scheme produces a list of the vertices in $G_2$, ranked according to how likely they are judged to be the corresponding vertex of interest in $G_2$. The vertex nomination problem and related information retrieval tasks have attracted much attention in the machine learning literature, with numerous applications to social and biological networks. However, the current framework has often been confined to a comparatively small class of network models, and the concept of statistically consistent vertex nomination schemes has been only shallowly explored. In this paper, we extend the vertex nomination problem to a very general statistical model of graphs. Further, drawing inspiration from the long-established classification framework in the pattern recognition literature, we provide definitions for the key notions of Bayes optimality and consistency in our extended vertex nomination framework, including a derivation of the Bayes optimal vertex nomination scheme. In addition, we prove that no universally consistent vertex nomination schemes exist. Illustrative examples are provided throughout.
△ Less
Submitted 9 December, 2018; v1 submitted 15 November, 2017;
originally announced November 2017.
-
Statistical inference on random dot product graphs: a survey
Authors:
Avanti Athreya,
Donniell E. Fishkind,
Keith Levin,
Vince Lyzinski,
Youngser Park,
Yichen Qin,
Daniel L. Sussman,
Minh Tang,
Joshua T. Vogelstein,
Carey E. Priebe
Abstract:
The random dot product graph (RDPG) is an independent-edge random graph that is analytically tractable and, simultaneously, either encompasses or can successfully approximate a wide range of random graphs, from relatively simple stochastic block models to complex latent position graphs. In this survey paper, we describe a comprehensive paradigm for statistical inference on random dot product graph…
▽ More
The random dot product graph (RDPG) is an independent-edge random graph that is analytically tractable and, simultaneously, either encompasses or can successfully approximate a wide range of random graphs, from relatively simple stochastic block models to complex latent position graphs. In this survey paper, we describe a comprehensive paradigm for statistical inference on random dot product graphs, a paradigm centered on spectral embeddings of adjacency and Laplacian matrices. We examine the analogues, in graph inference, of several canonical tenets of classical Euclidean inference: in particular, we summarize a body of existing results on the consistency and asymptotic normality of the adjacency and Laplacian spectral embeddings, and the role these spectral embeddings can play in the construction of single- and multi-sample hypothesis tests for graph data. We investigate several real-world applications, including community detection and classification in large social networks and the determination of functional and biologically relevant network properties from an exploratory data analysis of the Drosophila connectome. We outline requisite background and current open problems in spectral graph inference.
△ Less
Submitted 16 September, 2017;
originally announced September 2017.
-
A central limit theorem for an omnibus embedding of multiple random graphs and implications for multiscale network inference
Authors:
Keith Levin,
Avanti Athreya,
Minh Tang,
Vince Lyzinski,
Youngser Park,
Carey E. Priebe
Abstract:
Performing statistical analyses on collections of graphs is of import to many disciplines, but principled, scalable methods for multi-sample graph inference are few. Here we describe an "omnibus" embedding in which multiple graphs on the same vertex set are jointly embedded into a single space with a distinct representation for each graph. We prove a central limit theorem for this embedding and de…
▽ More
Performing statistical analyses on collections of graphs is of import to many disciplines, but principled, scalable methods for multi-sample graph inference are few. Here we describe an "omnibus" embedding in which multiple graphs on the same vertex set are jointly embedded into a single space with a distinct representation for each graph. We prove a central limit theorem for this embedding and demonstrate how it streamlines graph comparison, obviating the need for pairwise subspace alignments. The omnibus embedding achieves near-optimal inference accuracy when graphs arise from a common distribution and yet retains discriminatory power as a test procedure for the comparison of different graphs. Moreover, this joint embedding and the accompanying central limit theorem are important for answering multiscale graph inference questions, such as the identification of specific subgraphs or vertices responsible for similarity or difference across networks. We illustrate this with a pair of analyses of connectome data derived from dMRI and fMRI scans of human subjects. In particular, we show that this embedding allows the identification of specific brain regions associated with population-level differences. Finally, we sketch how the omnibus embedding can be used to address pressing open problems, both theoretical and practical, in multisample graph inference.
△ Less
Submitted 25 June, 2019; v1 submitted 25 May, 2017;
originally announced May 2017.
-
On the Consistency of the Likelihood Maximization Vertex Nomination Scheme: Bridging the Gap Between Maximum Likelihood Estimation and Graph Matching
Authors:
Vince Lyzinski,
Keith Levin,
Donniell E. Fishkind,
Carey E. Priebe
Abstract:
Given a graph in which a few vertices are deemed interesting a priori, the vertex nomination task is to order the remaining vertices into a nomination list such that there is a concentration of interesting vertices at the top of the list. Previous work has yielded several approaches to this problem, with theoretical results in the setting where the graph is drawn from a stochastic block model (SBM…
▽ More
Given a graph in which a few vertices are deemed interesting a priori, the vertex nomination task is to order the remaining vertices into a nomination list such that there is a concentration of interesting vertices at the top of the list. Previous work has yielded several approaches to this problem, with theoretical results in the setting where the graph is drawn from a stochastic block model (SBM), including a vertex nomination analogue of the Bayes optimal classifier. In this paper, we prove that maximum likelihood (ML)-based vertex nomination is consistent, in the sense that the performance of the ML-based scheme asymptotically matches that of the Bayes optimal scheme. We prove theorems of this form both when model parameters are known and unknown. Additionally, we introduce and prove consistency of a related, more scalable restricted-focus ML vertex nomination scheme. Finally, we incorporate vertex and edge features into ML-based vertex nomination and briefly explore the empirical effectiveness of this approach.
△ Less
Submitted 27 August, 2016; v1 submitted 5 July, 2016;
originally announced July 2016.
-
Laplacian Eigenmaps from Sparse, Noisy Similarity Measurements
Authors:
Keith Levin,
Vince Lyzinski
Abstract:
Manifold learning and dimensionality reduction techniques are ubiquitous in science and engineering, but can be computationally expensive procedures when applied to large data sets or when similarities are expensive to compute. To date, little work has been done to investigate the tradeoff between computational resources and the quality of learned representations. We present both theoretical and e…
▽ More
Manifold learning and dimensionality reduction techniques are ubiquitous in science and engineering, but can be computationally expensive procedures when applied to large data sets or when similarities are expensive to compute. To date, little work has been done to investigate the tradeoff between computational resources and the quality of learned representations. We present both theoretical and experimental explorations of this question. In particular, we consider Laplacian eigenmaps embeddings based on a kernel matrix, and explore how the embeddings behave when this kernel matrix is corrupted by occlusion and noise. Our main theoretical result shows that under modest noise and occlusion assumptions, we can (with high probability) recover a good approximation to the Laplacian eigenmaps embedding based on the uncorrupted kernel matrix. Our results also show how regularization can aid this approximation. Experimentally, we explore the effects of noise and occlusion on Laplacian eigenmaps embeddings of two real-world data sets, one from speech processing and one from neuroscience, as well as a synthetic data set.
△ Less
Submitted 16 August, 2016; v1 submitted 12 March, 2016;
originally announced March 2016.
