-
Pointspectrum: Equivariance Meets Laplacian Filtering for Graph Representation Learning
Authors:
Marinos Poiitis,
Pavlos Sermpezis,
Athena Vakali
Abstract:
Graph Representation Learning (GRL) has become essential for modern graph data mining and learning tasks. GRL aims to capture the graph's structural information and exploit it in combination with node and edge attributes to compute low-dimensional representations. While Graph Neural Networks (GNNs) have been used in state-of-the-art GRL architectures, they have been shown to suffer from over smoot…
▽ More
Graph Representation Learning (GRL) has become essential for modern graph data mining and learning tasks. GRL aims to capture the graph's structural information and exploit it in combination with node and edge attributes to compute low-dimensional representations. While Graph Neural Networks (GNNs) have been used in state-of-the-art GRL architectures, they have been shown to suffer from over smoothing when many GNN layers need to be stacked. In a different GRL approach, spectral methods based on graph filtering have emerged addressing over smoothing; however, up to now, they employ traditional neural networks that cannot efficiently exploit the structure of graph data. Motivated by this, we propose PointSpectrum, a spectral method that incorporates a set equivariant network to account for a graph's structure. PointSpectrum enhances the efficiency and expressiveness of spectral methods, while it outperforms or competes with state-of-the-art GRL methods. Overall, PointSpectrum addresses over smoothing by employing a graph filter and captures a graph's structure through set equivariance, lying on the intersection of GNNs and spectral methods. Our findings are promising for the benefits and applicability of this architectural shift for spectral methods and GRL.
△ Less
Submitted 7 September, 2021; v1 submitted 6 September, 2021;
originally announced September 2021.
-
What training reveals about neural network complexity
Authors:
Andreas Loukas,
Marinos Poiitis,
Stefanie Jegelka
Abstract:
This work explores the Benevolent Training Hypothesis (BTH) which argues that the complexity of the function a deep neural network (NN) is learning can be deduced by its training dynamics. Our analysis provides evidence for BTH by relating the NN's Lipschitz constant at different regions of the input space with the behavior of the stochastic training procedure. We first observe that the Lipschitz…
▽ More
This work explores the Benevolent Training Hypothesis (BTH) which argues that the complexity of the function a deep neural network (NN) is learning can be deduced by its training dynamics. Our analysis provides evidence for BTH by relating the NN's Lipschitz constant at different regions of the input space with the behavior of the stochastic training procedure. We first observe that the Lipschitz constant close to the training data affects various aspects of the parameter trajectory, with more complex networks having a longer trajectory, bigger variance, and often veering further from their initialization. We then show that NNs whose 1st layer bias is trained more steadily (i.e., slowly and with little variation) have bounded complexity even in regions of the input space that are far from any training point. Finally, we find that steady training with Dropout implies a training- and data-dependent generalization bound that grows poly-logarithmically with the number of parameters. Overall, our results support the intuition that good training behavior can be a useful bias towards good generalization.
△ Less
Submitted 29 October, 2021; v1 submitted 8 June, 2021;
originally announced June 2021.
-
On the Aggression Diffusion Modeling and Minimization in Online Social Networks
Authors:
Marinos Poiitis,
Athena Vakali,
Nicolas Kourtellis
Abstract:
Aggression in online social networks has been studied mostly from the perspective of machine learning which detects such behavior in a static context. However, the way aggression diffuses in the network has received little attention as it embeds modeling challenges. In fact, modeling how aggression propagates from one user to another, is an important research topic since it can enable effective ag…
▽ More
Aggression in online social networks has been studied mostly from the perspective of machine learning which detects such behavior in a static context. However, the way aggression diffuses in the network has received little attention as it embeds modeling challenges. In fact, modeling how aggression propagates from one user to another, is an important research topic since it can enable effective aggression monitoring, especially in media platforms which up to now apply simplistic user blocking techniques. In this paper, we address aggression propagation modeling and minimization on Twitter, since it is a popular microblogging platform at which aggression had several onsets. We propose various methods building on two well-known diffusion models, Independent Cascade (IC) and Linear Threshold (LT), to study the aggression evolution in the social network. We experimentally investigate how well each method can model aggression propagation using real Twitter data, while varying parameters, such as seed users selection, graph edge weighting, users' activation timing, etc. It is found that the best performing strategies are the ones to select seed users with a degree-based approach, weigh user edges based on their social circles' overlaps, and activate users according to their aggression levels. We further employ the best performing models to predict which ordinary real users could become aggressive (and vice versa) in the future, and achieve up to AUC=0.89 in this prediction task. Finally, we investigate aggression minimization by launching competitive cascades to "inform" and "heal" aggressors. We show that IC and LT models can be used in aggression minimization, providing less intrusive alternatives to the blocking techniques currently employed by popular online social network platforms.
△ Less
Submitted 30 August, 2021; v1 submitted 21 May, 2020;
originally announced May 2020.
