-
Modeling Graphs Beyond Hyperbolic: Graph Neural Networks in Symmetric Positive Definite Matrices
Authors:
Wei Zhao,
Federico Lopez,
J. Maxwell Riestenberg,
Michael Strube,
Diaaeldin Taha,
Steve Trettel
Abstract:
Recent research has shown that alignment between the structure of graph data and the geometry of an embedding space is crucial for learning high-quality representations of the data. The uniform geometry of Euclidean and hyperbolic spaces allows for representing graphs with uniform geometric and topological features, such as grids and hierarchies, with minimal distortion. However, real-world graph…
▽ More
Recent research has shown that alignment between the structure of graph data and the geometry of an embedding space is crucial for learning high-quality representations of the data. The uniform geometry of Euclidean and hyperbolic spaces allows for representing graphs with uniform geometric and topological features, such as grids and hierarchies, with minimal distortion. However, real-world graph data is characterized by multiple types of geometric and topological features, necessitating more sophisticated geometric embedding spaces. In this work, we utilize the Riemannian symmetric space of symmetric positive definite matrices (SPD) to construct graph neural networks that can robustly handle complex graphs. To do this, we develop an innovative library that leverages the SPD gyrocalculus tools \cite{lopez2021gyroSPD} to implement the building blocks of five popular graph neural networks in SPD. Experimental results demonstrate that our graph neural networks in SPD substantially outperform their counterparts in Euclidean and hyperbolic spaces, as well as the Cartesian product thereof, on complex graphs for node and graph classification tasks. We release the library and datasets at \url{https://github.com/andyweizhao/SPD4GNNs}.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
Vector-valued Distance and Gyrocalculus on the Space of Symmetric Positive Definite Matrices
Authors:
Federico López,
Beatrice Pozzetti,
Steve Trettel,
Michael Strube,
Anna Wienhard
Abstract:
We propose the use of the vector-valued distance to compute distances and extract geometric information from the manifold of symmetric positive definite matrices (SPD), and develop gyrovector calculus, constructing analogs of vector space operations in this curved space. We implement these operations and showcase their versatility in the tasks of knowledge graph completion, item recommendation, an…
▽ More
We propose the use of the vector-valued distance to compute distances and extract geometric information from the manifold of symmetric positive definite matrices (SPD), and develop gyrovector calculus, constructing analogs of vector space operations in this curved space. We implement these operations and showcase their versatility in the tasks of knowledge graph completion, item recommendation, and question answering. In experiments, the SPD models outperform their equivalents in Euclidean and hyperbolic space. The vector-valued distance allows us to visualize embeddings, showing that the models learn to disentangle representations of positive samples from negative ones.
△ Less
Submitted 26 October, 2021;
originally announced October 2021.
-
Symmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach
Authors:
Federico López,
Beatrice Pozzetti,
Steve Trettel,
Michael Strube,
Anna Wienhard
Abstract:
Learning faithful graph representations as sets of vertex embeddings has become a fundamental intermediary step in a wide range of machine learning applications. We propose the systematic use of symmetric spaces in representation learning, a class encompassing many of the previously used embedding targets. This enables us to introduce a new method, the use of Finsler metrics integrated in a Rieman…
▽ More
Learning faithful graph representations as sets of vertex embeddings has become a fundamental intermediary step in a wide range of machine learning applications. We propose the systematic use of symmetric spaces in representation learning, a class encompassing many of the previously used embedding targets. This enables us to introduce a new method, the use of Finsler metrics integrated in a Riemannian optimization scheme, that better adapts to dissimilar structures in the graph. We develop a tool to analyze the embeddings and infer structural properties of the data sets. For implementation, we choose Siegel spaces, a versatile family of symmetric spaces. Our approach outperforms competitive baselines for graph reconstruction tasks on various synthetic and real-world datasets. We further demonstrate its applicability on two downstream tasks, recommender systems and node classification.
△ Less
Submitted 9 June, 2021;
originally announced June 2021.
-
Hermitian Symmetric Spaces for Graph Embeddings
Authors:
Federico López,
Beatrice Pozzetti,
Steve Trettel,
Anna Wienhard
Abstract:
Learning faithful graph representations as sets of vertex embeddings has become a fundamental intermediary step in a wide range of machine learning applications. The quality of the embeddings is usually determined by how well the geometry of the target space matches the structure of the data. In this work we learn continuous representations of graphs in spaces of symmetric matrices over C. These s…
▽ More
Learning faithful graph representations as sets of vertex embeddings has become a fundamental intermediary step in a wide range of machine learning applications. The quality of the embeddings is usually determined by how well the geometry of the target space matches the structure of the data. In this work we learn continuous representations of graphs in spaces of symmetric matrices over C. These spaces offer a rich geometry that simultaneously admits hyperbolic and Euclidean subspaces, and are amenable to analysis and explicit computations. We implement an efficient method to learn embeddings and compute distances, and develop the tools to operate with such spaces. The proposed models are able to automatically adapt to very dissimilar arrangements without any apriori estimates of graph features. On various datasets with very diverse structural properties and reconstruction measures our model ties the results of competitive baselines for geometrically pure graphs and outperforms them for graphs with mixed geometric features, showcasing the versatility of our approach.
△ Less
Submitted 11 May, 2021;
originally announced May 2021.
-
Ray-marching Thurston geometries
Authors:
Rémi Coulon,
Elisabetta A. Matsumoto,
Henry Segerman,
Steve J. Trettel
Abstract:
We describe algorithms that produce accurate real-time interactive in-space views of the eight Thurston geometries using ray-marching. We give a theoretical framework for our algorithms, independent of the geometry involved. In addition to scenes within a geometry $X$, we also consider scenes within quotient manifolds and orbifolds $X / Γ$. We adapt the Phong lighting model to non-euclidean geomet…
▽ More
We describe algorithms that produce accurate real-time interactive in-space views of the eight Thurston geometries using ray-marching. We give a theoretical framework for our algorithms, independent of the geometry involved. In addition to scenes within a geometry $X$, we also consider scenes within quotient manifolds and orbifolds $X / Γ$. We adapt the Phong lighting model to non-euclidean geometries. The most difficult part of this is the calculation of light intensity, which relates to the area density of geodesic spheres. We also give extensive practical details for each geometry.
△ Less
Submitted 13 January, 2022; v1 submitted 29 October, 2020;
originally announced October 2020.
-
Non-Euclidean Virtual Reality IV: Sol
Authors:
Rémi Coulon,
Elisabetta A. Matsumoto,
Henry Segerman,
Steve Trettel
Abstract:
This article presents virtual reality software designed to explore the Sol geometry. The simulation is available on 3-dimensional.space/sol.html
This article presents virtual reality software designed to explore the Sol geometry. The simulation is available on 3-dimensional.space/sol.html
△ Less
Submitted 2 February, 2020;
originally announced February 2020.
