-
Sheaf theory: from deep geometry to deep learning
Authors:
Anton Ayzenberg,
Thomas Gebhart,
German Magai,
Grigory Solomadin
Abstract:
This paper provides an overview of the applications of sheaf theory in deep learning, data science, and computer science in general. The primary text of this work serves as a friendly introduction to applied and computational sheaf theory accessible to those with modest mathematical familiarity. We describe intuitions and motivations underlying sheaf theory shared by both theoretical researchers a…
▽ More
This paper provides an overview of the applications of sheaf theory in deep learning, data science, and computer science in general. The primary text of this work serves as a friendly introduction to applied and computational sheaf theory accessible to those with modest mathematical familiarity. We describe intuitions and motivations underlying sheaf theory shared by both theoretical researchers and practitioners, bridging classical mathematical theory and its more recent implementations within signal processing and deep learning. We observe that most notions commonly considered specific to cellular sheaves translate to sheaves on arbitrary posets, providing an interesting avenue for further generalization of these methods in applications, and we present a new algorithm to compute sheaf cohomology on arbitrary finite posets in response. By integrating classical theory with recent applications, this work reveals certain blind spots in current machine learning practices. We conclude with a list of problems related to sheaf-theoretic applications that we find mathematically insightful and practically instructive to solve. To ensure the exposition of sheaf theory is self-contained, a rigorous mathematical introduction is provided in appendices which moves from an introduction of diagrams and sheaves to the definition of derived functors, higher order cohomology, sheaf Laplacians, sheaf diffusion, and interconnections of these subjects therein.
△ Less
Submitted 21 February, 2025;
originally announced February 2025.
-
Go with the Flow? A Large-Scale Analysis of Health Care Delivery Networks in the United States Using Hodge Theory
Authors:
Thomas Gebhart,
Xiaojun Fu,
Russell Funk
Abstract:
Health care delivery is a collaborative process, requiring close coordination among networks of providers with specialized expertise. Yet in the United States, care is often spread across multiple disconnected providers (e.g., primary care physicians, specialists), leading to fragmented care delivery networks, and contributing to higher costs and lower quality. While this problem is well known, th…
▽ More
Health care delivery is a collaborative process, requiring close coordination among networks of providers with specialized expertise. Yet in the United States, care is often spread across multiple disconnected providers (e.g., primary care physicians, specialists), leading to fragmented care delivery networks, and contributing to higher costs and lower quality. While this problem is well known, there are relatively few quantitative tools available for characterizing care delivery networks at scale, thereby inhibiting deeper understanding of care fragmentation and efforts to address it. In this, study, we conduct a large-scale analysis of care delivery networks across the United States using the discrete Hodge decomposition, an emerging method of topological data analysis. Using this technique, we decompose networks of patient flows among physicians into three orthogonal subspaces: gradient (acyclic flow), harmonic (global cyclic flow), and curl (local cyclic flow). We document substantial variation in the relative importance of each subspace, suggesting that there may be systematic differences in the organization of care delivery networks across health care markets. Moreover, we find that the relative importance of each subspace is predictive of local care cost and quality, with outcomes tending to be better with greater curl flow and worse with greater harmonic flow.
△ Less
Submitted 13 December, 2021; v1 submitted 18 October, 2021;
originally announced October 2021.
-
Knowledge Sheaves: A Sheaf-Theoretic Framework for Knowledge Graph Embedding
Authors:
Thomas Gebhart,
Jakob Hansen,
Paul Schrater
Abstract:
Knowledge graph embedding involves learning representations of entities -- the vertices of the graph -- and relations -- the edges of the graph -- such that the resulting representations encode the known factual information represented by the knowledge graph and can be used in the inference of new relations. We show that knowledge graph embedding is naturally expressed in the topological and categ…
▽ More
Knowledge graph embedding involves learning representations of entities -- the vertices of the graph -- and relations -- the edges of the graph -- such that the resulting representations encode the known factual information represented by the knowledge graph and can be used in the inference of new relations. We show that knowledge graph embedding is naturally expressed in the topological and categorical language of \textit{cellular sheaves}: a knowledge graph embedding can be described as an approximate global section of an appropriate \textit{knowledge sheaf} over the graph, with consistency constraints induced by the knowledge graph's schema. This approach provides a generalized framework for reasoning about knowledge graph embedding models and allows for the expression of a wide range of prior constraints on embeddings. Further, the resulting embeddings can be easily adapted for reasoning over composite relations without special training. We implement these ideas to highlight the benefits of the extensions inspired by this new perspective.
△ Less
Submitted 18 March, 2023; v1 submitted 7 October, 2021;
originally announced October 2021.
-
Sheaf Neural Networks
Authors:
Jakob Hansen,
Thomas Gebhart
Abstract:
We present a generalization of graph convolutional networks by generalizing the diffusion operation underlying this class of graph neural networks. These sheaf neural networks are based on the sheaf Laplacian, a generalization of the graph Laplacian that encodes additional relational structure parameterized by the underlying graph. The sheaf Laplacian and associated matrices provide an extended ve…
▽ More
We present a generalization of graph convolutional networks by generalizing the diffusion operation underlying this class of graph neural networks. These sheaf neural networks are based on the sheaf Laplacian, a generalization of the graph Laplacian that encodes additional relational structure parameterized by the underlying graph. The sheaf Laplacian and associated matrices provide an extended version of the diffusion operation in graph convolutional networks, providing a proper generalization for domains where relations between nodes are non-constant, asymmetric, and varying in dimension. We show that the resulting sheaf neural networks can outperform graph convolutional networks in domains where relations between nodes are asymmetric and signed.
△ Less
Submitted 7 December, 2020;
originally announced December 2020.
-
The Emergence of Higher-Order Structure in Scientific and Technological Knowledge Networks
Authors:
Thomas Gebhart,
Russell J. Funk
Abstract:
The growth of science and technology is a recombinative process, wherein new discoveries and inventions are built from prior knowledge. Yet relatively little is known about the manner in which scientific and technological knowledge develop and coalesce into larger structures that enable or constrain future breakthroughs. Network science has recently emerged as a framework for measuring the structu…
▽ More
The growth of science and technology is a recombinative process, wherein new discoveries and inventions are built from prior knowledge. Yet relatively little is known about the manner in which scientific and technological knowledge develop and coalesce into larger structures that enable or constrain future breakthroughs. Network science has recently emerged as a framework for measuring the structure and dynamics of knowledge. While helpful, existing approaches struggle to capture the global properties of the underlying networks, leading to conflicting observations about the nature of scientific and technological progress. We bridge this methodological gap using tools from algebraic topology to characterize the higher-order structure of knowledge networks in science and technology across scale. We observe rapid growth in the higher-order structure of knowledge in many scientific and technological fields. This growth is not observable using traditional network measures. We further demonstrate that the emergence of higher-order structure coincides with decline in lower-order structure, and has historically far outpaced the corresponding emergence of higher-order structure in scientific and technological collaboration networks. Up to a point, increases in higher-order structure are associated with better outcomes, as measured by the novelty and impact of papers and patents. However, the nature of science and technology produced under higher-order regimes also appears to be qualitatively different from that produced under lower-order ones, with the former exhibiting greater linguistic abstractness and greater tendencies for building upon prior streams of knowledge.
△ Less
Submitted 23 November, 2020; v1 submitted 28 September, 2020;
originally announced September 2020.
-
Path homologies of deep feedforward networks
Authors:
Samir Chowdhury,
Thomas Gebhart,
Steve Huntsman,
Matvey Yutin
Abstract:
We provide a characterization of two types of directed homology for fully-connected, feedforward neural network architectures. These exact characterizations of the directed homology structure of a neural network architecture are the first of their kind. We show that the directed flag homology of deep networks reduces to computing the simplicial homology of the underlying undirected graph, which is…
▽ More
We provide a characterization of two types of directed homology for fully-connected, feedforward neural network architectures. These exact characterizations of the directed homology structure of a neural network architecture are the first of their kind. We show that the directed flag homology of deep networks reduces to computing the simplicial homology of the underlying undirected graph, which is explicitly given by Euler characteristic computations. We also show that the path homology of these networks is non-trivial in higher dimensions and depends on the number and size of the layers within the network. These results provide a foundation for investigating homological differences between neural network architectures and their realized structure as implied by their parameters.
△ Less
Submitted 16 October, 2019;
originally announced October 2019.
