-
Deep Learning as Ricci Flow
Authors:
Anthony Baptista,
Alessandro Barp,
Tapabrata Chakraborti,
Chris Harbron,
Ben D. MacArthur,
Christopher R. S. Banerji
Abstract:
Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general settin…
▽ More
Deep neural networks (DNNs) are powerful tools for approximating the distribution of complex data. It is known that data passing through a trained DNN classifier undergoes a series of geometric and topological simplifications. While some progress has been made toward understanding these transformations in neural networks with smooth activation functions, an understanding in the more general setting of non-smooth activation functions, such as the rectified linear unit (ReLU), which tend to perform better, is required. Here we propose that the geometric transformations performed by DNNs during classification tasks have parallels to those expected under Hamilton's Ricci flow - a tool from differential geometry that evolves a manifold by smoothing its curvature, in order to identify its topology. To illustrate this idea, we present a computational framework to quantify the geometric changes that occur as data passes through successive layers of a DNN, and use this framework to motivate a notion of `global Ricci network flow' that can be used to assess a DNN's ability to disentangle complex data geometries to solve classification problems. By training more than $1,500$ DNN classifiers of different widths and depths on synthetic and real-world data, we show that the strength of global Ricci network flow-like behaviour correlates with accuracy for well-trained DNNs, independently of depth, width and data set. Our findings motivate the use of tools from differential and discrete geometry to the problem of explainability in deep learning.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Geometry and symmetry in biochemical reaction systems
Authors:
Raffaella Mulas,
Rubén J. Sánchez-García,
Ben D. MacArthur
Abstract:
Complex systems of intracellular biochemical reactions have a central role in regulating cell identities and functions. Biochemical reaction systems are typically studied using the language and tools of graph theory. However, graph representations only describe pairwise interactions between molecular species, and so are not well suited to modelling complex sets of reactions that may involve numero…
▽ More
Complex systems of intracellular biochemical reactions have a central role in regulating cell identities and functions. Biochemical reaction systems are typically studied using the language and tools of graph theory. However, graph representations only describe pairwise interactions between molecular species, and so are not well suited to modelling complex sets of reactions that may involve numerous reactants and/or products. Here we make use of a recently-developed hypergraph theory of chemical reactions that naturally allows for higher-order interactions to explore the geometry and quantify functional redundancy in biochemical reactions systems. Our results constitute a general theory of automorphisms for oriented hypergraphs and describe the effect of automorphism group structure on hypergraph Laplacian spectra.
△ Less
Submitted 6 July, 2021; v1 submitted 2 October, 2020;
originally announced October 2020.
-
Stability and steady state of complex cooperative systems: a diakoptic approach
Authors:
Philip Greulich,
Benjamin D. MacArthur,
Cristina Parigini,
Rubén J. Sánchez García
Abstract:
Cooperative dynamics are common in ecology and population dynamics. However, their commonly high degree of complexity with a large number of coupled degrees of freedom renders them difficult to analyse. Here we present a graph-theoretical criterion, via a diakoptic approach (`divide-and-conquer') to determine a cooperative system's stability by decomposing the system's dependence graph into its st…
▽ More
Cooperative dynamics are common in ecology and population dynamics. However, their commonly high degree of complexity with a large number of coupled degrees of freedom renders them difficult to analyse. Here we present a graph-theoretical criterion, via a diakoptic approach (`divide-and-conquer') to determine a cooperative system's stability by decomposing the system's dependence graph into its strongly connected components (SCCs). In particular, we show that a linear cooperative system is Lyapunov stable if the SCCs of the associated dependence graph all have non-positive dominant eigenvalues, and if no SCCs which have dominant eigenvalue zero are connected by a path.
△ Less
Submitted 11 June, 2020; v1 submitted 6 March, 2019;
originally announced March 2019.
-
Microdynamics and Criticality of Adaptive Regulatory Networks
Authors:
Ben D. MacArthur,
Rubén J. Sánchez-García,
Avi Ma'ayan
Abstract:
We present a model of adaptive regulatory networks consisting of a simple biologically-motivated rewiring procedure coupled to an elementary stability criterion. The resulting networks exhibit a characteristic stationary heavy-tailed degree distribution, show complex structural microdynamics and self-organize to a dynamically critical state. We show analytically that the observed criticality resul…
▽ More
We present a model of adaptive regulatory networks consisting of a simple biologically-motivated rewiring procedure coupled to an elementary stability criterion. The resulting networks exhibit a characteristic stationary heavy-tailed degree distribution, show complex structural microdynamics and self-organize to a dynamically critical state. We show analytically that the observed criticality results from the formation and breaking of transient feedback loops during the adaptive process.
△ Less
Submitted 2 April, 2010; v1 submitted 10 December, 2009;
originally announced December 2009.
-
Spectral Characteristics of Network Redundancy
Authors:
Ben D. MacArthur,
Rubén J. Sánchez-García
Abstract:
Many real-world complex networks contain a significant amount of structural redundancy, in which multiple vertices play identical topological roles. Such redundancy arises naturally from the simple growth processes which form and shape many real-world systems. Since structurally redundant elements may be permuted without altering network structure, redundancy may be formally investigated by exam…
▽ More
Many real-world complex networks contain a significant amount of structural redundancy, in which multiple vertices play identical topological roles. Such redundancy arises naturally from the simple growth processes which form and shape many real-world systems. Since structurally redundant elements may be permuted without altering network structure, redundancy may be formally investigated by examining network automorphism (symmetry) groups. Here, we use a group-theoretic approach to give a complete description of spectral signatures of redundancy in undirected networks. In particular, we describe how a network's automorphism group may be used to directly associate specific eigenvalues and eigenvectors with specific network motifs.
△ Less
Submitted 10 September, 2009; v1 submitted 21 April, 2009;
originally announced April 2009.
-
On Automorphism Groups of Networks
Authors:
Ben D. MacArthur,
Rubén J. Sánchez-García,
James W. Anderson
Abstract:
We consider the size and structure of the automorphism groups of a variety of empirical `real-world' networks and find that, in contrast to classical random graph models, many real-world networks are richly symmetric. We relate automorphism group structure to network topology and discuss generic forms of symmetry and their origin in real-world networks.
We consider the size and structure of the automorphism groups of a variety of empirical `real-world' networks and find that, in contrast to classical random graph models, many real-world networks are richly symmetric. We relate automorphism group structure to network topology and discuss generic forms of symmetry and their origin in real-world networks.
△ Less
Submitted 21 September, 2007; v1 submitted 22 May, 2007;
originally announced May 2007.
