-
A Technical Note on the Architectural Effects on Maximum Dependency Lengths of Recurrent Neural Networks
Authors:
Jonathan S. Kent,
Michael M. Murray
Abstract:
This work proposes a methodology for determining the maximum dependency length of a recurrent neural network (RNN), and then studies the effects of architectural changes, including the number and neuron count of layers, on the maximum dependency lengths of traditional RNN, gated recurrent unit (GRU), and long-short term memory (LSTM) models.
This work proposes a methodology for determining the maximum dependency length of a recurrent neural network (RNN), and then studies the effects of architectural changes, including the number and neuron count of layers, on the maximum dependency lengths of traditional RNN, gated recurrent unit (GRU), and long-short term memory (LSTM) models.
△ Less
Submitted 19 July, 2024;
originally announced August 2024.
-
The physics of higher-order interactions in complex systems
Authors:
Federico Battiston,
Enrico Amico,
Alain Barrat,
Ginestra Bianconi,
Guilherme Ferraz de Arruda,
Benedetta Franceschiello,
Iacopo Iacopini,
Sonia Kéfi,
Vito Latora,
Yamir Moreno,
Micah M. Murray,
Tiago P. Peixoto,
Francesco Vaccarino,
Giovanni Petri
Abstract:
Complex networks have become the main paradigm for modelling the dynamics of interacting systems. However, networks are intrinsically limited to describing pairwise interactions, whereas real-world systems are often characterized by higher-order interactions involving groups of three or more units. Higher-order structures, such as hypergraphs and simplicial complexes, are therefore a better tool t…
▽ More
Complex networks have become the main paradigm for modelling the dynamics of interacting systems. However, networks are intrinsically limited to describing pairwise interactions, whereas real-world systems are often characterized by higher-order interactions involving groups of three or more units. Higher-order structures, such as hypergraphs and simplicial complexes, are therefore a better tool to map the real organization of many social, biological and man-made systems. Here, we highlight recent evidence of collective behaviours induced by higher-order interactions, and we outline three key challenges for the physics of higher-order systems.
△ Less
Submitted 12 October, 2021;
originally announced October 2021.
-
A computational model for grid maps in neural populations
Authors:
Fabio Anselmi,
Micah M. Murray,
Benedetta Franceschiello
Abstract:
Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this work we introduce a novel theoretical and algorithmic framework able to explain the emergence of hexagonal grid-like response patterns from head direction cells' responses. We show that this pattern is a result…
▽ More
Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this work we introduce a novel theoretical and algorithmic framework able to explain the emergence of hexagonal grid-like response patterns from head direction cells' responses. We show that this pattern is a result of minimal variance encoding of neurons. The novelty lies into the formulation of the encoding problem through the modern Frame Theory language, specifically that of equiangular Frames, providing new insights about the optimality of hexagonal grid receptive fields. The model proposed overcomes some crucial limitations of the current attractor and oscillatory models. It is based on the well-accepted and tested hypothesis of Hebbian learning, providing a simplified cortical-based framework that does not require the presence of theta velocity-driven oscillations (oscillatory model) or translational symmetries in the synaptic connections (attractor model). We moreover demonstrate that the proposed encoding mechanism naturally explains axis alignment of neighbor grid cells and maps shifts, rotations and scaling of the stimuli onto the shape of grid cells' receptive fields, giving a straightforward explanation of the experimental evidence of grid cells remapping under transformations of environmental cues.
△ Less
Submitted 24 July, 2019; v1 submitted 18 February, 2019;
originally announced February 2019.
