-
Abstraction requires breadth: a renormalisation group approach
Authors:
Carlo Orientale Caputo,
Elias Seiffert,
Matteo Marsili
Abstract:
Abstraction is the process of extracting the essential features from raw data while ignoring irrelevant details. This is similar to the process of focusing on large-scale properties, systematically removing irrelevant small-scale details, implemented in the renormalisation group of statistical physics. This analogy is suggestive because the fixed points of the renormalisation group offer an ideal…
▽ More
Abstraction is the process of extracting the essential features from raw data while ignoring irrelevant details. This is similar to the process of focusing on large-scale properties, systematically removing irrelevant small-scale details, implemented in the renormalisation group of statistical physics. This analogy is suggestive because the fixed points of the renormalisation group offer an ideal candidate of a truly abstract -- i.e. data independent -- representation. It has been observed that abstraction emerges with depth in neural networks. Deep layers of neural network capture abstract characteristics of data, such as "cat-ness" or "dog-ness" in images, by combining the lower level features encoded in shallow layers (e.g. edges). Yet we argue that depth alone is not enough to develop truly abstract representations. We advocate that the level of abstraction crucially depends on how broad the training set is. We address the issue within a renormalisation group approach where a representation is expanded to encompass a broader set of data. We take the unique fixed point of this transformation -- the Hierarchical Feature Model -- as a candidate for an abstract representation. This theoretical picture is tested in numerical experiments based on Deep Belief Networks trained on data of different breadth. These show that representations in deep layers of neural networks approach the Hierarchical Feature Model as the data gets broader, in agreement with theoretical predictions.
△ Less
Submitted 19 February, 2025; v1 submitted 1 July, 2024;
originally announced July 2024.
-
Bayesian Inference of Minimally Complex Models with Interactions of Arbitrary Order
Authors:
Clélia de Mulatier,
Matteo Marsili
Abstract:
Finding the model that best describes a high-dimensional dataset is a daunting task, even more so if one aims to consider all possible high-order patterns of the data, going beyond pairwise models. For binary data, we show that this task becomes feasible when restricting the search to a family of simple models, that we call Minimally Complex Models (MCMs). MCMs are maximum entropy models that have…
▽ More
Finding the model that best describes a high-dimensional dataset is a daunting task, even more so if one aims to consider all possible high-order patterns of the data, going beyond pairwise models. For binary data, we show that this task becomes feasible when restricting the search to a family of simple models, that we call Minimally Complex Models (MCMs). MCMs are maximum entropy models that have interactions of arbitrarily high order grouped into independent components of minimal complexity. They are simple in information-theoretic terms, which means they can only fit well certain types of data patterns and are therefore easy to falsify. We show that Bayesian model selection restricted to these models is computationally feasible and has many advantages. First, the model evidence, which balances goodness-of-fit against complexity, can be computed efficiently without any parameter fitting, enabling very fast explorations of the space of MCMs. Second, the family of MCMs is invariant under gauge transformations, which can be used to develop a representation-independent approach to statistical modeling. For small systems (up to 15 variables), combining these two results allows us to select the best MCM among all, even though the number of models is already extremely large. For larger systems, we propose simple heuristics to find optimal MCMs in reasonable times. Besides, inference and sampling can be performed without any computational effort. Finally, because MCMs have interactions of any order, they can reveal the presence of important high-order dependencies in the data, providing a new approach to explore high-order dependencies in complex systems. We apply our method to synthetic data and real-world examples, illustrating how MCMs portray the structure of dependencies among variables in a simple manner, extracting falsifiable predictions on symmetries and invariance from the data.
△ Less
Submitted 27 August, 2024; v1 submitted 2 August, 2020;
originally announced August 2020.
-
Estimating the impact of preventive quarantine with reverse epidemiology
Authors:
Jacopo Grilli,
Matteo Marsili,
Guido Sanguinetti
Abstract:
The impact of mitigation or control measures on an epidemics can be estimated by fitting the parameters of a compartmental model to empirical data, and running the model forward with modified parameters that account for a specific measure. This approach has several drawbacks, stemming from biases or lack of availability of data and instability of parameter estimates. Here we take the opposite appr…
▽ More
The impact of mitigation or control measures on an epidemics can be estimated by fitting the parameters of a compartmental model to empirical data, and running the model forward with modified parameters that account for a specific measure. This approach has several drawbacks, stemming from biases or lack of availability of data and instability of parameter estimates. Here we take the opposite approach -- that we call reverse epidemiology. Given the data, we reconstruct backward in time an ensemble of networks of contacts, and we assess the impact of measures on that specific realization of the contagion process. This approach is robust because it only depends on parameters that describe the evolution of the disease within one individual (e.g. latency time) and not on parameters that describe the spread of the epidemics in a population. Using this method, we assess the impact of preventive quarantine on the ongoing outbreak of Covid-19 in Italy. This gives an estimate of how many infected could have been avoided had preventive quarantine been enforced at a given time.
△ Less
Submitted 7 April, 2020;
originally announced April 2020.
-
Multiscale relevance and informative encoding in neuronal spike trains
Authors:
Ryan John Cubero,
Matteo Marsili,
Yasser Roudi
Abstract:
Neuronal responses to complex stimuli and tasks can encompass a wide range of time scales. Understanding these responses requires measures that characterize how the information on these response patterns are represented across multiple temporal resolutions. In this paper we propose a metric -- which we call multiscale relevance (MSR) -- to capture the dynamical variability of the activity of singl…
▽ More
Neuronal responses to complex stimuli and tasks can encompass a wide range of time scales. Understanding these responses requires measures that characterize how the information on these response patterns are represented across multiple temporal resolutions. In this paper we propose a metric -- which we call multiscale relevance (MSR) -- to capture the dynamical variability of the activity of single neurons across different time scales. The MSR is a non-parametric, fully featureless indicator in that it uses only the time stamps of the firing activity without resorting to any a priori covariate or invoking any specific structure in the tuning curve for neural activity. When applied to neural data from the mEC and from the ADn and PoS regions of freely-behaving rodents, we found that neurons having low MSR tend to have low mutual information and low firing sparsity across the correlates that are believed to be encoded by the region of the brain where the recordings were made. In addition, neurons with high MSR contain significant information on spatial navigation and allow to decode spatial position or head direction as efficiently as those neurons whose firing activity has high mutual information with the covariate to be decoded and significantly better than the set of neurons with high local variations in their interspike intervals. Given these results, we propose that the MSR can be used as a measure to rank and select neurons for their information content without the need to appeal to any a priori covariate.
△ Less
Submitted 20 December, 2019; v1 submitted 28 February, 2018;
originally announced February 2018.
-
Translating ceRNA susceptibilities into correlation functions
Authors:
Araks Martirosyan,
Matteo Marsili,
Andrea De Martino
Abstract:
Competition to bind microRNAs induces an effective positive crosstalk between their targets, therefore known as `competing endogenous RNAs' or ceRNAs. While such an effect is known to play a significant role in specific conditions, estimating its strength from data and, experimentally, in physiological conditions appears to be far from simple. Here we show that the susceptibility of ceRNAs to diff…
▽ More
Competition to bind microRNAs induces an effective positive crosstalk between their targets, therefore known as `competing endogenous RNAs' or ceRNAs. While such an effect is known to play a significant role in specific conditions, estimating its strength from data and, experimentally, in physiological conditions appears to be far from simple. Here we show that the susceptibility of ceRNAs to different types of perturbations affecting their competitors (and hence their tendency to crosstalk) can be encoded in quantities as intuitive and as simple to measure as correlation functions. We confirm this scenario by extensive numerical simulations and validate it by re-analyzing PTEN's crosstalk pattern from TCGA breast cancer dataset. These results clarify the links between different quantities used to estimate the intensity of ceRNA crosstalk and provide new keys to analyze transcriptional datasets and effectively probe ceRNA networks in silico.
△ Less
Submitted 19 January, 2017;
originally announced January 2017.
-
Trade-offs in delayed information transmission in biochemical networks
Authors:
Francesca Mancini,
Matteo Marsili,
Aleksandra M. Walczak
Abstract:
In order to transmit biochemical signals, biological regulatory systems dissipate energy with concomitant entropy production. Additionally, signaling often takes place in challenging environmental conditions. In a simple model regulatory circuit given by an input and a delayed output, we explore the trade-offs between information transmission and the system's energetic efficiency. We determine the…
▽ More
In order to transmit biochemical signals, biological regulatory systems dissipate energy with concomitant entropy production. Additionally, signaling often takes place in challenging environmental conditions. In a simple model regulatory circuit given by an input and a delayed output, we explore the trade-offs between information transmission and the system's energetic efficiency. We determine the maximally informative network, given a fixed amount of entropy production and delayed response, exploring both the case with and without feedback. We find that feedback allows the circuit to overcome energy constraints and transmit close to the maximum available information even in the dissipationless limit. Negative feedback loops, characteristic of shock responses, are optimal at high dissipation. Close to equilibrium positive feedback loops, known for their stability, become more informative. Asking how the signaling network should be constructed to best function in the worst possible environment, rather than an optimally tuned one or in steady state, we discover that at large dissipation the same universal motif is optimal in all of these conditions.
△ Less
Submitted 14 April, 2015;
originally announced April 2015.
-
Identifying relevant positions in proteins by Critical Variable Selection
Authors:
Silvia Grigolon,
Silvio Franz,
Matteo Marsili
Abstract:
Evolution in its course found a variety of solutions to the same optimisation problem. The advent of high-throughput genomic sequencing has made available extensive data from which, in principle, one can infer the underlying structure on which biological functions rely. In this paper, we present a new method aimed at extracting sites encoding structural and func- tional properties from a set of pr…
▽ More
Evolution in its course found a variety of solutions to the same optimisation problem. The advent of high-throughput genomic sequencing has made available extensive data from which, in principle, one can infer the underlying structure on which biological functions rely. In this paper, we present a new method aimed at extracting sites encoding structural and func- tional properties from a set of protein primary sequences, namely a Multiple Sequence Alignment. The method, called Critical Variable Selection, is based on the idea that subsets of relevant sites cor- respond to subsequences that occur with a particularly broad frequency distribution in the dataset. By applying this algorithm to in silico sequences, to the Response Regulator Receiver and to the Voltage Sensor Domain of Ion Channels, we show that this procedure recovers not only information encoded in single site statistics and pairwise correlations but it also captures dependencies going beyond pairwise correlations. The method proposed here is complementary to Statistical Coupling Analysis, in that the most relevant sites predicted by the two methods markedly differ. We find robust and consistent results for datasets as small as few hundred sequences, that reveal a hidden hierarchy of sites that is consistent with present knowledge on biologically relevant sites and evo- lutionary dynamics. This suggests that Critical Variable Selection is able to identify in a Multiple Sequence Alignment a core of sites encoding functional and structural information.
△ Less
Submitted 19 January, 2016; v1 submitted 12 March, 2015;
originally announced March 2015.
-
Phenotypic constraints promote latent versatility and carbon efficiency in metabolic networks
Authors:
Marco Bardoscia,
Matteo Marsili,
Areejit Samal
Abstract:
System-level properties of metabolic networks may be the direct product of natural selection or arise as a by-product of selection on other properties. Here we study the effect of direct selective pressure for growth or viability in particular environments on two properties of metabolic networks: latent versatility to function in additional environments and carbon usage efficiency. Using a Markov…
▽ More
System-level properties of metabolic networks may be the direct product of natural selection or arise as a by-product of selection on other properties. Here we study the effect of direct selective pressure for growth or viability in particular environments on two properties of metabolic networks: latent versatility to function in additional environments and carbon usage efficiency. Using a Markov Chain Monte Carlo (MCMC) sampling based on Flux Balance Analysis (FBA), we sample from a known biochemical universe random viable metabolic networks that differ in the number of directly constrained environments. We find that the latent versatility of sampled metabolic networks increases with the number of directly constrained environments and with the size of the networks. We then show that the average carbon wastage of sampled metabolic networks across the constrained environments decreases with the number of directly constrained environments and with the size of the networks. Our work expands the growing body of evidence about nonadaptive origins of key functional properties of biological networks.
△ Less
Submitted 31 July, 2015; v1 submitted 20 August, 2014;
originally announced August 2014.
-
Fishing out collective memory of migratory schools
Authors:
Giancarlo De Luca,
Patrizio Mariani,
Brian R. MacKenzie,
Matteo Marsili
Abstract:
Animals form groups for many reasons but there are costs and benefit associated with group formation. One of the benefits is collective memory. In groups on the move, social interactions play a crucial role in the cohesion and the ability to make consensus decisions. When migrating from spawning to feeding areas fish schools need to retain a collective memory of the destination site over thousand…
▽ More
Animals form groups for many reasons but there are costs and benefit associated with group formation. One of the benefits is collective memory. In groups on the move, social interactions play a crucial role in the cohesion and the ability to make consensus decisions. When migrating from spawning to feeding areas fish schools need to retain a collective memory of the destination site over thousand of kilometers and changes in group formation or individual preference can produce sudden changes in migration pathways. We propose a modelling framework, based on stochastic adaptive networks, that can reproduce this collective behaviour. We assume that three factors control group formation and school migration behaviour: the intensity of social interaction, the relative number of informed individuals and the preference that each individual has for the particular migration area. We treat these factors independently and relate the individuals' preferences to the experience and memory for certain migration sites. We demonstrate that removal of knowledgable individuals or alteration of individual preference can produce rapid changes in group formation and collective behavior. For example, intensive fishing targeting the migratory species and also their preferred prey can reduce both terms to a point at which migration to the destination sites is suddenly stopped. The conceptual approaches represented by our modelling framework may therefore be able to explain large-scale changes in fish migration and spatial distribution.
△ Less
Submitted 20 March, 2014; v1 submitted 29 May, 2013;
originally announced May 2013.
-
Time-dependent information transmission in a model regulatory circuit
Authors:
Francesca Mancini,
Chris H. Wiggins,
Matteo Marsili,
Aleksandra M. Walczak
Abstract:
Many biological regulatory systems process signals out of steady state and respond with a physiological delay. A simple model of regulation which respects these features shows how the ability of a delayed output to transmit information is limited: at short times by the timescale of the dynamic input, at long times by that of the dynamic output. We find that topologies of maximally informative netw…
▽ More
Many biological regulatory systems process signals out of steady state and respond with a physiological delay. A simple model of regulation which respects these features shows how the ability of a delayed output to transmit information is limited: at short times by the timescale of the dynamic input, at long times by that of the dynamic output. We find that topologies of maximally informative networks correspond to commonly occurring biological circuits linked to stress response and that circuits functioning out of steady state may exploit absorbing states to transmit information optimally.
△ Less
Submitted 12 August, 2013; v1 submitted 19 February, 2013;
originally announced February 2013.
-
The Effect of Nonstationarity on Models Inferred from Neural Data
Authors:
Joanna Tyrcha,
Yasser Roudi,
Matteo Marsili,
John Hertz
Abstract:
Neurons subject to a common non-stationary input may exhibit a correlated firing behavior. Correlations in the statistics of neural spike trains also arise as the effect of interaction between neurons. Here we show that these two situations can be distinguished, with machine learning techniques, provided the data are rich enough. In order to do this, we study the problem of inferring a kinetic Isi…
▽ More
Neurons subject to a common non-stationary input may exhibit a correlated firing behavior. Correlations in the statistics of neural spike trains also arise as the effect of interaction between neurons. Here we show that these two situations can be distinguished, with machine learning techniques, provided the data are rich enough. In order to do this, we study the problem of inferring a kinetic Ising model, stationary or nonstationary, from the available data. We apply the inference procedure to two data sets: one from salamander retinal ganglion cells and the other from a realistic computational cortical network model. We show that many aspects of the concerted activity of the salamander retinal neurons can be traced simply to the external input. A model of non-interacting neurons subject to a non-stationary external field outperforms a model with stationary input with couplings between neurons, even accounting for the differences in the number of model parameters. When couplings are added to the non-stationary model, for the retinal data, little is gained: the inferred couplings are generally not significant. Likewise, the distribution of the sizes of sets of neurons that spike simultaneously and the frequency of spike patterns as function of their rank (Zipf plots) are well-explained by an independent-neuron model with time-dependent external input, and adding connections to such a model does not offer significant improvement. For the cortical model data, robust couplings, well correlated with the real connections, can be inferred using the non-stationary model. Adding connections to this model slightly improves the agreement with the data for the probability of synchronous spikes but hardly affects the Zipf plot.
△ Less
Submitted 31 January, 2013; v1 submitted 26 March, 2012;
originally announced March 2012.
-
One way to grow, many ways to shrink: the reversible Von Neumann expanding model
Authors:
A. De Martino,
M. Figliuzzi,
M. Marsili
Abstract:
We study the solutions of Von Neumann's expanding model with reversible processes for an infinite reaction network. We show that, contrary to the irreversible case, the solution space need not be convex in contracting phases (i.e. phases where the concentrations of reagents necessarily decrease over time). At optimality, this implies that, while multiple dynamical paths of global contraction exist…
▽ More
We study the solutions of Von Neumann's expanding model with reversible processes for an infinite reaction network. We show that, contrary to the irreversible case, the solution space need not be convex in contracting phases (i.e. phases where the concentrations of reagents necessarily decrease over time). At optimality, this implies that, while multiple dynamical paths of global contraction exist, optimal expansion is achieved by a unique time evolution of reaction fluxes. This scenario is investigated in a statistical mechanics framework by a replica symmetric theory. The transition from a non-convex to a convex solution space, which turns out to be well described by a phenomenological order parameter (the fraction of unused reversible reactions) is analyzed numerically.
△ Less
Submitted 7 June, 2010;
originally announced June 2010.
-
Identifying essential genes in E. coli from a metabolic optimization principle
Authors:
C. Martelli,
A. De Martino,
E. Marinari,
M. Marsili,
I. Perez Castillo
Abstract:
Understanding the organization of reaction fluxes in cellular metabolism from the stoichiometry and the topology of the underlying biochemical network is a central issue in systems biology. In this task, it is important to devise reasonable approximation schemes that rely on the stoichiometric data only, because full-scale kinetic approaches are computationally affordable only for small networks…
▽ More
Understanding the organization of reaction fluxes in cellular metabolism from the stoichiometry and the topology of the underlying biochemical network is a central issue in systems biology. In this task, it is important to devise reasonable approximation schemes that rely on the stoichiometric data only, because full-scale kinetic approaches are computationally affordable only for small networks (e.g. red blood cells, about 50 reactions). Methods commonly employed are based on finding the stationary flux configurations that satisfy mass-balance conditions for metabolites, often coupling them to local optimization rules (e.g. maximization of biomass production) to reduce the size of the solution space to a single point. Such methods have been widely applied and have proven able to reproduce experimental findings for relatively simple organisms in specific conditions. Here we define and study a constraint-based model of cellular metabolism where neither mass balance nor flux stationarity are postulated, and where the relevant flux configurations optimize the global growth of the system. In the case of E. coli, steady flux states are recovered as solutions, though mass-balance conditions are violated for some metabolites, implying a non-zero net production of the latter. Such solutions furthermore turn out to provide the correct statistics of fluxes for the bacterium E. coli in different environments and compare well with the available experimental evidence on individual fluxes. Conserved metabolic pools play a key role in determining growth rate and flux variability. Finally, we are able to connect phenomenological gene essentiality with `frozen' fluxes (i.e. fluxes with smaller allowed variability) in E. coli metabolism.
△ Less
Submitted 6 February, 2009;
originally announced February 2009.
-
Species lifetime distribution for simple models of ecologies
Authors:
S. Pigolotti,
A. Flammini,
M. Marsili,
A. Maritan
Abstract:
Interpretation of empirical results based on a taxa's lifetime distribution shows apparently conflicting results. Species' lifetime is reported to be exponentially distributed, whereas higher order taxa, such as families or genera, follow a broader distribution, compatible with power law decay. We show that both these evidences are consistent with a simple evolutionary model that does not requir…
▽ More
Interpretation of empirical results based on a taxa's lifetime distribution shows apparently conflicting results. Species' lifetime is reported to be exponentially distributed, whereas higher order taxa, such as families or genera, follow a broader distribution, compatible with power law decay. We show that both these evidences are consistent with a simple evolutionary model that does not require specific assumptions on species interaction. The model provides a zero-order description of the dynamics of ecological communities and its species lifetime distribution can be computed exactly. Different behaviors are found: an initial $t^{-3/2}$ power law, emerging from a random walk type of dynamics, which crosses over to a steeper $t^{-2}$ branching process-like regime and finally is cutoff by an exponential decay which becomes weaker and weaker as the total population increases. Sampling effects can also be taken into account and shown to be relevant: if species in the fossil record were sampled according to the Fisher log-series distribution, lifetime should be distributed according to a $t^{-1}$ power law. Such variability of behaviors in a simple model, combined with the scarcity of data available, cast serious doubts on the possibility to validate theories of evolution on the basis of species lifetime data.
△ Less
Submitted 2 November, 2005;
originally announced November 2005.
-
Diffusion, peer pressure and tailed distributions
Authors:
Fabio Cecconi,
Matteo Marsili,
Jayanth R. Banavar,
Amos Maritan
Abstract:
We present a general, physically motivated non-linear and non-local advection equation in which the diffusion of interacting random walkers competes with a local drift arising from a kind of peer pressure. We show, using a mapping to an integrable dynamical system, that on varying a parameter, the steady state behaviour undergoes a transition from the standard diffusive behavior to a localized s…
▽ More
We present a general, physically motivated non-linear and non-local advection equation in which the diffusion of interacting random walkers competes with a local drift arising from a kind of peer pressure. We show, using a mapping to an integrable dynamical system, that on varying a parameter, the steady state behaviour undergoes a transition from the standard diffusive behavior to a localized stationary state characterized by a tailed distribution. Finally, we show that recent empirical laws on economic growth can be explained as a collective phenomenon due to peer pressure interaction.
△ Less
Submitted 2 July, 2002; v1 submitted 13 February, 2002;
originally announced February 2002.
