Showing 1–8 of 8 results for author: Holmes, C M

Optimization and variability can coexist

Authors: Marianne Bauer, William Bialek, Chase Goddard, Caroline M. Holmes, Kamesh Krishnamurthy, Stephanie E. Palmer, Rich Pang, David J. Schwab, Lee Susman

Abstract: Many biological systems perform close to their physical limits, but promoting this optimality to a general principle seems to require implausibly fine tuning of parameters. Using examples from a wide range of systems, we show that this intuition is wrong. Near an optimum, functional performance depends on parameters in a "sloppy'' way, with some combinations of parameters being only weakly constra… ▽ More Many biological systems perform close to their physical limits, but promoting this optimality to a general principle seems to require implausibly fine tuning of parameters. Using examples from a wide range of systems, we show that this intuition is wrong. Near an optimum, functional performance depends on parameters in a "sloppy'' way, with some combinations of parameters being only weakly constrained. Absent any other constraints, this predicts that we should observe widely varying parameters, and we make this precise: the entropy in parameter space can be extensive even if performance on average is very close to optimal. This removes a major objection to optimization as a general principle, and rationalizes the observed variability. △ Less

Submitted 29 May, 2025; originally announced May 2025.

Inferring genotype-phenotype maps using attention models

Authors: Krishna Rijal, Caroline M. Holmes, Samantha Petti, Gautam Reddy, Michael M. Desai, Pankaj Mehta

Abstract: Predicting phenotype from genotype is a central challenge in genetics. Traditional approaches in quantitative genetics typically analyze this problem using methods based on linear regression. These methods generally assume that the genetic architecture of complex traits can be parameterized in terms of an additive model, where the effects of loci are independent, plus (in some cases) pairwise epis… ▽ More Predicting phenotype from genotype is a central challenge in genetics. Traditional approaches in quantitative genetics typically analyze this problem using methods based on linear regression. These methods generally assume that the genetic architecture of complex traits can be parameterized in terms of an additive model, where the effects of loci are independent, plus (in some cases) pairwise epistatic interactions between loci. However, these models struggle to analyze more complex patterns of epistasis or subtle gene-environment interactions. Recent advances in machine learning, particularly attention-based models, offer a promising alternative. Initially developed for natural language processing, attention-based models excel at capturing context-dependent interactions and have shown exceptional performance in predicting protein structure and function. Here, we apply attention-based models to quantitative genetics. We analyze the performance of this attention-based approach in predicting phenotype from genotype using simulated data across a range of models with increasing epistatic complexity, and using experimental data from a recent quantitative trait locus mapping study in budding yeast. We find that our model demonstrates superior out-of-sample predictions in epistatic regimes compared to standard methods. We also explore a more general multi-environment attention-based model to jointly analyze genotype-phenotype maps across multiple environments and show that such architectures can be used for "transfer learning" - predicting phenotypes in novel environments with limited training data. △ Less

Submitted 14 April, 2025; originally announced April 2025.

Emergence of local irreversibility in complex interacting systems

Authors: Christopher W. Lynn, Caroline M. Holmes, William Bialek, David J. Schwab

Abstract: Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components:… ▽ More Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components: an independent term reflecting the dynamics of individual elements and an interaction term driven by the dependencies among elements. Adapting tools from non--equilibrium physics, we further decompose the interaction term into contributions from pairs of elements, triplets, and higher--order terms. We illustrate our methods on models of cellular sensing and logical computations, as well as on patterns of neural activity in the retina as it responds to visual inputs. We find that neural activity can define the arrow of time even when the visual inputs do not, and that the dominant contribution to this breaking of detailed balance comes from interactions among pairs of neurons. △ Less

Submitted 3 June, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: 16 pages, 10 figures

Decomposing the local arrow of time in interacting systems

Authors: Christopher W. Lynn, Caroline M. Holmes, William Bialek, David J. Schwab

Abstract: We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non--negative terms contributed by correlations among pairs, triplets, and higher--order combinations of… ▽ More We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non--negative terms contributed by correlations among pairs, triplets, and higher--order combinations of variables. We illustrate this decomposition on simple models of noisy logical computations, and then apply it to the analysis of patterns of neural activity in the retina as it responds to complex dynamic visual scenes. We find that neural activity breaks detailed balance even when the visual inputs do not, and that this irreversibility arises primarily from interactions between pairs of neurons. △ Less

Submitted 3 June, 2022; v1 submitted 29 December, 2021; originally announced December 2021.

Comments: 6 pages, 3 figures

A simple regulatory architecture allows learning the statistical structure of a changing environment

Authors: Stefan Landmann, Caroline M. Holmes, Mikhail Tikhonov

Abstract: Bacteria live in environments that are continuously fluctuating and changing. Exploiting any predictability of such fluctuations can lead to an increased fitness. On longer timescales bacteria can "learn" the structure of these fluctuations through evolution. However, on shorter timescales, inferring the statistics of the environment and acting upon this information would need to be accomplished b… ▽ More Bacteria live in environments that are continuously fluctuating and changing. Exploiting any predictability of such fluctuations can lead to an increased fitness. On longer timescales bacteria can "learn" the structure of these fluctuations through evolution. However, on shorter timescales, inferring the statistics of the environment and acting upon this information would need to be accomplished by physiological mechanisms. Here, we use a model of metabolism to show that a simple generalization of a common regulatory motif (end-product inhibition) is sufficient both for learning continuous-valued features of the statistical structure of the environment and for translating this information into predictive behavior; moreover, it accomplishes these tasks near-optimally. We discuss plausible genetic circuits that could instantiate the mechanism we describe, including one similar to the architecture of two-component signaling, and argue that the key ingredients required for such predictive behavior are readily accessible to bacteria. △ Less

Submitted 31 December, 2020; originally announced January 2021.

Comments: 9 pages, 5 figures + Supplementary information

Estimation of mutual information for real-valued data with error bars and controlled bias

Authors: Caroline M. Holmes, Ilya Nemenman

Abstract: Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good estimators provably do not exist. Kraskov et al. (PRE, 2004) introduced a successful mutual information estimation approach based on the statistics of distances betwee… ▽ More Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good estimators provably do not exist. Kraskov et al. (PRE, 2004) introduced a successful mutual information estimation approach based on the statistics of distances between neighboring data points, which empirically works for a wide class of underlying probability distributions. Here we improve this estimator by (i) expanding its range of applicability, and by providing (ii) a self-consistent way of verifying the absence of bias, (iii) a method for estimation of its variance, and (iv) a criterion for choosing the values of the free parameter of the estimator. We demonstrate the performance of our estimator on synthetic data sets, as well as on neurophysiological and systems biology data sets. △ Less

Submitted 21 March, 2019; originally announced March 2019.

Comments: 10 pages, 8 figures

Journal ref: Phys. Rev. E 100, 022404 (2019)

Increased adaptability to rapid environmental change can more than make up for the two-fold cost of males

Authors: Caroline M. Holmes, Ilya Nemenman, Daniel B. Weissman

Abstract: The famous "two-fold cost of sex" is really the cost of anisogamy -- why should females mate with males who do not contribute resources to offspring, rather than isogamous partners who contribute equally? In typical anisogamous populations, a single very fit male can have an enormous number of offspring, far larger than is possible for any female or isogamous individual. If the sexual selection on… ▽ More The famous "two-fold cost of sex" is really the cost of anisogamy -- why should females mate with males who do not contribute resources to offspring, rather than isogamous partners who contribute equally? In typical anisogamous populations, a single very fit male can have an enormous number of offspring, far larger than is possible for any female or isogamous individual. If the sexual selection on males aligns with the natural selection on females, anisogamy thus allows much more rapid adaptation via super-successful males. We show via simulations that this effect can be sufficient to overcome the two-fold cost and maintain anisogamy against isogamy in populations adapting to environmental change. The key quantity is the variance in male fitness -- if this exceeds what is possible in an isogamous population, anisogamous populations can win out in direct competition by adapting faster. △ Less

Submitted 6 June, 2018; originally announced June 2018.

Comments: 15 pages, 3 figures

Motor control by precisely timed spike patterns

Authors: Kyle H. Srivastava, Caroline M. Holmes, Michiel Vellema, Andrea Pack, Coen P. H. Elemans, Ilya Nemenman, Samuel J. Sober

Abstract: A fundamental problem in neuroscience is to understand how sequences of action potentials ("spikes") encode information about sensory signals and motor outputs. Although traditional theories of neural coding assume that information is conveyed by the total number of spikes fired (spike rate), recent studies of sensory and motor activity have shown that far more information is carried by the millis… ▽ More A fundamental problem in neuroscience is to understand how sequences of action potentials ("spikes") encode information about sensory signals and motor outputs. Although traditional theories of neural coding assume that information is conveyed by the total number of spikes fired (spike rate), recent studies of sensory and motor activity have shown that far more information is carried by the millisecond-scale timing patterns of action potentials (spike timing). However, it is unknown whether or how subtle differences in spike timing drive differences in perception or behavior, leaving it unclear whether the information carried by spike timing actually plays a causal role in brain function. Here we demonstrate how a precise spike timing code is read out downstream by the muscles to control behavior. We provide both correlative and causal evidence to show that the nervous system uses millisecond-scale variations in the timing of spikes within multi-spike patterns to regulate a relatively simple behavior - respiration in the Bengalese finch, a songbird. These findings suggest that a fundamental assumption of current theories of motor coding requires revision, and that significant improvements in applications, such as neural prosthetic devices, can be achieved by using precise spike timing information. △ Less

Submitted 30 May, 2016; v1 submitted 29 May, 2016; originally announced May 2016.

Comments: 48 pages, 16 figures