-
Thermodynamic Bound on Energy and Negentropy Costs of Inference in Deep Neural Networks
Authors:
Alexei V. Tkachenko
Abstract:
The fundamental thermodynamic bound is derived for the energy cost of inference in Deep Neural Networks (DNNs). By applying Landauer's principle, we demonstrate that the linear operations in DNNs can, in principle, be performed reversibly, whereas the non-linear activation functions impose an unavoidable energy cost. The resulting theoretical lower bound on the inference energy is determined by th…
▽ More
The fundamental thermodynamic bound is derived for the energy cost of inference in Deep Neural Networks (DNNs). By applying Landauer's principle, we demonstrate that the linear operations in DNNs can, in principle, be performed reversibly, whereas the non-linear activation functions impose an unavoidable energy cost. The resulting theoretical lower bound on the inference energy is determined by the average number of neurons undergoing state transition for each inference. We also restate the thermodynamic bound in terms of negentropy, a metric which is more universal than energy for assessing thermodynamic cost of information processing. Concept of negentropy is further elaborated in the context of information processing in biological and engineered system as well as human intelligence. Our analysis provides insight into the physical limits of DNN efficiency and suggests potential directions for developing energy-efficient AI architectures that leverage reversible analog computing.
△ Less
Submitted 12 March, 2025;
originally announced March 2025.
-
Stochastic social behavior coupled to COVID-19 dynamics leads to waves, plateaus and an endemic state
Authors:
Alexei V. Tkachenko,
Sergei Maslov,
Tong Wang,
Ahmed Elbanna,
George N. Wong,
Nigel Goldenfeld
Abstract:
It is well recognized that population heterogeneity plays an important role in the spread of epidemics. While individual variations in social activity are often assumed to be persistent, i.e. constant in time, here we discuss the consequences of dynamic heterogeneity. By integrating the stochastic dynamics of social activity into traditional epidemiological models we demonstrate the emergence of a…
▽ More
It is well recognized that population heterogeneity plays an important role in the spread of epidemics. While individual variations in social activity are often assumed to be persistent, i.e. constant in time, here we discuss the consequences of dynamic heterogeneity. By integrating the stochastic dynamics of social activity into traditional epidemiological models we demonstrate the emergence of a new long timescale governing the epidemic in broad agreement with empirical data. Our model captures multiple features of real-life epidemics such as COVID-19, including prolonged plateaus and multiple waves, which are transiently suppressed due to the dynamic nature of social activity. The existence of the long timescale due to the interplay between epidemic and social dynamics provides a unifying picture of how a fast-paced epidemic typically will transition to the endemic state.
△ Less
Submitted 19 February, 2021; v1 submitted 28 January, 2021;
originally announced January 2021.
-
Time-dependent heterogeneity leads to transient suppression of the COVID-19 epidemic, not herd immunity
Authors:
Alexei V. Tkachenko,
Sergei Maslov,
Ahmed Elbanna,
George N. Wong,
Zachary J. Weiner,
Nigel Goldenfeld
Abstract:
Epidemics generally spread through a succession of waves that reflect factors on multiple timescales. On short timescales, super-spreading events lead to burstiness and overdispersion, while long-term persistent heterogeneity in susceptibility is expected to lead to a reduction in the infection peak and the herd immunity threshold (HIT). Here, we develop a general approach to encompass both timesc…
▽ More
Epidemics generally spread through a succession of waves that reflect factors on multiple timescales. On short timescales, super-spreading events lead to burstiness and overdispersion, while long-term persistent heterogeneity in susceptibility is expected to lead to a reduction in the infection peak and the herd immunity threshold (HIT). Here, we develop a general approach to encompass both timescales, including time variations in individual social activity, and demonstrate how to incorporate them phenomenologically into a wide class of epidemiological models through parameterization. We derive a non-linear dependence of the effective reproduction number Re on the susceptible population fraction S. We show that a state of transient collective immunity (TCI) emerges well below the HIT during early, high-paced stages of the epidemic. However, this is a fragile state that wanes over time due to changing levels of social activity, and so the infection peak is not an indication of herd immunity: subsequent waves can and will emerge due to behavioral changes in the population, driven (e.g.) by seasonal factors. Transient and long-term levels of heterogeneity are estimated by using empirical data from the COVID-19 epidemic as well as from real-life face-to-face contact networks. These results suggest that the hardest-hit areas, such as NYC, have achieved TCI following the first wave of the epidemic, but likely remain below the long-term HIT. Thus, in contrast to some previous claims, these regions can still experience subsequent waves.
△ Less
Submitted 29 January, 2021; v1 submitted 10 August, 2020;
originally announced August 2020.
-
Modeling COVID-19 dynamics in Illinois under non-pharmaceutical interventions
Authors:
George N. Wong,
Zachary J. Weiner,
Alexei V. Tkachenko,
Ahmed Elbanna,
Sergei Maslov,
Nigel Goldenfeld
Abstract:
We present modeling of the COVID-19 epidemic in Illinois, USA, capturing the implementation of a Stay-at-Home order and scenarios for its eventual release. We use a non-Markovian age-of-infection model that is capable of handling long and variable time delays without changing its model topology. Bayesian estimation of model parameters is carried out using Markov Chain Monte Carlo (MCMC) methods. T…
▽ More
We present modeling of the COVID-19 epidemic in Illinois, USA, capturing the implementation of a Stay-at-Home order and scenarios for its eventual release. We use a non-Markovian age-of-infection model that is capable of handling long and variable time delays without changing its model topology. Bayesian estimation of model parameters is carried out using Markov Chain Monte Carlo (MCMC) methods. This framework allows us to treat all available input information, including both the previously published parameters of the epidemic and available local data, in a uniform manner. To accurately model deaths as well as demand on the healthcare system, we calibrate our predictions to total and in-hospital deaths as well as hospital and ICU bed occupancy by COVID-19 patients. We apply this model not only to the state as a whole but also its sub-regions in order to account for the wide disparities in population size and density. Without prior information on non-pharmaceutical interventions (NPIs), the model independently reproduces a mitigation trend closely matching mobility data reported by Google and Unacast. Forward predictions of the model provide robust estimates of the peak position and severity and also enable forecasting the regional-dependent results of releasing Stay-at-Home orders. The resulting highly constrained narrative of the epidemic is able to provide estimates of its unseen progression and inform scenarios for sustainable monitoring and control of the epidemic.
△ Less
Submitted 15 June, 2020; v1 submitted 3 June, 2020;
originally announced June 2020.
-
Onset of natural selection in auto-catalytic heteropolymers
Authors:
Alexei V. Tkachenko,
Sergei Maslov
Abstract:
Reduction of information entropy along with ever-increasing complexity are among the key signatures of living matter. Understanding the onset of such behavior in early prebiotic world is essential for solving the problem of origins of life. To elucidate this transition, we study a theoretical model of information-storing heteropolymers capable of template-assisted ligation and subjected to cyclic…
▽ More
Reduction of information entropy along with ever-increasing complexity are among the key signatures of living matter. Understanding the onset of such behavior in early prebiotic world is essential for solving the problem of origins of life. To elucidate this transition, we study a theoretical model of information-storing heteropolymers capable of template-assisted ligation and subjected to cyclic non-equilibrium driving forces. We discover that this simple physical system undergoes a spontaneous reduction of the information entropy due to the competition of chains for constituent monomers. This natural-selection-like process ultimately results in the survival of a limited subset of polymer sequences. Importantly, the number of surviving sequences remains exponentially large, thus opening up the possibility of further increase in complexity due to Darwinian evolution. We also propose potential experimental implementations of our model using either biopolymers or artificial nano-structures.
△ Less
Submitted 17 October, 2017;
originally announced October 2017.
-
Onset of autocatalysis of information-coding polymers
Authors:
Alexei V. Tkachenko,
Sergei Maslov
Abstract:
Self-replicating systems based on information-coding polymers are of crucial importance in biology. They also recently emerged as a paradigm in material design on nano- and micro-scales. We present a general theoretical and numerical analysis of the problem of spontaneous emergence of autocatalysis for heteropolymers capable of template-assisted ligation driven by cyclic changes in the environment…
▽ More
Self-replicating systems based on information-coding polymers are of crucial importance in biology. They also recently emerged as a paradigm in material design on nano- and micro-scales. We present a general theoretical and numerical analysis of the problem of spontaneous emergence of autocatalysis for heteropolymers capable of template-assisted ligation driven by cyclic changes in the environment. Our central result is the existence of the first order transition between the regime dominated by free monomers and that with a self-sustaining population of sufficiently long chains. We provide a simple, mathematically tractable model supported by numerical simulations, which predicts the distribution of chain lengths and the onset of autocatalysis in terms of the overall monomer concentration and two fundamental rate constants. Another key result of our study is the emergence of the kinetically-limited optimal overlap length between a template and each of its two substrates. The template-assisted ligation allows for heritable transmission of the information encoded in chain sequences thus opening up the possibility of long-term memory and evolvability in such systems.
△ Less
Submitted 14 July, 2015; v1 submitted 12 May, 2014;
originally announced May 2014.
-
A Generalized Theory of DNA Looping and Cyclization
Authors:
David P. Wilson,
Alexei V. Tkachenko,
Jens-Christian Meiners
Abstract:
We have developed a generalized semi-analytic approach for efficiently computing cyclization and looping $J$ factors of DNA under arbitrary binding constraints. Many biological systems involving DNA-protein interactions impose precise boundary conditions on DNA, which necessitates a treatment beyond the Shimada-Yamakawa model for ring cyclization. Our model allows for DNA to be treated as a hete…
▽ More
We have developed a generalized semi-analytic approach for efficiently computing cyclization and looping $J$ factors of DNA under arbitrary binding constraints. Many biological systems involving DNA-protein interactions impose precise boundary conditions on DNA, which necessitates a treatment beyond the Shimada-Yamakawa model for ring cyclization. Our model allows for DNA to be treated as a heteropolymer with sequence-dependent intrinsic curvature and stiffness. In this framework, we independently compute enthlapic and entropic contributions to the $J$ factor and show that even at small length scales $(\sim \ell_{p})$ entropic effects are significant. We propose a simple analytic formula to describe our numerical results for a homogenous DNA in planar loops, which can be used to predict experimental cyclization and loop formation rates as a function of loop size and binding geometry. We also introduce an effective torsional persistence length that describes the coupling between twist and bending of DNA when looped.
△ Less
Submitted 18 November, 2009;
originally announced November 2009.
-
Kinetic limitations of cooperativity based drug delivery systems
Authors:
Nicholas A. Licata,
Alexei V. Tkachenko
Abstract:
We study theoretically a novel drug delivery system that utilizes the overexpression of certain proteins in cancerous cells for cell specific chemotherapy. The system consists of dendrimers conjugated with "keys" (ex: folic acid) which "key-lock" bind to particular cell membrane proteins (ex: folate receptor). The increased concentration of "locks" on the surface leads to a longer residence time…
▽ More
We study theoretically a novel drug delivery system that utilizes the overexpression of certain proteins in cancerous cells for cell specific chemotherapy. The system consists of dendrimers conjugated with "keys" (ex: folic acid) which "key-lock" bind to particular cell membrane proteins (ex: folate receptor). The increased concentration of "locks" on the surface leads to a longer residence time for the dendrimer and greater incorporation into the cell. Cooperative binding of the nanocomplexes leads to an enhancement of cell specificity. However, both our theory and detailed analysis of in-vitro experiments indicate that the degree of cooperativity is kinetically limited. We demonstrate that cooperativity and hence the specificity to particular cell type can be increased by making the strength of individual bonds weaker, and suggest a particular implementation of this idea. The implications of the work for optimizing the design of drug delivery vehicles are discussed.
△ Less
Submitted 7 March, 2008; v1 submitted 17 August, 2007;
originally announced August 2007.
-
Elasticity of strongly stretched ssDNA
Authors:
Alexei V. Tkachenko
Abstract:
We present a simple model which describes elastic response of single-stranded DNA (ssDNA) to stretching, including the regime of very high force (up to 1000 pN). ssDNA is modelled as a discreet persistent chain, whose ground state is a zigzag rather than a straight line configuration. This mimics the underlying molecular architecture and helps to explain the experimentally observed staturation o…
▽ More
We present a simple model which describes elastic response of single-stranded DNA (ssDNA) to stretching, including the regime of very high force (up to 1000 pN). ssDNA is modelled as a discreet persistent chain, whose ground state is a zigzag rather than a straight line configuration. This mimics the underlying molecular architecture and helps to explain the experimentally observed staturation of the stretching curve at very high force.
△ Less
Submitted 11 March, 2007;
originally announced March 2007.
-
Role of boundary constraints in DNA cyclization
Authors:
Alexei V. Tkachenko
Abstract:
We modify the classical Shimada-Yamakawa theory of DNA looping by generalizing the form of boundary constraints. This generalization is important in the context of DNA cyclization experiments since it mimics the reduced local rigidity of the "nicked" DNA loop. Our results indicate that the non-trivial boundary constraints may be responsible for the existing dramatic discrepancy between various D…
▽ More
We modify the classical Shimada-Yamakawa theory of DNA looping by generalizing the form of boundary constraints. This generalization is important in the context of DNA cyclization experiments since it mimics the reduced local rigidity of the "nicked" DNA loop. Our results indicate that the non-trivial boundary constraints may be responsible for the existing dramatic discrepancy between various DNA cyclization experiments. The developed effective Hamiltonian method may be further extended to a broad class of DNA looping problems.
△ Less
Submitted 11 March, 2007;
originally announced March 2007.
-
Effects of Kinks on DNA Elasticity
Authors:
Yuri O. Popov,
Alexei V. Tkachenko
Abstract:
We study the elastic response of a worm-like polymer chain with reversible kink-like structural defects. This is a generic model for (a) the double-stranded DNA with sharp bends induced by binding of certain proteins, and (b) effects of trans-gauche rotations in the backbone of the single-stranded DNA. The problem is solved both analytically and numerically by generalizing the well-known analogy…
▽ More
We study the elastic response of a worm-like polymer chain with reversible kink-like structural defects. This is a generic model for (a) the double-stranded DNA with sharp bends induced by binding of certain proteins, and (b) effects of trans-gauche rotations in the backbone of the single-stranded DNA. The problem is solved both analytically and numerically by generalizing the well-known analogy to the Quantum Rotator. In the small stretching force regime, we find that the persistence length is renormalized due to the presence of the kinks. In the opposite regime, the response to the strong stretching is determined solely by the bare persistence length with exponential corrections due to the ``ideal gas of kinks''. This high-force behavior changes significantly in the limit of high bending rigidity of the chain. In that case, the leading corrections to the mechanical response are likely to be due to the formation of multi-kink structures, such as kink pairs.
△ Less
Submitted 28 February, 2005; v1 submitted 22 October, 2004;
originally announced October 2004.
-
Unfolding and unzipping of single-stranded DNA by stretching
Authors:
Alexei V. Tkachenko
Abstract:
We present a theoretical study of single-stranded DNA under stretching. Within the proposed framework, the effects of basepairing on the mechanical response of the molecule can be studied in combination with an arbitrary underlying model of chain elasticity. In a generic case, we show that the stretching curve of ssDNA exhibits two distinct features: the second-order "unfolding" phase transition…
▽ More
We present a theoretical study of single-stranded DNA under stretching. Within the proposed framework, the effects of basepairing on the mechanical response of the molecule can be studied in combination with an arbitrary underlying model of chain elasticity. In a generic case, we show that the stretching curve of ssDNA exhibits two distinct features: the second-order "unfolding" phase transition, and a sharp crossover, reminiscent of the first-order "unzipping" transition in dsDNA. We apply the theory to the particular cases of Worm-like Chain (WLC) and Freely-Joint Chain (FJC) models, and discuss the universal and model--dependent features of the mechanical response of ssDNA. In particular, we show that variation of the width of the unzipping crossover with interaction strength is very sensitive to the energetics of hairpin loops. This opens a new way of testing the elastic properties of ssDNA.
△ Less
Submitted 26 July, 2004; v1 submitted 10 April, 2003;
originally announced April 2003.
-
Electrostatic effects in DNA stretching
Authors:
Alexei V. Tkachenko
Abstract:
The response of a semiflexible polyelectrolyte chain to stretching in the regimes of moderate and weak screening is studied theoretically, with a special focus on DNA experiments. By using the nonlinear Poisson--Boltzmann description of electrostatic self--interactions of the chain, we explicitly demonstrate the applicability of the concept of effective charge to certain aspects of the problem.…
▽ More
The response of a semiflexible polyelectrolyte chain to stretching in the regimes of moderate and weak screening is studied theoretically, with a special focus on DNA experiments. By using the nonlinear Poisson--Boltzmann description of electrostatic self--interactions of the chain, we explicitly demonstrate the applicability of the concept of effective charge to certain aspects of the problem. This charge can be extracted from the far--field asymptotic behavior of electrostatic potential of the fully aligned chain. Surprisingly, in terms of the effective charge, the electrostatically renormalized persistence length can be formally described by the classical Odijk-Skolnick-Fixman (OSF) formula, whose domain of applicability is limited to linearized Debye Huckel (DH) approximation. However, the short--length behavior of the chain in the nonlinear regime deviates from the of DH--based result, even upon the charge renormalization. This difference is revealed in calculated stretching curves of strongly charged DNA, and our results are in good agreement with the recent experiments. In the limit of weak screening we predict the elastic response to have a distinctive two-stage character, with a peculiar intermediate "unstretchable" regime.
△ Less
Submitted 27 May, 2006; v1 submitted 4 March, 2003;
originally announced March 2003.
-
With a little help of DNA: Morphological diversity of colloidal self-assembly
Authors:
Alexei V. Tkachenko
Abstract:
We study theoretically a binary system in which an attraction of unlike particles is combined with a type-independent soft core repulsion.
The possible experimental implementation of the system is a mixture of DNA-covered colloids, in which both the repulsion and the attraction may be induced by DNA solution. The system is shown to exhibit surprisingly diverse and unusual morphologies. Among t…
▽ More
We study theoretically a binary system in which an attraction of unlike particles is combined with a type-independent soft core repulsion.
The possible experimental implementation of the system is a mixture of DNA-covered colloids, in which both the repulsion and the attraction may be induced by DNA solution. The system is shown to exhibit surprisingly diverse and unusual morphologies. Among them are the diamond lattice and the membrane phase with in-plane square order, a striking example of spontaneous compactification.
△ Less
Submitted 21 June, 2001;
originally announced June 2001.
