-
The physical observer in a Szilard engine with uncertainty
Authors:
Dorian Daimer,
Susanne Still
Abstract:
Information engines model ``Maxwell's demon" mechanistically. However, the demon's strategy is pre-described by an external experimenter, and information engines are conveniently designed such that observables contain complete information about variables pertinent to work extraction. In real world scenarios, it is more realistic to encounter partial observability, which forces the physical observe…
▽ More
Information engines model ``Maxwell's demon" mechanistically. However, the demon's strategy is pre-described by an external experimenter, and information engines are conveniently designed such that observables contain complete information about variables pertinent to work extraction. In real world scenarios, it is more realistic to encounter partial observability, which forces the physical observer, an integral part of the information engine, to make inferences from incomplete knowledge. Here, we use the fact that an algorithm for computing optimal strategies can be directly derived from maximizing overall engine work output. For a simple binary decision problem, we discover interesting optimal strategies that differ notably from naive coarse graining. They inspire a model class of simple, yet compelling, parameterized soft partitionings of the observable.
△ Less
Submitted 12 July, 2024; v1 submitted 19 September, 2023;
originally announced September 2023.
-
Information engine in a nonequilibrium bath
Authors:
Tushar K. Saha,
Jannik Ehrich,
Momčilo Gavrilov,
Susanne Still,
David A. Sivak,
John Bechhoefer
Abstract:
Information engines can convert thermal fluctuations of a bath at temperature $T$ into work at rates of order $k_\mathrm{B}T$ per relaxation time of the system. We show experimentally that such engines, when in contact with a bath that is out of equilibrium, can extract much more work. We place a heavy, micron-scale bead in a harmonic potential that ratchets up to capture favorable fluctuations. A…
▽ More
Information engines can convert thermal fluctuations of a bath at temperature $T$ into work at rates of order $k_\mathrm{B}T$ per relaxation time of the system. We show experimentally that such engines, when in contact with a bath that is out of equilibrium, can extract much more work. We place a heavy, micron-scale bead in a harmonic potential that ratchets up to capture favorable fluctuations. Adding a fluctuating electric field increases work extraction up to ten times, limited only by the strength of applied field. Our results connect Maxwell's demon with energy harvesting and an estimate of efficiency shows that information engines in nonequilibrium baths can greatly outperform conventional engines.
△ Less
Submitted 30 July, 2022;
originally announced August 2022.
-
Energetic cost of feedback control
Authors:
Jannik Ehrich,
Susanne Still,
David A. Sivak
Abstract:
Successful feedback control of small systems allows for the rectification of thermal fluctuations, converting them into useful energy; however, control itself requires work. This paper emphasizes the fact that the controller is a physical entity interacting with the feedback-controlled system. For a specifically designed class of controllers, reciprocal interactions become nonreciprocal due to lar…
▽ More
Successful feedback control of small systems allows for the rectification of thermal fluctuations, converting them into useful energy; however, control itself requires work. This paper emphasizes the fact that the controller is a physical entity interacting with the feedback-controlled system. For a specifically designed class of controllers, reciprocal interactions become nonreciprocal due to large timescale separation, which considerably simplifies the situation. We introduce a minimally dissipative controller model, illustrating the findings using a simple example. We find that the work required to run the controller must at least compensate for the decrease in entropy due to the control operation.
△ Less
Submitted 3 May, 2023; v1 submitted 21 June, 2022;
originally announced June 2022.
-
Partially Observable Szilard Engines
Authors:
Susanne Still,
Dorian Daimer
Abstract:
Leo Szilard pointed out that Maxwell's demon can be replaced by machinery, thereby laying the foundation for understanding the physical nature of information. Szilard's information engine still serves as a canonical example after almost a hundred years, despite recent significant growth of the area. The role the demon plays can be reduced to mapping observable data to a meta-stable memory, which i…
▽ More
Leo Szilard pointed out that Maxwell's demon can be replaced by machinery, thereby laying the foundation for understanding the physical nature of information. Szilard's information engine still serves as a canonical example after almost a hundred years, despite recent significant growth of the area. The role the demon plays can be reduced to mapping observable data to a meta-stable memory, which is utilized to extract work. While Szilard showed that the map can be implemented mechanistically, it was chosen a priori. The choice of how to construct a meaningful memory constitutes the demon's intelligence. Recently, it was shown that this can be automated as well. To that end, generalized, partially observable information engines were introduced, providing a basis for understanding the physical nature of information processing. Partial observability is ubiquitous in real world systems which have limited sensor types and information acquisition bandwidths. Generalized information engines can run work extraction at a different temperature, T' > T, from the memory forming process. This enables the combined treatment of heat engines and information engines. We study the physical characteristics of intelligent observers by introducing a canonical model that displays physical richness, despite its simplicity. A minor change to Szilard's engine - inserting the divider at an angle - results in a family of partially observable Szilard engines. Their analysis shows how the demon's intelligence can be automated. For each angle, and for each value of T'/T, an optimal memory can be found, enabling the engine to run with minimal dissipation. Those optimal memories are probabilistic maps, computed algorithmically. We discuss how they can be implemented with a simple physical system, characterize their performance, and compare their quality to that of naive, deterministic quantizations of the observable.
△ Less
Submitted 23 August, 2022; v1 submitted 26 March, 2021;
originally announced March 2021.
-
Optimal work extraction and mutual information in a generalized Szilárd engine
Authors:
Juyong Song,
Susanne Still,
Rafael Díaz Hernández Rojas,
Isaac Pérez Castillo,
Matteo Marsili
Abstract:
A 1929 Gedankenexperiment proposed by Szilárd, often referred to as "Szilárd's engine", has served as a foundation for computing fundamental thermodynamic bounds to information processing. While Szilárd's original box could be partitioned into two halves and contains one gas molecule, we calculate here the maximal average work that can be extracted in a system with $N$ particles and $q$ partitions…
▽ More
A 1929 Gedankenexperiment proposed by Szilárd, often referred to as "Szilárd's engine", has served as a foundation for computing fundamental thermodynamic bounds to information processing. While Szilárd's original box could be partitioned into two halves and contains one gas molecule, we calculate here the maximal average work that can be extracted in a system with $N$ particles and $q$ partitions, given an observer which counts the molecules in each partition, and given a work extraction mechanism that is limited to pressure equalization. We find that the average extracted work is proportional to the mutual information between the one-particle position and the vector containing the counts of how many particles are in each partition. We optimize this quantity over the initial locations of the dividing walls, and find that there exists a critical number of particles $N^{\star}(q)$ below which the extracted work is maximized by a symmetric configuration of the $q$ partitions, and above which the optimal partitioning is asymmetric. Overall, the average extracted work is maximized for a number of particles $\hat{N}(q)<N^{\star}(q)$, with a symmetric partition. We calculate asymptotic values for $N\rightarrow \infty$.
△ Less
Submitted 18 May, 2021; v1 submitted 9 October, 2019;
originally announced October 2019.
-
Physical Limitations of Work Extraction from Temporal Correlations
Authors:
Elan Stopnitzky,
Susanne Still,
Thomas E. Ouldridge,
Lee Altenberg
Abstract:
Recently proposed information-exploiting systems designed to extract work from a single heat bath utilize temporal correlations on an input tape. We study how enforcing time-continuous dynamics, which is necessary to ensure the device is physically realizable, constrains possible designs and drastically diminishes efficiency. We show that these problems can be circumvented by means of applying an…
▽ More
Recently proposed information-exploiting systems designed to extract work from a single heat bath utilize temporal correlations on an input tape. We study how enforcing time-continuous dynamics, which is necessary to ensure the device is physically realizable, constrains possible designs and drastically diminishes efficiency. We show that these problems can be circumvented by means of applying an external, time-varying protocol. This turns the device from a "passive", free-running machine into an "actively" driven one.
△ Less
Submitted 18 August, 2018;
originally announced August 2018.
-
Thermodynamic cost and benefit of memory
Authors:
Susanne Still
Abstract:
This letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are allowed to use temperature differences. It is shown that the retention of irrelevant information limits efficiency. A data representation strategy is derived from op…
▽ More
This letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are allowed to use temperature differences. It is shown that the retention of irrelevant information limits efficiency. A data representation strategy is derived from optimizing a fundamental physical limit to information processing: minimizing the lower bound on dissipation leads to a data compression method that maximally retains relevant, predictive, information. In that sense, predictive inference emerges as the strategy that least precludes energy efficiency.
△ Less
Submitted 3 October, 2019; v1 submitted 29 April, 2017;
originally announced May 2017.
-
Marginal and Conditional Second Laws of Thermodynamics
Authors:
Gavin E. Crooks,
Susanne E. Still
Abstract:
We consider the entropy production of a strongly coupled bipartite system. The total entropy production can be partitioned into various components, which we use to define local versions of the Second Law that are valid without the usual idealization of weak coupling. The key insight is that causal intervention offers a way to identify those parts of the entropy production that result from feedback…
▽ More
We consider the entropy production of a strongly coupled bipartite system. The total entropy production can be partitioned into various components, which we use to define local versions of the Second Law that are valid without the usual idealization of weak coupling. The key insight is that causal intervention offers a way to identify those parts of the entropy production that result from feedback between the sub-systems. From this the central relations describing the thermodynamics of strongly coupled systems follow in a few lines.
△ Less
Submitted 2 June, 2018; v1 submitted 14 November, 2016;
originally announced November 2016.
-
The thermodynamics of prediction
Authors:
Susanne Still,
David A. Sivak,
Anthony J. Bell,
Gavin E. Crooks
Abstract:
A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations, and a fraction of this information is predictive of future ones. The remaining nonpredictive information reflects model complexity that does not improve predictiv…
▽ More
A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations, and a fraction of this information is predictive of future ones. The remaining nonpredictive information reflects model complexity that does not improve predictive power, and thus represents the ineffectiveness of the model. We expose the fundamental equivalence between this model inefficiency and thermodynamic inefficiency, measured by dissipation. Our results hold arbitrarily far from thermodynamic equilibrium and are applicable to a wide range of systems, including biomolecular machines. They highlight a profound connection between the effective use of information and efficient thermodynamic operation: any system constructed to keep memory about its environment and to operate with maximal energetic efficiency has to be predictive.
△ Less
Submitted 5 October, 2012; v1 submitted 15 March, 2012;
originally announced March 2012.
-
Optimal Causal Inference: Estimating Stored Information and Approximating Causal Architecture
Authors:
Susanne Still,
James P. Crutchfield,
Christopher J. Ellison
Abstract:
We introduce an approach to inferring the causal architecture of stochastic dynamical systems that extends rate distortion theory to use causal shielding---a natural principle of learning. We study two distinct cases of causal inference: optimal causal filtering and optimal causal estimation.
Filtering corresponds to the ideal case in which the probability distribution of measurement sequences i…
▽ More
We introduce an approach to inferring the causal architecture of stochastic dynamical systems that extends rate distortion theory to use causal shielding---a natural principle of learning. We study two distinct cases of causal inference: optimal causal filtering and optimal causal estimation.
Filtering corresponds to the ideal case in which the probability distribution of measurement sequences is known, giving a principled method to approximate a system's causal structure at a desired level of representation. We show that, in the limit in which a model complexity constraint is relaxed, filtering finds the exact causal architecture of a stochastic dynamical system, known as the causal-state partition. From this, one can estimate the amount of historical information the process stores. More generally, causal filtering finds a graded model-complexity hierarchy of approximations to the causal architecture. Abrupt changes in the hierarchy, as a function of approximation, capture distinct scales of structural organization.
For nonideal cases with finite data, we show how the correct number of underlying causal states can be found by optimal causal estimation. A previously derived model complexity control term allows us to correct for the effect of statistical fluctuations in probability estimates and thereby avoid over-fitting.
△ Less
Submitted 19 August, 2010; v1 submitted 11 August, 2007;
originally announced August 2007.
-
Structure or Noise?
Authors:
Susanne Still,
James P. Crutchfield
Abstract:
We show how rate-distortion theory provides a mechanism for automated theory building by naturally distinguishing between regularity and randomness. We start from the simple principle that model variables should, as much as possible, render the future and past conditionally independent. From this, we construct an objective function for model making whose extrema embody the trade-off between a mo…
▽ More
We show how rate-distortion theory provides a mechanism for automated theory building by naturally distinguishing between regularity and randomness. We start from the simple principle that model variables should, as much as possible, render the future and past conditionally independent. From this, we construct an objective function for model making whose extrema embody the trade-off between a model's structural complexity and its predictive power. The solutions correspond to a hierarchy of models that, at each level of complexity, achieve optimal predictive power at minimal cost. In the limit of maximal prediction the resulting optimal model identifies a process's intrinsic organization by extracting the underlying causal states. In this limit, the model's complexity is given by the statistical complexity, which is known to be minimal for achieving maximum prediction. Examples show how theory building can profit from analyzing a process's causal compressibility, which is reflected in the optimal models' rate-distortion curve--the process's characteristic for optimally balancing structure and noise at different levels of representation.
△ Less
Submitted 29 June, 2008; v1 submitted 4 August, 2007;
originally announced August 2007.
-
Network information and connected correlations
Authors:
Elad Schneidman,
Susanne Still,
Michael J. Berry II,
William Bialek
Abstract:
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate…
▽ More
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.
△ Less
Submitted 15 July, 2003;
originally announced July 2003.
