Showing 1–26 of 26 results for author: Crutchfield, J P

Entropy Production by Underdamped Langevin Dynamics

Authors: Jinghao Lyu, Kyle J. Ray, James P. Crutchfield

Abstract: Entropy production (EP) is a central quantity in nonequilibrium physics as it monitors energy dissipation, irreversibility, and free energy differences during thermodynamic transformations. Estimating EP, however, is challenging both theoretically and experimentally due to limited access to the system dynamics. For overdamped Langevin dynamics and Markov jump processes it was recently proposed tha… ▽ More Entropy production (EP) is a central quantity in nonequilibrium physics as it monitors energy dissipation, irreversibility, and free energy differences during thermodynamic transformations. Estimating EP, however, is challenging both theoretically and experimentally due to limited access to the system dynamics. For overdamped Langevin dynamics and Markov jump processes it was recently proposed that, from thermodynamic uncertainty relations (TUR), short-time cumulant currents can be used to estimate EP without knowledge of the dynamics. Yet, estimation of EP in underdamped Langevin systems remains an active challenge. To address this, we derive a modified TUR that relates the statistics of two specific novel currents -- one cumulant current and one stochastic current -- to a system's EP. These two distinct but related currents are used to constrain EP in the modified TUR. One highlight is that there always exists a family of currents such that the uncertainty relations saturate, even for long-time averages and in nonsteady-state scenarios. Another is that our method only requires limited knowledge of the dynamics -- specifically, the damping-coefficient to mass ratio and the diffusion constant. This uncertainty relation allows estimating EP for both overdamped and underdamped Langevin dynamics. We validate the method numerically, through applications to several underdamped systems, to underscore the flexibility in obtaining EP in nonequilibrium Langevin systems. △ Less

Submitted 24 June, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Comments: 15 pages, 5 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/EstimateEP.htm

Whales in Space: Experiencing Aquatic Animals in Their Natural Place with the Hydroambiphone

Authors: James P. Crutchfield, David D. Dunn, Alexandra M. Jurgens

Abstract: Recording the undersea three-dimensional bioacoustic sound field in real-time promises major benefits to marine behavior studies. We describe a novel hydrophone array -- the hydroambiphone (HAP) -- that adapts ambisonic spatial-audio theory to sound propagation in ocean waters to realize many of these benefits through spatial localization and acoustic immersion. Deploying it to monitor the humpbac… ▽ More Recording the undersea three-dimensional bioacoustic sound field in real-time promises major benefits to marine behavior studies. We describe a novel hydrophone array -- the hydroambiphone (HAP) -- that adapts ambisonic spatial-audio theory to sound propagation in ocean waters to realize many of these benefits through spatial localization and acoustic immersion. Deploying it to monitor the humpback whales (Megaptera novaeangliae) of southeast Alaska demonstrates that HAP recording provides a qualitatively-improved experience of their undersea behaviors; revealing, for example, new aspects of social coordination during bubble-net feeding. On the practical side, spatialized hydrophone recording greatly reduces post-field analytical and computational challenges -- such as the "cocktail party problem" of distinguishing single sources in a complicated and crowded auditory environment -- that are common to field recordings. On the scientific side, comparing the HAP's capabilities to single-hydrophone and nonspatialized recordings yields new insights into the spatial information that allows animals to thrive in complex acoustic environments. Spatialized bioacoustics markedly improves access to the humpbacks' undersea acoustic environment and expands our appreciation of their rich vocal lives. △ Less

Submitted 27 December, 2023; originally announced December 2023.

Comments: 11 pages, 4 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/whalesinspace.htm

On Principles of Emergent Organization

Authors: Adam T. Rupe, James P. Crutchfield

Abstract: After more than a century of concerted effort, physics still lacks basic principles of spontaneous self-organization. To appreciate why, we first state the problem, outline historical approaches, and survey the present state of the physics of self-organization. This frames the particular challenges arising from mathematical intractability and the resulting need for computational approaches, as wel… ▽ More After more than a century of concerted effort, physics still lacks basic principles of spontaneous self-organization. To appreciate why, we first state the problem, outline historical approaches, and survey the present state of the physics of self-organization. This frames the particular challenges arising from mathematical intractability and the resulting need for computational approaches, as well as those arising from a chronic failure to define structure. Then, an overview of two modern mathematical formulations of organization -- intrinsic computation and evolution operators -- lays out a way to overcome these challenges. Together, the vantage point they afford shows how to account for the emergence of structured states via a statistical mechanics of systems arbitrarily far from equilibrium. The result is a constructive path forward to principles of organization that builds on mathematical identification of structure. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: 51 pages, 7 figures; https://csc.ucdavis.edu/~cmg/compmech/pubs/tsfpoo.htm

Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization

Authors: Adam Rupe, Karthik Kashinath, Nalini Kumar, James P. Crutchfield

Abstract: Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically-grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice… ▽ More Spontaneous self-organization is ubiquitous in systems far from thermodynamic equilibrium. While organized structures that emerge dominate transport properties, universal representations that identify and describe these key objects remain elusive. Here, we introduce a theoretically-grounded framework for describing emergent organization that, via data-driven algorithms, is constructive in practice. Its building blocks are spacetime lightcones that embody how information propagates across a system through local interactions. We show that predictive equivalence classes of lightcones -- local causal states -- capture organized behaviors and coherent structures in complex spatiotemporal systems. Employing an unsupervised physics-informed machine learning algorithm and a high-performance computing implementation, we demonstrate automatically discovering coherent structures in two real world domain science problems. We show that local causal states identify vortices and track their power-law decay behavior in two-dimensional fluid turbulence. We then show how to detect and track familiar extreme weather events -- hurricanes and atmospheric rivers -- and discover other novel coherent structures associated with precipitation extremes in high-resolution climate data at the grid-cell level. △ Less

Submitted 28 September, 2023; v1 submitted 25 April, 2023; originally announced April 2023.

Nonequilibrium Statistical Mechanics and Optimal Prediction of Partially-Observed Complex Systems

Authors: Adam Rupe, Velimir V. Vesselinov, James P. Crutchfield

Abstract: Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation… ▽ More Only a subset of degrees of freedom are typically accessible or measurable in real-world systems. As a consequence, the proper setting for empirical modeling is that of partially-observed systems. Notably, data-driven models consistently outperform physics-based models for systems with few observable degrees of freedom; e.g., hydrological systems. Here, we provide an operator-theoretic explanation for this empirical success. To predict a partially-observed system's future behavior with physics-based models, the missing degrees of freedom must be explicitly accounted for using data assimilation and model parametrization. Data-driven models, in contrast, employ delay-coordinate embeddings and their evolution under the Koopman operator to implicitly model the effects of the missing degrees of freedom. We describe in detail the statistical physics of partial observations underlying data-driven models using novel Maximum Entropy and Maximum Caliber measures. The resulting nonequilibrium Wiener projections applied to the Mori-Zwanzig formalism reveal how data-driven models may converge to the true dynamics of the observable degrees of freedom. Additionally, this framework shows how data-driven models infer the effects of unobserved degrees of freedom implicitly, in much the same way that physics models infer the effects explicitly. This provides a unified implicit-explicit modeling framework for predicting partially-observed systems, with hybrid physics-informed machine learning methods combining implicit and explicit aspects. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Homeostatic and Adaptive Energetics: Nonequilibrium Fluctuations Beyond Detailed Balance in Voltage-Gated Ion Channels

Authors: Mikhael T. Semaan, James P. Crutchfield

Abstract: Stochastic thermodynamics has largely succeeded in characterizing both equilibrium and far-from-equilibrium phenomena. Yet many opportunities remain for application to mesoscopic complex systems -- especially biological ones -- whose effective dynamics often violate detailed balance and whose microscopic degrees of freedom are often unknown or intractable. After reviewing excess and housekeeping e… ▽ More Stochastic thermodynamics has largely succeeded in characterizing both equilibrium and far-from-equilibrium phenomena. Yet many opportunities remain for application to mesoscopic complex systems -- especially biological ones -- whose effective dynamics often violate detailed balance and whose microscopic degrees of freedom are often unknown or intractable. After reviewing excess and housekeeping energetics -- the adaptive and homeostatic components of a system's dissipation -- we extend stochastic thermodynamics with a trajectory class fluctuation theorem for nonequilibrium steady-state, nondetailed-balanced complex systems. We then take up the neurobiological examples of voltage-gated sodium and potassium ion channels to apply and illustrate the theory, elucidating their nonequilibrium behavior under a biophysically plausible action potential drive. These results uncover challenges for future experiments and highlight the progress possible understanding the thermodynamics of complex systems -- without exhaustive knowledge of every underlying degree of freedom. △ Less

Submitted 6 November, 2022; v1 submitted 25 February, 2022; originally announced February 2022.

Comments: 17 pages, 9 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/toic.htm

Gigahertz Sub-Landauer Momentum Computing

Authors: Kyle J. Ray, James P. Crutchfield

Abstract: We introduce a fast and highly-efficient physically-realizable bit swap. Employing readily available and scalable Josephson junction microtechnology, the design implements the recently introduced paradigm of momentum computing. Its nanosecond speeds and sub-Landauer thermodynamic efficiency arise from dynamically storing memory in momentum degrees of freedom. As such, during the swap, the microsta… ▽ More We introduce a fast and highly-efficient physically-realizable bit swap. Employing readily available and scalable Josephson junction microtechnology, the design implements the recently introduced paradigm of momentum computing. Its nanosecond speeds and sub-Landauer thermodynamic efficiency arise from dynamically storing memory in momentum degrees of freedom. As such, during the swap, the microstate distribution is never near equilibrium and the memory-state dynamics fall far outside of stochastic thermodynamics that assumes detailed-balanced Markovian dynamics. The device implements a bit-swap operation -- a fundamental operation necessary to build reversible universal computing. Extensive, physically-calibrated simulations demonstrate that device performance is robust and that momentum computing can support thermodynamically-efficient, high-speed, large-scale general-purpose computing that circumvents Landauer's bound. △ Less

Submitted 18 November, 2022; v1 submitted 14 February, 2022; originally announced February 2022.

Comments: 18 pages, 11 figures, 5 appendices; http://csc.ucdavis.edu/~cmg/compmech/pubs/gslmc.htm

Nonequilibrium Thermodynamics in Measuring Carbon Footprints: Disentangling Structure and Artifact in Input-Output Accounting

Authors: Samuel P. Loomis, Mark Cooper, James P. Crutchfield

Abstract: Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequa… ▽ More Multiregional input-output (MRIO) tables, in conjunction with Leontief analysis, are widely-used to assess the geographical distribution of carbon emissions and the economic activities that cause them. Majorization, a tool originating in economics that has found utility in statistical mechanics, can provide insight into how Leontief analysis links disparities in emissions with global income inequality. We examine Leontief analysis as a model, drawing out similarities with modern nonequilibrium statistical mechanics. Paralleling the physical concept of thermo-majorization, we define the concept of eco-majorization and show it is a sufficient condition to determine the directionality of embodied emission flows. Surprisingly, relatively small trade deficits and a geographically heterogeneous emissions-per-dollar ratio greatly increases the appearance of eco-majorization, regardless of any further content in the MRIO tables used. Our results are bolstered by a statistical analysis of null models of MRIO tables, based on data provided by the Global Trade Aggregation Project9 △ Less

Submitted 12 November, 2021; v1 submitted 7 June, 2021; originally announced June 2021.

Comments: 14 pages, 5 figures; 1 appendix; http://csc.ucdavis.edu/~cmg/compmech/pubs/netacam.htm

Spacetime Autoencoders Using Local Causal States

Authors: Adam Rupe, James P. Crutchfield

Abstract: Local causal states are latent representations that capture organized pattern and structure in complex spatiotemporal systems. We expand their functionality, framing them as spacetime autoencoders. Previously, they were only considered as maps from observable spacetime fields to latent local causal state fields. Here, we show that there is a stochastic decoding that maps back from the latent field… ▽ More Local causal states are latent representations that capture organized pattern and structure in complex spatiotemporal systems. We expand their functionality, framing them as spacetime autoencoders. Previously, they were only considered as maps from observable spacetime fields to latent local causal state fields. Here, we show that there is a stochastic decoding that maps back from the latent fields to observable fields. Furthermore, their Markovian properties define a stochastic dynamic in the latent space. Combined with stochastic decoding, this gives a new method for forecasting spacetime fields. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Non-Markovian Momentum Computing: Universal and Efficient

Authors: Kyle J. Ray, Gregory W. Wimsatt, Alexander B. Boyd, James P. Crutchfield

Abstract: All computation is physically embedded. Reflecting this, a growing body of results embraces rate equations as the underlying mechanics of thermodynamic computation and biological information processing. Strictly applying the implied continuous-time Markov chains, however, excludes a universe of natural computing. We show that expanding the toolset to continuous-time hidden Markov chains substantia… ▽ More All computation is physically embedded. Reflecting this, a growing body of results embraces rate equations as the underlying mechanics of thermodynamic computation and biological information processing. Strictly applying the implied continuous-time Markov chains, however, excludes a universe of natural computing. We show that expanding the toolset to continuous-time hidden Markov chains substantially removes the constraints. The general point is made concrete by our analyzing two eminently-useful computations that are impossible to describe with a set of rate equations over the memory states. We design and analyze a thermodynamically-costless bit flip, providing a first counterexample to rate-equation modeling. We generalize this to a costless Fredkin gate---a key operation in reversible computing that is computation universal. Going beyond rate-equation dynamics is not only possible, but necessary if stochastic thermodynamics is to become part of the paradigm for physical information processing. △ Less

Submitted 2 October, 2020; originally announced October 2020.

Comments: 6 pages, 3 figures; Supplementary Material, 1 page; http://csc.ucdavis.edu/~cmg/compmech/pubs/cbdb.htm

Journal ref: Phys. Rev. Research 3, 023164 (2021)

Correlated structural evolution within multiplex networks

Authors: Haochen Wu, Ryan G. James, James P. Crutchfield, Raissa M. D'Souza

Abstract: Many natural, engineered, and social systems can be represented using the framework of a layered network, where each layer captures a different type of interaction between the same set of nodes. The study of such multiplex networks is a vibrant area of research. Yet, understanding how to quantify the correlations present between pairs of layers, and more so present in their co-evolution, is lackin… ▽ More Many natural, engineered, and social systems can be represented using the framework of a layered network, where each layer captures a different type of interaction between the same set of nodes. The study of such multiplex networks is a vibrant area of research. Yet, understanding how to quantify the correlations present between pairs of layers, and more so present in their co-evolution, is lacking. Such methods would enable us to address fundamental questions involving issues such as function, redundancy and potential disruptions. Here we show first how the edge-set of a multiplex network can be used to construct an estimator of a joint probability distribution describing edge existence over all layers. We then adapt an information-theoretic measure of general correlation called the conditional mutual information, which uses the estimated joint probability distribution, to quantify the pairwise correlations present between layers. The pairwise comparisons can also be temporal, allowing us to identify if knowledge of a certain layer can provide additional information about the evolution of another layer. We analyze datasets from three distinct domains---economic, political, and airline networks---to demonstrate how pairwise correlation in structure and dynamical evolution between layers can be identified and show that anomalies can serve as potential indicators of major events such as shocks. △ Less

Submitted 9 May, 2020; originally announced May 2020.

Comments: Accepted to Journal of Complex Networks

The Hidden Fragility of Complex Systems -- Consequences of Change, Changing Consequences

Authors: James P. Crutchfield

Abstract: Short-term survival and an exuberant plunge into building our future are generating a new kind of unintended consequence -- hidden fragility. This is a direct effect of the sophistication and structural complexity of the socio-technical systems humans create. It is inevitable. And so the challenge is, How much can we understand and predict about these systems and about the social dynamics that lea… ▽ More Short-term survival and an exuberant plunge into building our future are generating a new kind of unintended consequence -- hidden fragility. This is a direct effect of the sophistication and structural complexity of the socio-technical systems humans create. It is inevitable. And so the challenge is, How much can we understand and predict about these systems and about the social dynamics that lead to their construction? △ Less

Submitted 24 March, 2020; originally announced March 2020.

Comments: 11 pages; http://csc.ucdavis.edu/~cmg/compmech/pubs/focs.htm

Journal ref: Cultures of Change: Social Atoms and Electronic Lives G. Ascione, C. Massip, J. Perello, editors, ACTAR D Publishers, Barcelona, Spain (2009) 98-111

DisCo: Physics-Based Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems

Authors: Adam Rupe, Nalini Kumar, Vladislav Epifanov, Karthik Kashinath, Oleksandr Pavlyk, Frank Schlimbach, Mostofa Patwary, Sergey Maidanov, Victor Lee, Prabhat, James P. Crutchfield

Abstract: Extracting actionable insight from complex unlabeled scientific data is an open challenge and key to unlocking data-driven discovery in science. Complementary and alternative to supervised machine learning approaches, unsupervised physics-based methods based on behavior-driven theories hold great promise. Due to computational limitations, practical application on real-world domain science problems… ▽ More Extracting actionable insight from complex unlabeled scientific data is an open challenge and key to unlocking data-driven discovery in science. Complementary and alternative to supervised machine learning approaches, unsupervised physics-based methods based on behavior-driven theories hold great promise. Due to computational limitations, practical application on real-world domain science problems has lagged far behind theoretical development. We present our first step towards bridging this divide - DisCo - a high-performance distributed workflow for the behavior-driven local causal state theory. DisCo provides a scalable unsupervised physics-based representation learning method that decomposes spatiotemporal systems into their structurally relevant components, which are captured by the latent local causal state variables. Complex spatiotemporal systems are generally highly structured and organize around a lower-dimensional skeleton of coherent structures, and in several firsts we demonstrate the efficacy of DisCo in capturing such structures from observational and simulated scientific data. To the best of our knowledge, DisCo is also the first application software developed entirely in Python to scale to over 1000 machine nodes, providing good performance along with ensuring domain scientists' productivity. We developed scalable, performant methods optimized for Intel many-core processors that will be upstreamed to open-source Python library packages. Our capstone experiment, using newly developed DisCo workflow and libraries, performs unsupervised spacetime segmentation analysis of CAM5.1 climate simulation data, processing an unprecedented 89.5 TB in 6.6 minutes end-to-end using 1024 Intel Haswell nodes on the Cori supercomputer obtaining 91% weak-scaling and 64% strong-scaling efficiency. △ Less

Submitted 25 September, 2019; originally announced September 2019.

Towards Unsupervised Segmentation of Extreme Weather Events

Authors: Adam Rupe, Karthik Kashinath, Nalini Kumar, Victor Lee, Prabhat, James P. Crutchfield

Abstract: Extreme weather is one of the main mechanisms through which climate change will directly impact human society. Coping with such change as a global community requires markedly improved understanding of how global warming drives extreme weather events. While alternative climate scenarios can be simulated using sophisticated models, identifying extreme weather events in these simulations requires aut… ▽ More Extreme weather is one of the main mechanisms through which climate change will directly impact human society. Coping with such change as a global community requires markedly improved understanding of how global warming drives extreme weather events. While alternative climate scenarios can be simulated using sophisticated models, identifying extreme weather events in these simulations requires automation due to the vast amounts of complex high-dimensional data produced. Atmospheric dynamics, and hydrodynamic flows more generally, are highly structured and largely organize around a lower dimensional skeleton of coherent structures. Indeed, extreme weather events are a special case of more general hydrodynamic coherent structures. We present a scalable physics-based representation learning method that decomposes spatiotemporal systems into their structurally relevant components, which are captured by latent variables known as local causal states. For complex fluid flows we show our method is capable of capturing known coherent structures, and with promising segmentation results on CAM5.1 water vapor data we outline the path to extreme weather identification from unlabeled climate model simulation data. △ Less

Submitted 16 September, 2019; originally announced September 2019.

A Physics-Based Approach to Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems

Authors: A. Rupe, J. P. Crutchfield, K. Kashinath, Prabhat

Abstract: Given that observational and numerical climate data are being produced at ever more prodigious rates, increasingly sophisticated and automated analysis techniques have become essential. Deep learning is quickly becoming a standard approach for such analyses and, while great progress is being made, major challenges remain. Unlike commercial applications in which deep learning has led to surprising… ▽ More Given that observational and numerical climate data are being produced at ever more prodigious rates, increasingly sophisticated and automated analysis techniques have become essential. Deep learning is quickly becoming a standard approach for such analyses and, while great progress is being made, major challenges remain. Unlike commercial applications in which deep learning has led to surprising successes, scientific data is highly complex and typically unlabeled. Moreover, interpretability and detecting new mechanisms are key to scientific discovery. To enhance discovery we present a complementary physics-based, data-driven approach that exploits the causal nature of spatiotemporal data sets generated by local dynamics (e.g. hydrodynamic flows). We illustrate how novel patterns and coherent structures can be discovered in cellular automata and outline the path from them to climate data. △ Less

Submitted 10 September, 2017; originally announced September 2017.

Comments: 4 pages, 1 figure; http://csc.ucdavis.edu/~cmg/compmech/pubs/ci2017_Rupe_et_al.htm

Fluctuations When Driving Between Nonequilibrium Steady States

Authors: P. M. Riechers, J. P. Crutchfield

Abstract: Maintained by environmental fluxes, biological systems are thermodynamic processes that operate far from equilibrium without detailed-balance dynamics. Yet, they often exhibit well defined nonequilibrium steady states (NESSs). More importantly, critical thermodynamic functionality arises directly from transitions among their NESSs, driven by environmental switching. Here, we identify constraints o… ▽ More Maintained by environmental fluxes, biological systems are thermodynamic processes that operate far from equilibrium without detailed-balance dynamics. Yet, they often exhibit well defined nonequilibrium steady states (NESSs). More importantly, critical thermodynamic functionality arises directly from transitions among their NESSs, driven by environmental switching. Here, we identify constraints on excess thermodynamic quantities that ride above the NESS housekeeping background. We do this by extending the Crooks fluctuation theorem to transitions among NESSs, without invoking an unphysical dual dynamics. This and corresponding integral fluctuation theorems determine how much work must be expended when controlling systems maintained far from equilibrium. This generalizes feedback control theory, showing that Maxwellian Demons can leverage mesoscopic-state information to take advantage of the excess energetics in NESS transitions. Altogether, these point to universal thermodynamic laws that are immediately applicable to the accessible degrees of freedom within the effective dynamic at any emergent level of hierarchical organization. By way of illustration, this readily allows analyzing a voltage-gated sodium ion channel whose molecular conformational dynamics play a critical functional role in propagating action potentials in mammalian neuronal membranes. △ Less

Submitted 28 October, 2016; originally announced October 2016.

Comments: 25 pages, 7 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/ftdness.htm

Correlation-powered Information Engines and the Thermodynamics of Self-Correction

Authors: Alexander B. Boyd, Dibyendu Mandal, James P. Crutchfield

Abstract: Information engines can use structured environments as a resource to generate work by randomizing ordered inputs and leveraging the increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir. We give a broadly applicable expression for the work production of an information engine, generally modeled as a memoryful channel that communicates inputs to outputs as it inte… ▽ More Information engines can use structured environments as a resource to generate work by randomizing ordered inputs and leveraging the increased Shannon entropy to transfer energy from a thermal reservoir to a work reservoir. We give a broadly applicable expression for the work production of an information engine, generally modeled as a memoryful channel that communicates inputs to outputs as it interacts with an evolving environment. The expression establishes that an information engine must have more than one memory state in order to leverage input environment correlations. To emphasize this functioning, we designed an information engine powered solely by temporal correlations and not by statistical biases, as employed by previous engines. Key to this is the engine's ability to synchronize---the engine automatically returns to a desired dynamical phase when thrown into an unwanted, dissipative phase by corruptions in the input---that is, by unanticipated environmental fluctuations. This self-correcting mechanism is robust up to a critical level of corruption, beyond which the system fails to act as an engine. We give explicit analytical expressions for both work and critical corruption level and summarize engine performance via a thermodynamic-function phase diagram over engine control parameters. The results reveal a new thermodynamic mechanism based on nonergodicity that underlies error correction as it operates to support resilient engineered and biological systems. △ Less

Submitted 13 August, 2016; v1 submitted 27 June, 2016; originally announced June 2016.

Comments: 22 pages, 13 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/tos.htm

Journal ref: Phys. Rev. E 95, 012152 (2017)

Statistical Signatures of Structural Organization: The case of long memory in renewal processes

Authors: Sarah E. Marzen, James P. Crutchfield

Abstract: Identifying and quantifying memory are often critical steps in developing a mechanistic understanding of stochastic processes. These are particularly challenging and necessary when exploring processes that exhibit long-range correlations. The most common signatures employed rely on second-order temporal statistics and lead, for example, to identifying long memory in processes with power-law autoco… ▽ More Identifying and quantifying memory are often critical steps in developing a mechanistic understanding of stochastic processes. These are particularly challenging and necessary when exploring processes that exhibit long-range correlations. The most common signatures employed rely on second-order temporal statistics and lead, for example, to identifying long memory in processes with power-law autocorrelation function and Hurst exponent greater than $1/2$. However, most stochastic processes hide their memory in higher-order temporal correlations. Information measures---specifically, divergences in the mutual information between a process' past and future (excess entropy) and minimal predictive memory stored in a process' causal states (statistical complexity)---provide a different way to identify long memory in processes with higher-order temporal correlations. However, there are no ergodic stationary processes with infinite excess entropy for which information measures have been compared to autocorrelation functions and Hurst exponents. Here, we show that fractal renewal processes---those with interevent distribution tails $\propto t^{-α}$---exhibit long memory via a phase transition at $α= 1$. Excess entropy diverges only there and statistical complexity diverges there and for all $α< 1$. When these processes do have power-law autocorrelation function and Hurst exponent greater than $1/2$, they do not have divergent excess entropy. This analysis breaks the intuitive association between these different quantifications of memory. We hope that the methods used here, based on causal states, provide some guide as to how to construct and analyze other long memory processes. △ Less

Submitted 6 December, 2015; originally announced December 2015.

Comments: 13 pages, 2 figures, 3 appendixes; http://csc.ucdavis.edu/~cmg/compmech/pubs/lrmrp.htm

Identifying Functional Thermodynamics in Autonomous Maxwellian Ratchets

Authors: A. B. Boyd, D. Mandal, J. P. Crutchfield

Abstract: We introduce a family of Maxwellian Demons for which correlations among information bearing degrees of freedom can be calculated exactly and in compact analytical form. This allows one to precisely determine Demon functional thermodynamic operating regimes, when previous methods either misclassify or simply fail due to approximations they invoke. This reveals that these Demons are more functional… ▽ More We introduce a family of Maxwellian Demons for which correlations among information bearing degrees of freedom can be calculated exactly and in compact analytical form. This allows one to precisely determine Demon functional thermodynamic operating regimes, when previous methods either misclassify or simply fail due to approximations they invoke. This reveals that these Demons are more functional than previous candidates. They too behave either as engines, lifting a mass against gravity by extracting energy from a single heat reservoir, or as Landauer erasers, consuming external work to remove information from a sequence of binary symbols by decreasing their individual uncertainty. Going beyond these, our Demon exhibits a new functionality that erases bits not by simply decreasing individual-symbol uncertainty, but by increasing inter-bit correlations (that is, by adding temporal order) while increasing single-symbol uncertainty. In all cases, but especially in the new erasure regime, exactly accounting for informational correlations leads to tight bounds on Demon performance, expressed as a refined Second Law of Thermodynamics that relies on the Kolmogorov-Sinai entropy for dynamical processes and not on changes purely in system configurational entropy, as previously employed. We rigorously derive the refined Second Law under minimal assumptions and so it applies quite broadly---for Demons with and without memory and input sequences that are correlated or not. We note that general Maxwellian Demons readily violate previously proposed, alternative such bounds, while the current bound still holds. △ Less

Submitted 21 December, 2015; v1 submitted 6 July, 2015; originally announced July 2015.

Comments: 13 pages, 9 figures, http://csc.ucdavis.edu/~cmg/compmech/pubs/mrd.htm

Bayesian Structural Inference for Hidden Processes

Authors: Christopher C. Strelioff, James P. Crutchfield

Abstract: We introduce a Bayesian approach to discovering patterns in structurally complex processes. The proposed method of Bayesian Structural Inference (BSI) relies on a set of candidate unifilar HMM (uHMM) topologies for inference of process structure from a data series. We employ a recently developed exact enumeration of topological epsilon-machines. (A sequel then removes the topological restriction.)… ▽ More We introduce a Bayesian approach to discovering patterns in structurally complex processes. The proposed method of Bayesian Structural Inference (BSI) relies on a set of candidate unifilar HMM (uHMM) topologies for inference of process structure from a data series. We employ a recently developed exact enumeration of topological epsilon-machines. (A sequel then removes the topological restriction.) This subset of the uHMM topologies has the added benefit that inferred models are guaranteed to be epsilon-machines, irrespective of estimated transition probabilities. Properties of epsilon-machines and uHMMs allow for the derivation of analytic expressions for estimating transition probabilities, inferring start states, and comparing the posterior probability of candidate model topologies, despite process internal structure being only indirectly present in data. We demonstrate BSI's effectiveness in estimating a process's randomness, as reflected by the Shannon entropy rate, and its structure, as quantified by the statistical complexity. We also compare using the posterior distribution over candidate models and the single, maximum a posteriori model for point estimation and show that the former more accurately reflects uncertainty in estimated values. We apply BSI to in-class examples of finite- and infinite-order Markov processes, as well to an out-of-class, infinite-state hidden process. △ Less

Submitted 9 December, 2013; v1 submitted 5 September, 2013; originally announced September 2013.

Comments: 20 pages, 11 figures, 1 table; supplementary materials, 15 pages, 11 figures, 6 tables; http://csc.ucdavis.edu/~cmg/compmech/pubs/bsihp.htm

Report number: Santa Fe Institute Working Paper 13-09-027

Journal ref: Phys. Rev. E 89, 042119 (2014)

How Hidden are Hidden Processes? A Primer on Crypticity and Entropy Convergence

Authors: John R. Mahoney, Christopher J. Ellison, Ryan G. James, James P. Crutchfield

Abstract: We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block en… ▽ More We investigate a stationary process's crypticity---a measure of the difference between its hidden state information and its observed information---using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block entropy and block-state entropy in a geometric setting, one that is more intuitive and that leads to a number of new results. For example, we connect crypticity to how an observer synchronizes to a process. We show that the block-causal-state entropy is a convex function of block length. We give a complete analysis of spin chains. We present a classification scheme that surveys stationary processes in terms of their possible cryptic and Markov orders. We illustrate related entropy convergence behaviors using a new form of foliated information diagram. Finally, along the way, we provide a variety of interpretations of crypticity and cryptic order to establish their naturalness and pervasiveness. Hopefully, these will inspire new applications in spatially extended and network dynamical systems. △ Less

Submitted 6 August, 2011; originally announced August 2011.

Comments: 18 pages, 18 figures; http://csc.ucdavis.edu/~cmg/compmech/pubs/iacp2.htm

Prediction, Retrodiction, and The Amount of Information Stored in the Present

Authors: Christopher J. Ellison, John R. Mahoney, James P. Crutchfield

Abstract: We introduce an ambidextrous view of stochastic dynamical systems, comparing their forward-time and reverse-time representations and then integrating them into a single time-symmetric representation. The perspective is useful theoretically, computationally, and conceptually. Mathematically, we prove that the excess entropy--a familiar measure of organization in complex systems--is the mutual inf… ▽ More We introduce an ambidextrous view of stochastic dynamical systems, comparing their forward-time and reverse-time representations and then integrating them into a single time-symmetric representation. The perspective is useful theoretically, computationally, and conceptually. Mathematically, we prove that the excess entropy--a familiar measure of organization in complex systems--is the mutual information not only between the past and future, but also between the predictive and retrodictive causal states. Practically, we exploit the connection between prediction and retrodiction to directly calculate the excess entropy. Conceptually, these lead one to discover new system invariants for stochastic dynamical systems: crypticity (information accessibility) and causal irreversibility. Ultimately, we introduce a time-symmetric representation that unifies all these quantities, compressing the two directional representations into one. The resulting compression offers a new conception of the amount of information stored in the present. △ Less

Submitted 21 May, 2009; originally announced May 2009.

Comments: 17 pages, 7 figures, 1 table; http://users.cse.ucdavis.edu/~cmg/compmech/pubs/pratisp.htm

Structure or Noise?

Authors: Susanne Still, James P. Crutchfield

Abstract: We show how rate-distortion theory provides a mechanism for automated theory building by naturally distinguishing between regularity and randomness. We start from the simple principle that model variables should, as much as possible, render the future and past conditionally independent. From this, we construct an objective function for model making whose extrema embody the trade-off between a mo… ▽ More We show how rate-distortion theory provides a mechanism for automated theory building by naturally distinguishing between regularity and randomness. We start from the simple principle that model variables should, as much as possible, render the future and past conditionally independent. From this, we construct an objective function for model making whose extrema embody the trade-off between a model's structural complexity and its predictive power. The solutions correspond to a hierarchy of models that, at each level of complexity, achieve optimal predictive power at minimal cost. In the limit of maximal prediction the resulting optimal model identifies a process's intrinsic organization by extracting the underlying causal states. In this limit, the model's complexity is given by the statistical complexity, which is known to be minimal for achieving maximum prediction. Examples show how theory building can profit from analyzing a process's causal compressibility, which is reflected in the optimal models' rate-distortion curve--the process's characteristic for optimally balancing structure and noise at different levels of representation. △ Less

Submitted 29 June, 2008; v1 submitted 4 August, 2007; originally announced August 2007.

Comments: 6 pages, 2 figures; http://cse.ucdavis.edu/~cmg/compmech/pubs/son.html

Automated Pattern Detection--An Algorithm for Constructing Optimally Synchronizing Multi-Regular Language Filters

Authors: Carl S. McTague, James P. Crutchfield

Abstract: In the computational-mechanics structural analysis of one-dimensional cellular automata the following automata-theoretic analogue of the \emph{change-point problem} from time series analysis arises: \emph{Given a string $σ$ and a collection $\{\mc{D}_i\}$ of finite automata, identify the regions of $σ$ that belong to each $\mc{D}_i$ and, in particular, the boundaries separating them.} We present… ▽ More In the computational-mechanics structural analysis of one-dimensional cellular automata the following automata-theoretic analogue of the \emph{change-point problem} from time series analysis arises: \emph{Given a string $σ$ and a collection $\{\mc{D}_i\}$ of finite automata, identify the regions of $σ$ that belong to each $\mc{D}_i$ and, in particular, the boundaries separating them.} We present two methods for solving this \emph{multi-regular language filtering problem}. The first, although providing the ideal solution, requires a stack, has a worst-case compute time that grows quadratically in $σ$'s length and conditions its output at any point on arbitrarily long windows of future input. The second method is to algorithmically construct a transducer that approximates the first algorithm. In contrast to the stack-based algorithm, however, the transducer requires only a finite amount of memory, runs in linear time, and gives immediate output for each letter read; it is, moreover, the best possible finite-state approximation with these three features. △ Less

Submitted 7 October, 2004; originally announced October 2004.

Comments: 18 pages, 12 figures, 2 appendices; http://www.santafe.edu/~cmg

Report number: Santa Fe Institute 04-09-027

Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction

Authors: Cosma Rohilla Shalizi, James P. Crutchfield

Abstract: Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis. Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis. △ Less

Submitted 16 June, 2000; originally announced June 2000.

Comments: 3 pages, no figures, submitted to PRE as a "brief report". Revision: added an acknowledgements section originally omitted by a LaTeX bug

Journal ref: Advances in Complex Systems, vol. 5, pp. 91--95 (2002)

The Evolutionary Design of Collective Computation in Cellular Automata

Authors: James P. Crutchfield, Melanie Mitchell, Rajarshi Das

Abstract: We investigate the ability of a genetic algorithm to design cellular automata that perform computations. The computational strategies of the resulting cellular automata can be understood using a framework in which ``particles'' embedded in space-time configurations carry information and interactions between particles effect information processing. This structural analysis can also be used to exp… ▽ More We investigate the ability of a genetic algorithm to design cellular automata that perform computations. The computational strategies of the resulting cellular automata can be understood using a framework in which ``particles'' embedded in space-time configurations carry information and interactions between particles effect information processing. This structural analysis can also be used to explain the evolutionary process by which the strategies were designed by the genetic algorithm. More generally, our goals are to understand how machine-learning processes can design complex decentralized systems with sophisticated collective computational abilities and to develop rigorous frameworks for understanding how the resulting dynamical systems perform computation. △ Less

Submitted 8 September, 1998; originally announced September 1998.

Comments: 49 pages, 20 figures

Report number: Santa Fe Institute Working Paper 98-09-080