-
Equivalence of information production and generalized entropies in complex processes
Authors:
Rudolf Hanel,
Stefan Thurner
Abstract:
Complex systems that are characterized by strong correlations and fat-tailed distribution functions have been argued to be incompatible within the framework of Boltzmann-Gibbs entropy. As an alternative, so-called generalized entropies were proposed and intensively studied. Here we show that this incompatibility is a misconception. For a broad class of processes, Boltzmann entropy the log multipli…
▽ More
Complex systems that are characterized by strong correlations and fat-tailed distribution functions have been argued to be incompatible within the framework of Boltzmann-Gibbs entropy. As an alternative, so-called generalized entropies were proposed and intensively studied. Here we show that this incompatibility is a misconception. For a broad class of processes, Boltzmann entropy the log multiplicity remains the valid entropy concept, however, for non-i.i.d., non-multinomial, and non-ergodic processes, Boltzmann entropy is not of Shannon form. The correct form of Boltzmann entropy can be shown to be identical with generalized entropies. We derive this result for all processes that can be mapped reversibly to adjoint representations where processes are i.i.d.. In these representations the information production is given by the Shannon entropy. We proof that over the original sampling space this yields functionals that are identical to generalized entropies. The problem of constructing adequate context-sensitive entropy functionals therefore can be translated into the much simpler problem of finding adjoint representations. The method provides a comprehensive framework for a statistical physics of strongly correlated systems and complex processes.
△ Less
Submitted 12 August, 2022;
originally announced August 2022.
-
Thermodynamics of structure-forming systems
Authors:
Jan Korbel,
Simon David Lindner,
Rudolf Hanel,
Stefan Thurner
Abstract:
Structure-forming systems are ubiquitous in nature, ranging from atoms building molecules to self-assembly of colloidal amphibolic particles. The understanding of the underlying thermodynamics of such systems remains an important problem. Here we derive the entropy for structure-forming systems that differs from Boltzmann-Gibbs entropy by a term that explicitly captures clustered states. For large…
▽ More
Structure-forming systems are ubiquitous in nature, ranging from atoms building molecules to self-assembly of colloidal amphibolic particles. The understanding of the underlying thermodynamics of such systems remains an important problem. Here we derive the entropy for structure-forming systems that differs from Boltzmann-Gibbs entropy by a term that explicitly captures clustered states. For large systems and low concentrations, the approach is equivalent to the grand-canonical ensemble; for small systems, we find significant deviations. We derive the detailed fluctuation theorem and Crooks' work fluctuation theorem for structure-forming systems. The connection to the theory of particle self-assembly is discussed. We apply the results to several physical systems. We present the phase diagram for patchy particles described by the Kern-Frenkel potential. We show that the Curie-Weiss model with molecule structures exhibits a first-order phase transition.
△ Less
Submitted 27 January, 2021; v1 submitted 14 April, 2020;
originally announced April 2020.
-
Information geometry of scaling expansions of non-exponentially growing configuration spaces
Authors:
Jan Korbel,
Rudolf Hanel,
Stefan Thurner
Abstract:
Many stochastic complex systems are characterized by the fact that their configuration space doesn't grow exponentially as a function of the degrees of freedom. The use of scaling expansions is a natural way to measure the asymptotic growth of the configuration space volume in terms of the scaling exponents of the system. These scaling exponents can, in turn, be used to define universality classes…
▽ More
Many stochastic complex systems are characterized by the fact that their configuration space doesn't grow exponentially as a function of the degrees of freedom. The use of scaling expansions is a natural way to measure the asymptotic growth of the configuration space volume in terms of the scaling exponents of the system. These scaling exponents can, in turn, be used to define universality classes that uniquely determine the statistics of a system. Every system belongs to one of these classes. Here we derive the information geometry of scaling expansions of sample spaces. In particular, we present the deformed logarithms and the metric in a systematic and coherent way. We observe a phase transition for the curvature. The phase transition can be well measured by the characteristic length r, corresponding to a ball with radius 2r having the same curvature as the statistical manifold. Increasing characteristic length with respect to the size of the system is associated with sub-exponential sample space growth is associated with strongly constrained and correlated complex systems. Decreasing of the characteristic length corresponds to super-exponential sample space growth that occurs for example in systems that develop structure as they evolve. Constant curvature means exponential sample space growth that is associated with multinomial statistics, and traditional Boltzmann-Gibbs, or Shannon statistics applies. This allows us to characterize transitions between statistical manifolds corresponding to different families of probability distributions.
△ Less
Submitted 14 April, 2020; v1 submitted 17 January, 2020;
originally announced January 2020.
-
Information geometric duality of $φ$-deformed exponential families
Authors:
Jan Korbel,
Rudolf Hanel,
Stefan Thurner
Abstract:
In the world of generalized entropies---which, for example, play a role in physical systems with sub- and super-exponential phasespace growth per degree of freedom---there are two ways for implementing constraints in the maximum entropy principle: linear- and escort constraints. Both appear naturally in different contexts. Linear constraints appear e.g. in physical systems, when additional informa…
▽ More
In the world of generalized entropies---which, for example, play a role in physical systems with sub- and super-exponential phasespace growth per degree of freedom---there are two ways for implementing constraints in the maximum entropy principle: linear- and escort constraints. Both appear naturally in different contexts. Linear constraints appear e.g. in physical systems, when additional information about the system is available through higher moments. Escort distributions appear naturally in the context of multifractals and information geometry. It was shown recently that there exists a fundamental duality that relates both approaches on the basis of the corresponding deformed logarithms (deformed-log duality). Here we show that there exists another duality that arises in the context of information geometry, relating the Fisher information of $φ$-deformed exponential families that correspond to linear constraints (as studied by J. Naudts), with those that are based on escort constraints (as studied by S.-I. Amari). We explicitly demonstrate this information geometric duality for the case of $(c,d)$-entropy that covers all situations that are compatible with the first three Shannon-Khinchin axioms, and that include Shannon, Tsallis, Anteneodo-Plastino entropy, and many more as special cases. Finally, we discuss the relation between the deformed-log duality and the information geometric duality, and mention that the escort distributions arising in the two dualities are generally different and only coincide for the case of the Tsallis deformation.
△ Less
Submitted 24 December, 2018;
originally announced December 2018.
-
The Transformation-Groupoid Structure of the q-Gaussian Family
Authors:
Angel Akio Tateishi,
Rudolf Hanel,
Stefan Thurner
Abstract:
The q-Gaussian function emerges naturally in various applications of statistical mechanics of non-ergodic and complex systems. In particular it was shown that in the theory of binary processes with correlations, the q-Gaussian can appear as a limiting distribution. Further, there exist several problems and situations where, depending on procedural or algorithmic details of data-processing, q-Gauss…
▽ More
The q-Gaussian function emerges naturally in various applications of statistical mechanics of non-ergodic and complex systems. In particular it was shown that in the theory of binary processes with correlations, the q-Gaussian can appear as a limiting distribution. Further, there exist several problems and situations where, depending on procedural or algorithmic details of data-processing, q-Gaussian distributions may yield distinct values of q, where one value is larger, the other smaller than one. To relate such pairs of q-Gaussians it would be convenient to map such distributions onto one another, ideally in a way, that any value of q can be mapped uniquely to any other value q'. So far a (duality) map from q -> q'=(7-5q)/(5-3q) was found, mapping q from the interval q\in [-\infty, 1] -> q'\in [1, 5/3]. Here we complete the theory of transformations of q-Gaussians by deriving a general map γ_{qq'}, that transforms normalizable q-Gaussian distributions onto one another for which q and q' are in the range of [1,3). By combining this with the previous result, a mapping from any value of q \in [-\infty,3) is possible to any other value q'\in [-\infty,3). We show that the action of γ_{qq'} on the set of q-Gaussian distributions is a transformation groupoid.
△ Less
Submitted 16 January, 2013;
originally announced January 2013.
-
Generalized entropies and logarithms and their duality relations
Authors:
Rudolf Hanel,
Stefan Thurner,
Murray Gell-Mann
Abstract:
For statistical systems that violate one of the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or non-ergodic complex systems. For such systems where the…
▽ More
For statistical systems that violate one of the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or non-ergodic complex systems. For such systems where the composability axiom is violated there exist only two ways to implement the maximum entropy principle, one using escort probabilities, the other not. The two ways are connected through a duality. Here we show that this duality fixes a unique escort probability, which allows us to derive a complete theory of the generalized logarithms that naturally arise from the violation of this axiom. We then show how the functional forms of these generalized logarithms are related to the asymptotic scaling behavior of the entropy.
△ Less
Submitted 9 November, 2012;
originally announced November 2012.
