Title:The Geometry of Statistical Data and Information: A Large Deviation Perspective

Abstract:Combinatorics, probabilities, and measurements are fundamental to understanding information. This work explores how the application of large deviation theory (LDT) in counting phenomena leads to the emergence of various entropy functions, including Shannon's entropy, mutual information, and relative and conditional entropies. In terms of these functions, we reveal an inherent geometrical structure through operations, including contractions, lift, change of basis, and projections. Legendre-Fenchel (LF) transform which is central to both LDT and Gibbs' method of thermodynamics, offers a novel energetic description of data. The manifold of empirical mean values of statistical data ad infinitum has a parametrization using LF conjugates w.r.t. an entropy function; this gives rise to a family of models as a dual space and the additivity known in statistical thermodynamic energetics. This work clearly introduces data into the current information geometry, and includes information projection defined through conditional expectations in Kolmogorov's probability theory.