-
Information geometry of chemical reaction networks: Cramer-Rao bound and absolute sensitivity revisited
Authors:
Dimitri Loutchko,
Yuki Sughiyama,
Tetsuya J. Kobayashi
Abstract:
Information geometry is based on classical Legendre duality but allows to incorporate additional structure such as algebraic constraints and Bregman divergence functions. It is naturally suited, and has been successfully used, to describe the thermodynamics of chemical reaction networks (CRNs) based on the Legendre duality between concentration and potential spaces, where algebraic constraints are…
▽ More
Information geometry is based on classical Legendre duality but allows to incorporate additional structure such as algebraic constraints and Bregman divergence functions. It is naturally suited, and has been successfully used, to describe the thermodynamics of chemical reaction networks (CRNs) based on the Legendre duality between concentration and potential spaces, where algebraic constraints are enforced by the stoichiometry. In this article, the Riemannian geometrical aspects of the theory are explored. It is shown that duality between concentration and potential spaces and the natural parametrizations of equilibrium subspace are isometries, which leads to a multivariate Cramer-Rao bound through the comparison of two Riemannian metric tensors.
In the subsequent part, the theory is applied to the recently introduced concept of absolute sensitivity. Using the Riemannian geometric tools, it is proven that the absolute sensitivity is a projection operator onto the tangent bundle of the equilibrium manifold. A linear algebraic characterization and explicit results on first order corrections to the thermodynamics of ideal solutions are provided. Finally, the theory is applied to the IDHKP-IDH glyoxylate bypass regulation system.
The novelty of the theory is that it is applicable to CRNs with non-ideal thermodynamical behavior, which are prevalent in highly crowded cellular environments due to various interactions between the chemicals. Indeed, the analyzed example shows remarkable behavior ranging from hypersensitivity to negative-self regulations. These are effects which usually require strongly nonlinear reaction kinetics. However, here, they are obtained by tuning thermodynamical interactions providing a complementary, and physically well-founded, viewpoint on such phenomena.
△ Less
Submitted 25 March, 2025;
originally announced March 2025.
-
Optimal control of stochastic reaction networks with entropic control cost and emergence of mode-switching strategies
Authors:
Shuhei A. Horiguchi,
Tetsuya J. Kobayashi
Abstract:
Controlling the stochastic dynamics of biological populations is a challenge that arises across various biological contexts. However, these dynamics are inherently nonlinear and involve a discrete state space, i.e., the number of molecules, cells, or organisms. Additionally, the possibility of extinction has a significant impact on both the dynamics and control strategies, particularly when the po…
▽ More
Controlling the stochastic dynamics of biological populations is a challenge that arises across various biological contexts. However, these dynamics are inherently nonlinear and involve a discrete state space, i.e., the number of molecules, cells, or organisms. Additionally, the possibility of extinction has a significant impact on both the dynamics and control strategies, particularly when the population size is small. These factors hamper the direct application of conventional control theories to biological systems. To address these challenges, we formulate the optimal control problem for stochastic population dynamics by utilizing a control cost function based on the Kullback-Leibler divergence. This approach naturally accounts for population-specific factors and simplifies the complex nonlinear Hamilton-Jacobi-Bellman equation into a linear form, facilitating efficient computation of optimal solutions. We demonstrate the effectiveness of our approach by applying it to the control of interacting random walkers, Moran processes, and SIR models, and observe the mode-switching phenomena in the control strategies. Our approach provides new opportunities for applying control theory to a wide range of biological problems.
△ Less
Submitted 25 September, 2024;
originally announced September 2024.
-
Theory for Optimal Estimation and Control under Resource Limitations and Its Applications to Biological Information Processing and Decision-Making
Authors:
Takehiro Tottori,
Tetsuya J. Kobayashi
Abstract:
Despite being optimized, the information processing of biological organisms exhibits significant variability in its complexity and capability. One potential source of this diversity is the limitation of resources required for information processing. However, we lack a theoretical framework that comprehends the relationship between biological information processing and resource limitations and inte…
▽ More
Despite being optimized, the information processing of biological organisms exhibits significant variability in its complexity and capability. One potential source of this diversity is the limitation of resources required for information processing. However, we lack a theoretical framework that comprehends the relationship between biological information processing and resource limitations and integrates it with decision-making conduced downstream of the information processing. In this paper, we propose a novel optimal estimation and control theory that accounts for the resource limitations inherent in biological systems. This theory explicitly formulates the memory that organisms can store and operate and obtains optimal memory dynamics using optimal control theory. This approach takes account of various resource limitations, such as memory capacity, intrinsic noise, and energy cost, and unifies state estimation and control. We apply this theory to minimal models of biological information processing and decision-making under resource limitations and find that such limitations induce discontinuous and non-monotonic phase transitions between memory-less and memory-based strategies. Therefore, this theory establishes a comprehensive framework for addressing biological information processing and decision-making under resource limitations, revealing the rich and complex behaviors that arise from resource limitations.
△ Less
Submitted 20 September, 2024;
originally announced September 2024.
-
Information Geometry of Dynamics on Graphs and Hypergraphs
Authors:
Tetsuya J. Kobayashi,
Dimitri Loutchko,
Atsushi Kamimura,
Shuhei A. Horiguchi,
Yuki Sughiyama
Abstract:
We introduce a new information-geometric structure associated with the dynamics on discrete objects such as graphs and hypergraphs. The presented setup consists of two dually flat structures built on the vertex and edge spaces, respectively. The former is the conventional duality between density and potential, e.g., the probability density and its logarithmic form induced by a convex thermodynamic…
▽ More
We introduce a new information-geometric structure associated with the dynamics on discrete objects such as graphs and hypergraphs. The presented setup consists of two dually flat structures built on the vertex and edge spaces, respectively. The former is the conventional duality between density and potential, e.g., the probability density and its logarithmic form induced by a convex thermodynamic function. The latter is the duality between flux and force induced by a convex and symmetric dissipation function, which drives the dynamics of the density. These two are connected topologically by the homological algebraic relation induced by the underlying discrete objects. The generalized gradient flow in this doubly dual flat structure is an extension of the gradient flows on Riemannian manifolds, which include Markov jump processes and nonlinear chemical reaction dynamics as well as the natural gradient and mirror descent. The information-geometric projections on this doubly dual flat structure lead to information-geometric extensions of the Helmholtz-Hodge decomposition and the Otto structure in $L^{2}$ Wasserstein geometry. The structure can be extended to non-gradient nonequilibrium flows, from which we also obtain the induced dually flat structure on cycle spaces. This abstract but general framework can extend the applicability of information geometry to various problems of linear and nonlinear dynamics.
△ Less
Submitted 5 August, 2023; v1 submitted 25 November, 2022;
originally announced November 2022.
-
Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control
Authors:
Takehiro Tottori,
Tetsuya J. Kobayashi
Abstract:
Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. In order to obtain the optimal control function of ML-POSC, a system of the forward Fokker-Planck (FP) equation and the backward Hamilton-Jacobi-Bellman (HJB) equation needs to be solved. In this work, we firstly show that the system of HJB-…
▽ More
Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. In order to obtain the optimal control function of ML-POSC, a system of the forward Fokker-Planck (FP) equation and the backward Hamilton-Jacobi-Bellman (HJB) equation needs to be solved. In this work, we firstly show that the system of HJB-FP equations can be interpreted via the Pontryagin's minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) to ML-POSC, which has been used in the Pontryagin's minimum principle. FBSM is an algorithm to compute the forward FP equation and the backward HJB equation alternately. Although the convergence of FBSM is generally not guaranteed, it is guaranteed in ML-POSC because the coupling of HJB-FP equations is limited to the optimal control function in ML-POSC.
△ Less
Submitted 8 November, 2022; v1 submitted 24 October, 2022;
originally announced October 2022.
-
Mean-Field Control Approach to Decentralized Stochastic Control with Finite-Dimensional Memories
Authors:
Takehiro Tottori,
Tetsuya J. Kobayashi
Abstract:
Decentralized stochastic control (DSC) considers the optimal control problem of a multi-agent system. However, DSC cannot be solved except in the special cases because the estimation among the agents is generally intractable. In this work, we propose memory-limited DSC (ML-DSC), in which each agent compresses the observation history into the finite-dimensional memory. Because this compression simp…
▽ More
Decentralized stochastic control (DSC) considers the optimal control problem of a multi-agent system. However, DSC cannot be solved except in the special cases because the estimation among the agents is generally intractable. In this work, we propose memory-limited DSC (ML-DSC), in which each agent compresses the observation history into the finite-dimensional memory. Because this compression simplifies the estimation among the agents, ML-DSC can be solved in more general cases based on the mean-field control theory. We demonstrate ML-DSC in the general LQG problem. Because estimation and control are not clearly separated in the general LQG problem, the Riccati equation is modified to the decentralized Riccati equation, which improves estimation as well as control. Our numerical experiment shows that the decentralized Riccati equation is superior to the conventional Riccati equation.
△ Less
Submitted 12 September, 2022;
originally announced September 2022.
-
Memory-Limited Partially Observable Stochastic Control and its Mean-Field Control Approach
Authors:
Takehiro Tottori,
Tetsuya J. Kobayashi
Abstract:
Control problems with incomplete information and memory limitation appear in many practical situations. Although partially observable stochastic control (POSC) is a conventional theoretical framework that considers the optimal control problem with incomplete information, it cannot consider memory limitation. Furthermore, POSC cannot be solved in practice except in the special cases. In order to ad…
▽ More
Control problems with incomplete information and memory limitation appear in many practical situations. Although partially observable stochastic control (POSC) is a conventional theoretical framework that considers the optimal control problem with incomplete information, it cannot consider memory limitation. Furthermore, POSC cannot be solved in practice except in the special cases. In order to address these issues, we propose an alternative theoretical framework, memory-limited POSC (ML-POSC). ML-POSC directly considers memory limitation as well as incomplete information, and it can be solved in practice by employing the mathematical technique of the mean-field control theory. ML-POSC can generalize the LQG problem to include memory limitation. Because estimation and control are not clearly separated in the LQG problem with memory limitation, the Riccati equation is modified to the partially observable Riccati equation, which improves estimation as well as control. Furthermore, we demonstrate the effectiveness of ML-POSC to a non-LQG problem by comparing it with the local LQG approximation.
△ Less
Submitted 22 September, 2022; v1 submitted 20 March, 2022;
originally announced March 2022.
-
Forward and Backward Bellman equations improve the efficiency of EM algorithm for DEC-POMDP
Authors:
Takehiro Tottori,
Tetsuya J. Kobayashi
Abstract:
Decentralized partially observable Markov decision process (DEC-POMDP) models sequential decision making problems by a team of agents. Since the planning of DEC-POMDP can be interpreted as the maximum likelihood estimation for the latent variable model, DEC-POMDP can be solved by the EM algorithm. However, in EM for DEC-POMDP, the forward--backward algorithm needs to be calculated up to the infini…
▽ More
Decentralized partially observable Markov decision process (DEC-POMDP) models sequential decision making problems by a team of agents. Since the planning of DEC-POMDP can be interpreted as the maximum likelihood estimation for the latent variable model, DEC-POMDP can be solved by the EM algorithm. However, in EM for DEC-POMDP, the forward--backward algorithm needs to be calculated up to the infinite horizon, which impairs the computational efficiency. In this paper, we propose the Bellman EM algorithm (BEM) and the modified Bellman EM algorithm (MBEM) by introducing the forward and backward Bellman equations into EM. BEM can be more efficient than EM because BEM calculates the forward and backward Bellman equations instead of the forward--backward algorithm up to the infinite horizon. However, BEM cannot always be more efficient than EM when the size of problems is large because BEM calculates an inverse matrix. We circumvent this shortcoming in MBEM by calculating the forward and backward Bellman equations without the inverse matrix. Our numerical experiments demonstrate that the convergence of MBEM is faster than that of EM.
△ Less
Submitted 5 May, 2021; v1 submitted 19 March, 2021;
originally announced March 2021.
