Showing 1–3 of 3 results for author: Mahdian, S

Linear Algebraic Truncation Algorithm with A Posteriori Error Bounds for Computing Markov Chain Equilibrium Gradients

Authors: Saied Mahdian, Peter W. Glynn

Abstract: The numerical computation of equilibrium reward gradients for Markov chains appears in many applications for example within the policy improvement step arising in connection with average reward stochastic dynamic programming. When the state space is large or infinite, one will typically need to truncate the state space in order to arrive at a numerically tractable formulation. In this paper, we de… ▽ More The numerical computation of equilibrium reward gradients for Markov chains appears in many applications for example within the policy improvement step arising in connection with average reward stochastic dynamic programming. When the state space is large or infinite, one will typically need to truncate the state space in order to arrive at a numerically tractable formulation. In this paper, we derive the first computable a posteriori error bounds for equilibrium reward gradients that account for the error induced by the truncation. Our approach uses regeneration to express equilibrium quantities in terms of the expectations of cumulative rewards over regenerative cycles. Lyapunov functions are then used to bound the contributions to these cumulative rewards and their gradients from path excursions that take the chain outside the truncation set. Our numerical results indicate that our approach can provide highly accurate bounds with truncation sets of moderate size. We further extend our approach to Markov jump processes. △ Less

Submitted 9 January, 2025; originally announced January 2025.

A Numerical Truncation Approximation with A Posteriori Error Bounds for the Solution of Poisson's Equation

Authors: Saied Mahdian, Peter W. Glynn, Yuanyuan Liu

Abstract: The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for equilibrium Markovian rewards. In this paper, we consider the problem of numerically computing the solution to Poisson's equation when the state space is infinit… ▽ More The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for equilibrium Markovian rewards. In this paper, we consider the problem of numerically computing the solution to Poisson's equation when the state space is infinite or very large. In such settings, the state space must be truncated in order to make the problem computationally tractable. In this paper, we provide the first truncation approximation solution to Poisson's equation that comes with provable and computable a posteriori error bounds. Our theory applies to both discrete-time chains and continuous-time jump processes. Through numerical experiments, we show our method can provide highly accurate solutions and tight bounds. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Optimal Transport Relaxations with Application to Wasserstein GANs

Authors: Saied Mahdian, Jose Blanchet, Peter Glynn

Abstract: We propose a family of relaxations of the optimal transport problem which regularize the problem by introducing an additional minimization step over a small region around one of the underlying transporting measures. The type of regularization that we obtain is related to smoothing techniques studied in the optimization literature. When using our approach to estimate optimal transport costs based o… ▽ More We propose a family of relaxations of the optimal transport problem which regularize the problem by introducing an additional minimization step over a small region around one of the underlying transporting measures. The type of regularization that we obtain is related to smoothing techniques studied in the optimization literature. When using our approach to estimate optimal transport costs based on empirical measures, we obtain statistical learning bounds which are useful to guide the amount of regularization, while maintaining good generalization properties. To illustrate the computational advantages of our regularization approach, we apply our method to training Wasserstein GANs. We obtain running time improvements, relative to current benchmarks, with no deterioration in testing performance (via FID). The running time improvement occurs because our new optimality-based threshold criterion reduces the number of expensive iterates of the generating networks, while increasing the number of actor-critic iterations. △ Less

Submitted 7 June, 2019; originally announced June 2019.