-
Forward Reverse Kernel Regression for the Schrödinger bridge problem
Authors:
Denis Belomestny,
John. Schoenmakers
Abstract:
In this paper, we study the Schrödinger Bridge Problem (SBP), which is central to entropic optimal transport. For general reference processes and begin--endpoint distributions, we propose a forward-reverse iterative Monte Carlo procedure to approximate the Schrödinger potentials in a nonparametric way. In particular, we use kernel based Monte Carlo regression in the context of Picard iteration of…
▽ More
In this paper, we study the Schrödinger Bridge Problem (SBP), which is central to entropic optimal transport. For general reference processes and begin--endpoint distributions, we propose a forward-reverse iterative Monte Carlo procedure to approximate the Schrödinger potentials in a nonparametric way. In particular, we use kernel based Monte Carlo regression in the context of Picard iteration of a corresponding fixed point problem. By preserving in the iteration positivity and contractivity in a Hilbert metric sense, we develop a provably convergent algorithm. Furthermore, we provide convergence rates for the potential estimates and prove their optimality. Finally, as an application, we propose a non-nested Monte Carlo procedure for the final dimensional distributions of the Schrödinger Bridge process, based on the constructed potentials and the forward-reverse simulation method for conditional diffusions.
△ Less
Submitted 1 July, 2025;
originally announced July 2025.
-
Primal-dual regression approach for Markov decision processes with general state and action space
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring…
▽ More
We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of the value functions, and, provides tight approximations to the optimal policy. In particular, we prove tight error bounds for the estimated duality gap featuring polynomial dependence on the time horizon, and sublinear dependence on the cardinality/dimension of the possibly infinite state and action space.From a computational point of view the proposed method is efficient since, in contrast to usual duality-based methods for optimal control problems in the literature, the Monte Carlo procedures here involved do not require nested simulations.
△ Less
Submitted 4 October, 2022; v1 submitted 1 October, 2022;
originally announced October 2022.
-
Reinforced optimal control
Authors:
Christian Bayer,
Denis Belomestny,
Paul Hager,
Paolo Pigato,
John Schoenmakers,
Vladimir Spokoiny
Abstract:
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoen…
▽ More
Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](arXiv:1808.02341) proposes to reinforce the basis functions in the case of optimal stopping problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.
△ Less
Submitted 25 March, 2022; v1 submitted 24 November, 2020;
originally announced November 2020.
-
Optimal stopping via reinforced regression
Authors:
Denis Belomestny,
John Schoenmakers,
Vladimir Spokoiny,
Bakhyt Zharkynbay
Abstract:
In this note we propose a new approach towards solving numerically optimal stopping problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical e…
▽ More
In this note we propose a new approach towards solving numerically optimal stopping problems via reinforced regression based Monte Carlo algorithms. The main idea of the method is to reinforce standard linear regression algorithms in each backward induction step by adding new basis functions based on previously estimated continuation values. The proposed methodology is illustrated by a numerical example from mathematical finance.
△ Less
Submitted 1 July, 2019; v1 submitted 7 August, 2018;
originally announced August 2018.
-
Statistical Skorohod embedding problem and its generalizations
Authors:
Denis Belomestny,
John Schoenmakers
Abstract:
Given a Lévy process $L$, we consider the so-called statistical Skorohod embedding problem of recovering the distribution of an independent random time $T$ based on i.i.d. sample from $L_{T}.$ Our approach is based on the genuine use of the Mellin and Laplace transforms. We propose a consistent estimator for the density of $T,$ derive its convergence rates and prove their optimality. It turns out…
▽ More
Given a Lévy process $L$, we consider the so-called statistical Skorohod embedding problem of recovering the distribution of an independent random time $T$ based on i.i.d. sample from $L_{T}.$ Our approach is based on the genuine use of the Mellin and Laplace transforms. We propose a consistent estimator for the density of $T,$ derive its convergence rates and prove their optimality. It turns out that the convergence rates heavily depend on the decay of the Mellin transform of $T.$ We also consider the application of our results to the problem of statistical inference for variance-mean mixture models and for time-changed Lévy processes.
△ Less
Submitted 3 July, 2014;
originally announced July 2014.
