-
Turnpike Property of Stochastic Linear-Quadratic Optimal Control Problems in Large Horizons with Regime Switching I: Homogeneous Cases
Authors:
Hongwei Mei,
Rui Wang,
Jiongmin Yong
Abstract:
This paper is concerned with optimal control problems for a linear homogeneous stochastic differential equation having regime switching with purely quadratic functional in the large time horizons. We establish the so-called turnpike properties for the optimal pairs. The key is to prove a proper convergence of the solutions to the differential Riccati equations to the algebraic Riccati equation. Ev…
▽ More
This paper is concerned with optimal control problems for a linear homogeneous stochastic differential equation having regime switching with purely quadratic functional in the large time horizons. We establish the so-called turnpike properties for the optimal pairs. The key is to prove a proper convergence of the solutions to the differential Riccati equations to the algebraic Riccati equation. Even for the problems without regime switchings, our result provides a refined estimate compared to those in the previous literature, which also provides a new tool for further research.
△ Less
Submitted 10 June, 2025;
originally announced June 2025.
-
Infinite Horizon Mean-Field Linear-Quadratic Optimal Control Problems with Switching and Indefinite-Weighted Costs
Authors:
Hongwei Mei,
Rui Wang,
Qingmeng Wei,
Jiongmin Yong
Abstract:
This paper is concerned with an infinite horizon stochastic linear quadratic (LQ, for short) optimal control problems with conditional mean-field terms in a switching environment. Different from [17], the cost functionals do not have positive-definite weights here. When the problems are merely finite, we construct a sequence of asymptotic optimal controls and derive their closed-loop representatio…
▽ More
This paper is concerned with an infinite horizon stochastic linear quadratic (LQ, for short) optimal control problems with conditional mean-field terms in a switching environment. Different from [17], the cost functionals do not have positive-definite weights here. When the problems are merely finite, we construct a sequence of asymptotic optimal controls and derive their closed-loop representations. For the solvability, an equivalence result between the open-loop and closed-loop cases is established through algebraic Riccati equations and infinite horizon backward stochastic differential equations. It can be seen that the research in [17] with positive-definite weights is a special case of the current paper.
△ Less
Submitted 21 March, 2025;
originally announced March 2025.
-
Linear-Quadratic Optimal Control for Mean-Field Stochastic Differential Equations in Infinite-Horizon with Regime Switching
Authors:
Hongwei Mei,
Qingmeng Wei,
Jiongmin Yong
Abstract:
This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with conditional mean-field term in a switching regime environment. The orthogonal decomposition introduced in [21] has been adopted. Desired algebraic Riccati equations (AREs, for short) and a system of backward stochastic differential equations (BSDEs, for short) in infinite t…
▽ More
This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with conditional mean-field term in a switching regime environment. The orthogonal decomposition introduced in [21] has been adopted. Desired algebraic Riccati equations (AREs, for short) and a system of backward stochastic differential equations (BSDEs, for short) in infinite time horizon with the coefficients depending on the Markov chain have been derived. The determination of closed-loop optimal strategy follows from the solvability of ARE and BSDE. Moreover, the solvability of BSDEs leads to a characterization of open-loop solvability of the optimal control problem.
△ Less
Submitted 1 January, 2025;
originally announced January 2025.
-
A Limit Order Book Model for High Frequency Trading with Rough Volatility
Authors:
Yun Chen-Shue,
Yukun Li,
Jiongmin Yong
Abstract:
We introduce a model for limit order book of a certain security with two main features: First, both the limit orders and market orders for the given asset are allowed to appear and interact with each other. Second, the high frequency trading activities are allowed and described by the scaling limit of nearly-unstable multi-dimensional Hawkes processes with power law decay. The model has been deriv…
▽ More
We introduce a model for limit order book of a certain security with two main features: First, both the limit orders and market orders for the given asset are allowed to appear and interact with each other. Second, the high frequency trading activities are allowed and described by the scaling limit of nearly-unstable multi-dimensional Hawkes processes with power law decay. The model has been derived as a stochastic partial differential equation (SPDE, for short), under certain intuitive identifications. Its diffusion coefficient is determined by a Volterra integral equation driven by a Hawkes process, whose Hurst exponent is less than 1/2 (so that the relevant process is negatively correlated). As a result, the volatility path of the SPDE is rougher than that driven by a (standard) Brownian motion. The well-posedness follows from a result in literature. Hence, a foundation is laid down for further studies in this direction.
△ Less
Submitted 21 December, 2024;
originally announced December 2024.
-
Solvability of Coupled Forward-Backward Volterra Integral Equations
Authors:
Wenyang Li,
Hanxiao Wang,
Jiongmin Yong
Abstract:
Motivated by the optimality system associated with controlled (forward) Volterra integral equations (FVIEs, for short), the well-posedness of coupled forward-backward Voterra integral equations (FBVIEs, for short) is studied. The main feature of FBVIEs is that the unknown $\{(X(t,s),Y(t,s))\}$ has two arguments. By taking $t$ as a parameter and $s$ as a (time) variable, one can regard FBVIE as a s…
▽ More
Motivated by the optimality system associated with controlled (forward) Volterra integral equations (FVIEs, for short), the well-posedness of coupled forward-backward Voterra integral equations (FBVIEs, for short) is studied. The main feature of FBVIEs is that the unknown $\{(X(t,s),Y(t,s))\}$ has two arguments. By taking $t$ as a parameter and $s$ as a (time) variable, one can regard FBVIE as a system of ordinary differential equations (ODEs, for short), with infinite-dimensional space values $\{(X(\cdot,s),Y(\cdot,s));\,s\in[0,T]\}$. To establish the well-posedness of such an FBVIE, a new non-local monotonicity condition is introduced, by which a bridge in infinite-dimensional spaces is constructed. Then by generalizing the method of continuation developed by \cite{Hu-Peng1995,Yong1997,Peng-Wu1999} for differential equations, we have established the well-posedness of FBVIEs.The key is to apply the chain rule to the mapping $t\mapsto\big[\int_\cdot^T\langle Y(s,s),X(s,\cdot)\rangle ds +\langle G(X(T,T)),X(T,\cdot)\rangle\big](t)$.
△ Less
Submitted 5 December, 2024;
originally announced December 2024.
-
Long-Time Behaviors of Stochastic Linear-Quadratic Optimal Control Problems
Authors:
Jiamin Jian,
Sixian Jin,
Qingshuo Song,
Jiongmin Yong
Abstract:
This paper investigates the asymptotic behavior of the solution to a linear-quadratic stochastic optimal control problems. The so-called probability cell problem is introduced the first time. It serves as the probability interpretation of the well-known cell problem in the homogenization of Hamilton-Jacobi equations. By establishing a connection between this problem and the ergodic cost problem, w…
▽ More
This paper investigates the asymptotic behavior of the solution to a linear-quadratic stochastic optimal control problems. The so-called probability cell problem is introduced the first time. It serves as the probability interpretation of the well-known cell problem in the homogenization of Hamilton-Jacobi equations. By establishing a connection between this problem and the ergodic cost problem, we reveal the turnpike properties of the linear-quadratic stochastic optimal control problems from various perspectives.
△ Less
Submitted 17 September, 2024;
originally announced September 2024.
-
Long-Time Behavior of Zero-Sum Linear-Quadratic Stochastic Differential Games
Authors:
Jingrui Sun,
Jiongmin Yong
Abstract:
The paper investigates the long-time behavior of zero-sum linear-quadratic stochastic differential games, aiming to demonstrate that, under appropriate conditions, both the saddle strategy and the optimal state process exhibit the exponential turnpike property. Namely, for the majority of the time horizon, the distributions of the saddle strategy and the optimal state process closely stay near cer…
▽ More
The paper investigates the long-time behavior of zero-sum linear-quadratic stochastic differential games, aiming to demonstrate that, under appropriate conditions, both the saddle strategy and the optimal state process exhibit the exponential turnpike property. Namely, for the majority of the time horizon, the distributions of the saddle strategy and the optimal state process closely stay near certain (time-invariant) distributions $ν_1^*$, $ν_2^*$ and $μ^*$, respectively. Additionally, as a byproduct, we solve the infinite horizon version of the differential game and derive closed-loop representations for its open-loop saddle strategy, which has not been proved in the literature.
△ Less
Submitted 4 June, 2024;
originally announced June 2024.
-
Linear-Quadratic Optimal Control Problem for Mean-Field Stochastic Differential Equations with a Type of Random Coefficients
Authors:
Hongwei Mei,
Qingmeng Wei,
Jiongmin Yong
Abstract:
Motivated by linear-quadratic optimal control problems (LQ problems, for short) for mean-field stochastic differential equations (SDEs, for short) with the coefficients containing regime switching governed by a Markov chain, we consider an LQ problem for an SDE with the coefficients being adapted to a filtration independent of the Brownian motion driving the control system. Classical approach of c…
▽ More
Motivated by linear-quadratic optimal control problems (LQ problems, for short) for mean-field stochastic differential equations (SDEs, for short) with the coefficients containing regime switching governed by a Markov chain, we consider an LQ problem for an SDE with the coefficients being adapted to a filtration independent of the Brownian motion driving the control system. Classical approach of completing the square is applied to the current problem and obvious shortcomings are indicated. Open-loop and closed-loop solvability are introduced and characterized.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
Multi-Dimensional Super-Linear Backward Stochastic Volterra Integral Equations
Authors:
Shengjun Fan,
Tianxiao Wang,
Jiongmin Yong
Abstract:
In this paper, a systematic investigation is carried out for the general solvability of multi-dimensional backward stochastic Volterra integral equations (BSVIEs) with the generators being super-linear in the adjustment variable $Z$. Two major situations are discussed: (i) When the free term is bounded with the dependence of the generator on $Z$ being of ``diagonally strictly'' quadratic growth an…
▽ More
In this paper, a systematic investigation is carried out for the general solvability of multi-dimensional backward stochastic Volterra integral equations (BSVIEs) with the generators being super-linear in the adjustment variable $Z$. Two major situations are discussed: (i) When the free term is bounded with the dependence of the generator on $Z$ being of ``diagonally strictly'' quadratic growth and being sub-quadratically coupled with off-diagonal components; (ii) When the free term is unbounded having exponential moments of arbitrary order with the dependence of the generator on $Z$ being diagonally no more than quadratic and being independent of off-diagonal components. Besides, for the case that the generator is super-quadratic in $Z$, some negative results are presented.
△ Less
Submitted 8 November, 2022;
originally announced November 2022.
-
Turnpike Properties for Mean-Field Linear-Quadratic Optimal Control Problems
Authors:
Jingrui Sun,
Jiongmin Yong
Abstract:
This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static…
▽ More
This paper is concerned with an optimal control problem for a mean-field linear stochastic differential equation with a quadratic functional in the infinite time horizon. Under suitable conditions, including the stabilizability, the (strong) exponential, integral, and mean-square turnpike properties for the optimal pair are established. The keys are to correctly formulate the corresponding static optimization problem and find the equations determining the correction processes. These have revealed the main feature of the stochastic problems which are significantly different from the deterministic version of the theory.
△ Less
Submitted 23 September, 2022;
originally announced September 2022.
-
Optimal Controls for Forward-Backward Stochastic Differential Equations: Time-Inconsistency and Time-Consistent Solutions
Authors:
Hanxiao Wang,
Jiongmin Yong,
Chao Zhou
Abstract:
This paper is concerned with an optimal control problem for a forward-backward stochastic differential equation (FBSDE, for short) with a recursive cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). It is found that such an optimal control problem is time-inconsistent in general, even if the cost functional is reduced to a classical Bolza type one as…
▽ More
This paper is concerned with an optimal control problem for a forward-backward stochastic differential equation (FBSDE, for short) with a recursive cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). It is found that such an optimal control problem is time-inconsistent in general, even if the cost functional is reduced to a classical Bolza type one as in Peng [50], Lim-Zhou [41], and Yong [74]. Therefore, instead of finding a global optimal control (which is time-inconsistent), we will look for a time-consistent and locally optimal equilibrium strategy, which can be constructed via the solution of an associated equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation. A verification theorem for the local optimality of the equilibrium strategy is proved by means of the generalized Feynman-Kac formula for BSVIEs and some stability estimates of the representation for parabolic partial differential equations (PDEs, for short). Under certain conditions, it is proved that the equilibrium HJB equation, which is a nonlocal PDE, admits a unique classical solution. As special cases and applications, the linear-quadratic problems, a mean-variance model, a social planner problem with heterogeneous Epstein-Zin utilities, and a Stackelberg game are briefly investigated. It turns out that our framework can cover not only the optimal control problems for FBSDEs studied in [50,41,74], and so on, but also the problems of the general discounting and some nonlinear appearance of conditional expectations for the terminal state, studied in Yong [75,77] and Björk-Khapko-Murgoci [7].
△ Less
Submitted 19 September, 2022;
originally announced September 2022.
-
A Stochastic Maximum Principle Approach for Reinforcement Learning with Parameterized Environment
Authors:
Richard Archibald,
Feng Bao,
Jiongmin Yong
Abstract:
In this work, we introduce a stochastic maximum principle (SMP) approach for solving the reinforcement learning problem with the assumption that the unknowns in the environment can be parameterized based on physics knowledge. For the development of numerical algorithms, we shall apply an effective online parameter estimation method as our exploration technique to estimate the environment parameter…
▽ More
In this work, we introduce a stochastic maximum principle (SMP) approach for solving the reinforcement learning problem with the assumption that the unknowns in the environment can be parameterized based on physics knowledge. For the development of numerical algorithms, we shall apply an effective online parameter estimation method as our exploration technique to estimate the environment parameter during the training procedure, and the exploitation for the optimal policy will be achieved by an efficient backward action learning method for policy improvement under the SMP framework. Numerical experiments will be presented to demonstrate that our SMP approach for reinforcement learning can produce reliable control policy, and the gradient descent type optimization in the SMP solver requires less training episodes compared with the standard dynamic programming principle based methods.
△ Less
Submitted 6 January, 2023; v1 submitted 3 August, 2022;
originally announced August 2022.
-
Backward Stochastic Differential Equations and Backward Stochastic Volterra Integral Equations with Anticipating Generators
Authors:
Hanxiao Wang,
Jiongmin Yong,
Chao Zhou
Abstract:
For a backward stochastic differential equation (BSDE, for short), when the generator is not progressively measurable, it might not admit adapted solutions, shown by an example. However, for backward stochastic Volterra integral equations (BSVIEs, for short), the generators are allowed to be anticipating. This gives, among other things, an essential difference between BSDEs and BSVIEs. Under some…
▽ More
For a backward stochastic differential equation (BSDE, for short), when the generator is not progressively measurable, it might not admit adapted solutions, shown by an example. However, for backward stochastic Volterra integral equations (BSVIEs, for short), the generators are allowed to be anticipating. This gives, among other things, an essential difference between BSDEs and BSVIEs. Under some proper conditions, the well-posedness of such kinds of BSVIEs is established. Further, the results are extended to path-dependent BSVIEs, in which the generators can depend on the future paths of unknown processes. An additional finding is that for path-dependent BSVIEs, in general, the situation of anticipating generators is not avoidable and the adaptedness condition similar to that imposed for anticipated BSDEs by Peng--Yang [22] is not necessary.
△ Less
Submitted 24 June, 2022;
originally announced June 2022.
-
Spike Variations for Stochastic Volterra Integral Equations
Authors:
Tianxiao Wang,
Jiongmin Yong
Abstract:
Spike variation technique plays a crucial role in deriving Pontryagin's type maximum principle of optimal controls for differential equations of several types, including ordinary differential equations (ODEs), partial differential equations (PDEs), and stochastic differentia equations (SDEs), when the control domains are not assumed to be convex. This technique also applies to (deterministic forwa…
▽ More
Spike variation technique plays a crucial role in deriving Pontryagin's type maximum principle of optimal controls for differential equations of several types, including ordinary differential equations (ODEs), partial differential equations (PDEs), and stochastic differentia equations (SDEs), when the control domains are not assumed to be convex. This technique also applies to (deterministic forward) Volterra intrgral equations (FVIEs). It is natural to expect that such a technique could be extended to the case of (forward) stochastic Volterra integral equations (FSVIEs). However, by mimicking the case of SDEs, one encounters an essential difficulty of handling an involved quadratic term. To overcome the difficulty, we introduce an auxiliary process for which one can use Itô's formula, and adopt a trick used in linear-quadratic stochastic optimal control problems. Then a suitable representation of the above-mentioned quadratic form is obtained, and the second order adjoint equations are derived. Consequently, the maximum principle of Pontryagin type is established. Some relevant extensions are investigated as well.
△ Less
Submitted 11 September, 2022; v1 submitted 26 May, 2022;
originally announced May 2022.
-
Linear-Quadratic Optimal Controls for Stochastic Volterra Integral Equations: Causal State Feedback and Path-Dependent Riccati Equations
Authors:
Hanxiao Wang,
Jiongmin Yong,
Chao Zhou
Abstract:
A linear-quadratic optimal control problem for a forward stochastic Volterra integral equation (FSVIE, for short) is considered. Under the usual convexity conditions, open-loop optimal control exists, which can be characterized by the optimality system, a coupled system of an FSVIE and a Type-II backward SVIE (BSVIE, for short). To obtain a causal state feedback representation for the open-loop op…
▽ More
A linear-quadratic optimal control problem for a forward stochastic Volterra integral equation (FSVIE, for short) is considered. Under the usual convexity conditions, open-loop optimal control exists, which can be characterized by the optimality system, a coupled system of an FSVIE and a Type-II backward SVIE (BSVIE, for short). To obtain a causal state feedback representation for the open-loop optimal control, a path-dependent Riccati equation for an operator-valued function is introduced, via which the optimality system can be decoupled. In the process of decoupling, a Type-III BSVIE is introduced whose adapted solution can be used to represent the adapted M-solution of the corresponding Type-II BSVIE. Under certain conditions, it is proved that the path-dependent Riccati equation admits a unique solution, which means that the decoupling field for the optimality system is found. Therefore a causal state feedback representation of the open-loop optimal control is constructed. An additional interesting finding is that when the control only appears in the diffusion term, not in the drift term of the state system, the causal state feedback reduces to a Markovian state feedback.
△ Less
Submitted 19 April, 2022;
originally announced April 2022.
-
Turnpike Properties for Stochastic Linear-Quadratic Optimal Control Problems
Authors:
Jingrui Sun,
Hanxiao Wang,
Jiongmin Yong
Abstract:
This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time horizon $[0,T]$ as $T\rightarrow\infty$. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike…
▽ More
This paper analyzes the limiting behavior of stochastic linear-quadratic optimal control problems in finite time horizon $[0,T]$ as $T\rightarrow\infty$. The so-called turnpike properties are established for such problems, under stabilizability condition which is weaker than the controllability, normally imposed in the similar problem for ordinary differential systems. In dealing with the turnpike problem, a crucial issue is to determine the corresponding static optimization problem. Intuitively mimicking deterministic situations, it seems to be natural to include both the drift and the diffusion as constraints in the static optimization problem. However, this would lead us to a wrong direction. It is found that the correct static problem should contain the diffusion as a part of the objective function, which reveals a deep feature of the stochastic turnpike problem.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
Causal State Feedback Representation for Linear Quadratic Optimal Control Problems of Singular Volterra Integral Equations
Authors:
Shuo Han,
Ping Lin,
Jiongmin Yong
Abstract:
This paper is concerned with a linear quadratic optimal control for a class of singular Volterra integral equations. Under proper convexity conditions, optimal control uniquely exists, and it could be characterized via Frechet derivative of the quadratic functional in a Hilbert space or via maximum principle type necessary conditions. However, these (equivalent) characterizations have a shortcomin…
▽ More
This paper is concerned with a linear quadratic optimal control for a class of singular Volterra integral equations. Under proper convexity conditions, optimal control uniquely exists, and it could be characterized via Frechet derivative of the quadratic functional in a Hilbert space or via maximum principle type necessary conditions. However, these (equivalent) characterizations have a shortcoming that the current value of the optimal control depends on the future values of the optimal state. Practically, this is not feasible. The main purpose of this paper is to obtain a causal state feedback representation of the optimal control.
△ Less
Submitted 16 September, 2021;
originally announced September 2021.
-
Non-Equivalence of Stochastic Optimal Control Problems with Open and Closed Loop Controls
Authors:
Jiongmin Yong,
Jianfeng Zhang
Abstract:
For an optimal control problem of an Itô's type stochastic differential equation, the control process could be taken as open-loop or closed-loop forms. In the standard literature, provided appropriate regularity, the value functions under these two types of controls are equal and are the unique (viscosity) solution to the corresponding (path-dependent) HJB equation. In this short note, we provide…
▽ More
For an optimal control problem of an Itô's type stochastic differential equation, the control process could be taken as open-loop or closed-loop forms. In the standard literature, provided appropriate regularity, the value functions under these two types of controls are equal and are the unique (viscosity) solution to the corresponding (path-dependent) HJB equation. In this short note, we provide a counterexample in the path dependent setting showing that these value functions can be different in general.
△ Less
Submitted 7 March, 2021; v1 submitted 26 December, 2020;
originally announced December 2020.
-
Remarks on Viscosity Super-Solutions of Quasi-Variational Inequalities
Authors:
Yue Zhou,
Xinwei Feng,
Jiongmin Yong
Abstract:
For Hamilton-Jacobi-Bellman (HJB) equations, with the standard definitions of viscosity super-solution and sub-solution, it is known that there is a comparison between any (viscosity) super-solutions and sub-solutions. This should be the same for HJB type quasi-variational inequalities (QVIs) arising from optimal impulse control problems. However, according to a natural adoption of the definition…
▽ More
For Hamilton-Jacobi-Bellman (HJB) equations, with the standard definitions of viscosity super-solution and sub-solution, it is known that there is a comparison between any (viscosity) super-solutions and sub-solutions. This should be the same for HJB type quasi-variational inequalities (QVIs) arising from optimal impulse control problems. However, according to a natural adoption of the definition found in Barles 1985, Barles 1985b, the uniqueness of the viscosity solution could be guaranteed, but the comparison between viscosity super- and sub-solutions could not be guaranteed. This paper introduces a modification of the definition for the viscosity super-solution of HJB type QVIs so that the desired comparison theorem will hold.
△ Less
Submitted 5 February, 2021; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Infinite Horizon Linear Quadratic Overtaking Optimal Control Problems
Authors:
Jianping Huang,
Jiongmin Yong,
Hua-Cheng Zhou
Abstract:
A linear control system with quadratic cost functional over infinite time horizon is considered without assuming controllability/stabilizability condition and the global integrability condition for the nonhomogeneous term of the state equation and the weight functions in the linear terms in the running cost rate function. Classical approaches do not apply for such kind of problems. Existence and n…
▽ More
A linear control system with quadratic cost functional over infinite time horizon is considered without assuming controllability/stabilizability condition and the global integrability condition for the nonhomogeneous term of the state equation and the weight functions in the linear terms in the running cost rate function. Classical approaches do not apply for such kind of problems. Existence and non-existence of overtaking optimal controls in various cases are established. Some concrete examples are presented. These results show that the overtaking optimality approach can be used to solve some of the above-mentioned problems and at the same time, the limitation of this approach is also revealed.
△ Less
Submitted 22 August, 2020;
originally announced August 2020.
-
Mean-Field Linear-Quadratic Stochastic Differential Games in an Infinite Horizon
Authors:
Xun Li,
Jingtao Shi,
Jiongmin Yong
Abstract:
This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. Existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of th…
▽ More
This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. Existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existence of a closed-loop Nash equilibrium is characterized by the solvability of a system of two coupled symmetric algebraic Riccati equations. Two-person mean-field linear-quadratic zero-sum stochastic differential games in an infinite time horizon are also considered. Both the existence of open-loop and closed-loop saddle points are characterized by the solvability of a system of two coupled generalized algebraic Riccati equations with static stabilizing solutions. Mean-field linear-quadratic stochastic optimal control problems in an infinite horizon are discussed as well, for which it is proved that the open-loop solvability and closed-loop solvability are equivalent.
△ Less
Submitted 7 April, 2021; v1 submitted 12 July, 2020;
originally announced July 2020.
-
An efficient numerical algorithm for solving data driven feedback control problems
Authors:
Richard Archibald,
Feng Bao,
Jiongmin Yong,
Tao Zhou
Abstract:
The goal of this paper is to solve a class of stochastic optimal control problems numerically, in which the state process is governed by an Itô type stochastic differential equation with control process entering both in the drift and the diffusion, and is observed partially. The optimal control of feedback form is determined based on the available observational data. We call this type of control p…
▽ More
The goal of this paper is to solve a class of stochastic optimal control problems numerically, in which the state process is governed by an Itô type stochastic differential equation with control process entering both in the drift and the diffusion, and is observed partially. The optimal control of feedback form is determined based on the available observational data. We call this type of control problems the data driven feedback control. The computational framework that we introduce to solve such type of problems aims to find the best estimate for the optimal control as a conditional expectation given the observational information. To make our method feasible in providing timely feedback to the controlled system from data, we develop an efficient stochastic optimization algorithm to implement our computational framework.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
Continuity of the Value Function for Deterministic Optimal Impulse Control with Terminal State Constraint
Authors:
Yue Zhou,
Xinwei Feng,
Jiongmin Yong
Abstract:
Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is continuous. Then by a Bellman dynamic programming method, the corresponding Hamilt…
▽ More
Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is continuous. Then by a Bellman dynamic programming method, the corresponding Hamilton-Jacobi-Bellman type quasi-variational inequality (QVI, for short) is derived for which the value function is a viscosity solution. The issue of whether the value function is characterized as the unique viscosity solution to this QVI is carefully addressed and the answer is left open challengingly.
△ Less
Submitted 7 November, 2020; v1 submitted 20 May, 2020;
originally announced May 2020.
-
A Finite Horizon Optimal Stochastic Impulse Control Problem with A Decision Lag
Authors:
Chang Li,
Jiongmin Yong
Abstract:
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the depende…
▽ More
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibit some special feature of the problem. The value function of this optimal impulse control problem is characterized as the unique viscosity solution to the corresponding HJB equation. An optimal impulse control is constructed provided the value function is given. Moreover, a limiting case with the waiting time approaching $0$ is discussed.
△ Less
Submitted 6 February, 2021; v1 submitted 9 May, 2020;
originally announced May 2020.
-
Optimal Ergodic Control of Linear Stochastic Differential Equations with Quadratic Cost Functionals Having Indefinite Weights
Authors:
Hongwei Mei,
Qingmeng Wei,
Jiongmin Yong
Abstract:
An optimal ergodic control problem (EC problem, for short) is investigated for a linear stochastic differential equation with quadratic cost functional. Constant nonhomogeneous terms, not all zero, appear in the state equation, which lead to the asymptotic limit of the state non-zero. Under the stabilizability condition, for any (admissible) closed-loop strategy, an invariant measure is proved to…
▽ More
An optimal ergodic control problem (EC problem, for short) is investigated for a linear stochastic differential equation with quadratic cost functional. Constant nonhomogeneous terms, not all zero, appear in the state equation, which lead to the asymptotic limit of the state non-zero. Under the stabilizability condition, for any (admissible) closed-loop strategy, an invariant measure is proved to exist, which makes the ergodic cost functional well-defined and the EC problem well-formulated. Sufficient conditions, including those allowing the weighting matrices of cost functional to be indefinite, are introduced for finiteness and solvability for the EC problem. Some comparisons are made between the solvability of EC problem and the closed-loop solvability of stochastic linear quadratic optimal control problem in the infinite horizon. Regularized EC problem is introduced to be used to obtain the optimal value of the EC problem.
△ Less
Submitted 23 April, 2020;
originally announced April 2020.
-
Path Dependent Feynman-Kac Formula for Forward Backward Stochastic Volterra Integral Equations
Authors:
Hanxiao Wang,
Jiongmin Yong,
Jianfeng Zhang
Abstract:
This paper is concerned with the relationship between forward-backward stochastic Volterra integral equations (FBSVIEs, for short) and a system of (non-local in time) path dependent partial differential equations (PPDEs, for short). Due to the nature of Volterra type equations, the usual flow property (or semigroup property) does not hold. Inspired by Viens-Zhang \cite{Viens-Zhang-2019} and Wang-Y…
▽ More
This paper is concerned with the relationship between forward-backward stochastic Volterra integral equations (FBSVIEs, for short) and a system of (non-local in time) path dependent partial differential equations (PPDEs, for short). Due to the nature of Volterra type equations, the usual flow property (or semigroup property) does not hold. Inspired by Viens-Zhang \cite{Viens-Zhang-2019} and Wang-Yong \cite{Wang-Yong-2019}, auxiliary processes are introduced so that the flow property of adapted solutions to the FBSVIEs is recovered in a suitable sense, and thus the functional Itô's formula is applicable. Having achieved this stage, a natural PPDE is found so that the adapted solution of the backward SVIEs admits a representation in terms of the solution to the forward SVIE via the solution to a PPDE. On the other hand, the solution of the PPDE admits a representation in terms of adapted solution to the (path dependent) FBSVIE, which is referred to as a Feynman-Kac formula. This leads to the existence and uniqueness of a classical solution to the PPDE, under smoothness conditions on the coefficients of the FBSVIEs. Further, when the smoothness conditions are relaxed with the backward component of FBSVIE being one-dimensional, a new (and suitable) notion of viscosity solution is introduced for the PPDE, for which a comparison principle of the viscosity solutions is established, leading to the uniqueness of the viscosity solution. Finally, some results have been extended to coupled FBSVIEs and type-II BSVIEs, and a representation formula for the path derivatives of PPDE solution is obtained by a closer investigation of linear FBSVIEs.
△ Less
Submitted 23 January, 2021; v1 submitted 13 April, 2020;
originally announced April 2020.
-
Social Optima in Mean Field Linear-Quadratic-Gaussian Control with Volatility Uncertainty
Authors:
Jianhui Huang,
Bing-Chang Wang,
Jiongmin Yong
Abstract:
This paper examines mean field linear-quadratic-Gaussian (LQG) social optimum control with volatility-uncertain common noise. The diffusion terms in the dynamics of agents contain an unknown volatility process driven by a common noise. We apply a robust optimization approach in which all agents view volatility uncertainty as an adversarial player. Based on the principle of person-by-person optimal…
▽ More
This paper examines mean field linear-quadratic-Gaussian (LQG) social optimum control with volatility-uncertain common noise. The diffusion terms in the dynamics of agents contain an unknown volatility process driven by a common noise. We apply a robust optimization approach in which all agents view volatility uncertainty as an adversarial player. Based on the principle of person-by-person optimality and a two-step-duality technique for stochastic variational analysis, we construct an auxiliary optimal control problem for a representative agent. Through solving this problem combined with a consistent mean field approximation, we design a set of decentralized strategies, which are further shown to be asymptotically social optimal by perturbation analysis.
△ Less
Submitted 13 December, 2019;
originally announced December 2019.
-
Time-Inconsistent Stochastic Optimal Control Problems and Backward Stochastic Volterra Integral Equations
Authors:
Hanxiao Wang,
Jiongmin Yong
Abstract:
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general discounting (including exponential and non-exponential) situation with a recursive feature. It is known that such a problem is time-inconsistent in general. Ther…
▽ More
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general discounting (including exponential and non-exponential) situation with a recursive feature. It is known that such a problem is time-inconsistent in general. Therefore, instead of finding a global optimal control, we look for a time-consistent locally near optimal equilibrium strategy. With the idea of multi-person differential games, a family of approximate equilibrium strategies is constructed associated with partitions of the time intervals. By sending the mesh size of the time interval partition to zero, an equilibrium Hamilton--Jacobi--Bellman (HJB, for short) equation is derived, through which the equilibrium valued function and an equilibrium strategy are obtained. Under certain conditions, a verification theorem is proved and the well-posedness of the equilibrium HJB is established. As a sort of Feynman-Kac formula for the equilibrium HJB equation, a new class of BSVIEs (containing the diagonal value $Z(r,r)$ of $Z(\cd,\cd)$) is naturally introduced and the well-posedness of such kind of equations is briefly presented.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Anti-lock Brake System for Integrated Electric Parking Brake Actuator Based on Sliding-mode Control
Authors:
Dongliang Wang,
Yiyong Yang,
Wei Yu,
Jiawang Yong,
Xiaoxu Dong
Abstract:
Integrated electric parking brake (iEPB) is popularizing on passenger cars due to its easier operation and automatic functions. As a parking brake, EPB have to act as the secondary brake system in case of hydraulic brake failure. To guarantee the stability and safety of a car during iEPB braking, the rear slip ratio has to be controlled accurately within the optimized value to get the shortest bra…
▽ More
Integrated electric parking brake (iEPB) is popularizing on passenger cars due to its easier operation and automatic functions. As a parking brake, EPB have to act as the secondary brake system in case of hydraulic brake failure. To guarantee the stability and safety of a car during iEPB braking, the rear slip ratio has to be controlled accurately within the optimized value to get the shortest brake distance without undesired loss of control. In this paper, a sliding-mode controller (SMC) is investigated to achieve rear-wheel anti-lock brake control, which is robust against uncertainties and disturbance of the parameters. And a sliding-mode observer (SMO) is present to estimate the load torque of d.c. motor and calculate the brake torque. The tyre/road friction coefficient estimator is designed to obtain the optimal rear slip ratio timely. The simulation model of iEPB system is initially constructed in AMESim and the vehicle model is built in MATLAB/Simulink, and the complete system is co-simulated by these two software simultaneously with different road conditions. Simulation results show that the proposed observer and estimator are feasible. This study may provide a useful method to realize rear slip ratio control so that the safety and stability of vehicle could be improved significantly in specified condition
△ Less
Submitted 8 November, 2018; v1 submitted 25 October, 2018;
originally announced October 2018.
-
Recursive Utility Processes, Dynamic Risk Measures and Quadratic Backward Stochastic Volterra Integral Equations
Authors:
Hanxiao Wang,
Jingrui Sun,
Jiongmin Yong
Abstract:
For an $\cF_T$-measurable payoff of a European type contingent claim, the recursive utility process/dynamic risk measure can be described by the adapted solution to a backward stochastic differential equation (BSDE). However, for an $\cF_T$-measurable stochastic process (called a position process, not necessarily $\dbF$-adapted), mimicking BSDE's approach will lead to a time-inconsistent recursive…
▽ More
For an $\cF_T$-measurable payoff of a European type contingent claim, the recursive utility process/dynamic risk measure can be described by the adapted solution to a backward stochastic differential equation (BSDE). However, for an $\cF_T$-measurable stochastic process (called a position process, not necessarily $\dbF$-adapted), mimicking BSDE's approach will lead to a time-inconsistent recursive utility/dynamic risk measure. It is found that a more proper approach is to use the adapted solution to a backward stochastic Volterra integral equation (BSVIE). The corresponding notions are called equilibrium recursive utility and equilibrium dynamic risk measure, respectively. Motivated by this, the current paper is concerned with BSVIEs whose generators are allowed to have quadratic growth (in $Z(t,s)$). The existence and uniqueness for both the so-called adapted solutions and adapted M-solutions are established. A comparison theorem for adapted solutions to the so-called Type-I BSVIEs is established as well. As consequences of these results, some general continuous-time equilibrium dynamic risk measures and equilibrium recursive utility processes are constructed.
△ Less
Submitted 23 December, 2019; v1 submitted 23 October, 2018;
originally announced October 2018.
-
Optimization of the Principal Eigenvalue for Elliptic Operators
Authors:
Hongwei Lou,
Jiongmin Yong
Abstract:
Maximization and minimization problems of the principle eigenvalue for divergence form second order elliptic operators with the Dirichlet boundary condition are considered. The principal eigen map of such elliptic operators is introduced and some basic properties of this map, including continuity, concavity, and differentiability with respect to the parameter in the diffusibility matrix, are estab…
▽ More
Maximization and minimization problems of the principle eigenvalue for divergence form second order elliptic operators with the Dirichlet boundary condition are considered. The principal eigen map of such elliptic operators is introduced and some basic properties of this map, including continuity, concavity, and differentiability with respect to the parameter in the diffusibility matrix, are established. For maximization problem, the admissible control set is convexified to get the existence of an optimal convexified relaxed solution. Whereas, for minimization problem, the relaxation of the problem under $H$-convergence is introduced to get an optimal $H$-relaxed solution for certain interesting special cases. Some necessary optimality conditions are presented for both problems and a couple of illustrative examples are presented as well.
△ Less
Submitted 27 August, 2019; v1 submitted 29 September, 2018;
originally announced October 2018.
-
Indefinite Stochastic Linear-Quadratic Optimal Control Problems with Random Coefficients: Closed-Loop Representation of Open-Loop Optimal Controls
Authors:
Jingrui Sun,
Jie Xiong,
Jiongmin Yong
Abstract:
This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost fun…
▽ More
This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost functional (with respect to the control) is necessary; and the uniform convexity, which is slightly stronger, turns out to be sufficient, which also leads to the unique solvability of the associated stochastic Riccati equation. Further, it is shown that the open-loop optimal control admits a closed-loop representation. In addition, some sufficient conditions are obtained for the uniform convexity of the cost functional, which are strictly general than the classical conditions that the weighting matrix-valued processes are positive (semi-)definite.
△ Less
Submitted 10 November, 2019; v1 submitted 1 September, 2018;
originally announced September 2018.
-
Weak Closed-Loop Solvability of Stochastic Linear-Quadratic Optimal Control Problems
Authors:
Jingrui Sun,
Hanxiao Wang,
Jiongmin Yong
Abstract:
Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the u…
▽ More
Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.
△ Less
Submitted 13 June, 2018;
originally announced June 2018.
-
Backward Stochastic Volterra Integral Equations--- Representation of Adapted Solutions
Authors:
Tianxiao Wang,
Jiongmin Yong
Abstract:
For backward stochastic Volterra integral equations (BSVIEs, for short), under some mild conditions, the so-called adapted solutions or adapted M-solutions uniquely exist. However, satisfactory regularity of the solutions is difficult to obtain in general. Inspired by the decoupling idea of forward-backward stochastic differential equations, in this paper, for a class of BSVIEs, a representation o…
▽ More
For backward stochastic Volterra integral equations (BSVIEs, for short), under some mild conditions, the so-called adapted solutions or adapted M-solutions uniquely exist. However, satisfactory regularity of the solutions is difficult to obtain in general. Inspired by the decoupling idea of forward-backward stochastic differential equations, in this paper, for a class of BSVIEs, a representation of adapted M-solutions is established by means of the so-called representation partial differential equations and (forward) stochastic differential equations. Well-posedness of the representation partial differential equations are also proved in certain sense.
△ Less
Submitted 10 February, 2018;
originally announced February 2018.
-
Equilibrium Strategies for Time-Inconsistent Stochastic Switching Systems
Authors:
Hongwei Mei,
Jiongmin Yong
Abstract:
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) local…
▽ More
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is time-inconsistent in general. Therefore, instead of finding a global optimal control (which is not possible), we look for a time-consistent (approximately) locally optimal equilibrium strategy. Such a strategy can be represented through the solution to a system of partial differential equations, called an equilibrium Hamilton-Jacob-Bellman (HJB, for short) equation which is constructed via a sequence of multi-person differential games. A verification theorem is proved and, under proper conditions, the well-posedness of the equilibrium HJB equation is established as well.
△ Less
Submitted 27 December, 2017;
originally announced December 2017.
-
Controlled Singular Volterra Integral Equations and Pontryagin Maximum Principle
Authors:
Ping Lin,
Jiongmin Yong
Abstract:
This paper is concerned with a class of controlled singular Volterra integral equations, which could be used to describe problems involving memories. The well-known fractional order ordinary differential equations of the Riemann--Liouville or Caputo types are strictly special cases of the equations studied in this paper. Well-posedness and some regularity results in proper spaces are established f…
▽ More
This paper is concerned with a class of controlled singular Volterra integral equations, which could be used to describe problems involving memories. The well-known fractional order ordinary differential equations of the Riemann--Liouville or Caputo types are strictly special cases of the equations studied in this paper. Well-posedness and some regularity results in proper spaces are established for such kind of questions. For the associated optimal control problem, by using a Liapounoff's type theorem and the spike variation technique, we establish a Pontryagin's type maximum principle for optimal controls. Different from the existing literature, our method enables us to deal with the problem without assuming regularity conditions on the controls, the convexity condition on the control domain, and some additional unnecessary conditions on the nonlinear terms of the integral equation and the cost functional.
△ Less
Submitted 16 December, 2017;
originally announced December 2017.
-
Second-Order Necessary Conditions for Optimal Control of Semilinear Elliptic Equations with Leading Term Containing Controls
Authors:
Hongwei Lou,
Jiongmin Yong
Abstract:
An optimal control problem for a semilinear elliptic equation of divergence form is considered. Both the leading term and the semilinear term of the state equation contain the control. The well-known Pontryagin type maximum principle for the optimal controls is the first-order necessary condition. When such a first-order necessary condition is singular in some sense, certain type of the second-ord…
▽ More
An optimal control problem for a semilinear elliptic equation of divergence form is considered. Both the leading term and the semilinear term of the state equation contain the control. The well-known Pontryagin type maximum principle for the optimal controls is the first-order necessary condition. When such a first-order necessary condition is singular in some sense, certain type of the second-order necessary condition will come in naturally. The aim of this paper is to explore such kind of conditions for our optimal control problem.
△ Less
Submitted 25 March, 2017;
originally announced March 2017.
-
Linear Quadratic Stochastic Optimal Control Problems with Operator Coefficients: Open-Loop Solutions
Authors:
Qingmeng Wei,
Jiongmin Yong,
Zhiyong Yu
Abstract:
An optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic optimal control problems for mean-field stochastic differential equations. O…
▽ More
An optimal control problem is considered for linear stochastic differential equations with quadratic cost functional. The coefficients of the state equation and the weights in the cost functional are bounded operators on the spaces of square integrable random variables. The main motivation of our study is linear quadratic optimal control problems for mean-field stochastic differential equations. Open-loop solvability of the problem is investigated, which is characterized as the solvability of a system of linear coupled forward-backward stochastic differential equations (FBSDE, for short) with operator coefficients. Under proper conditions, the well-posedness of such an FBSDE is established, which leads to the existence of an open-loop optimal control. Finally, as an application of our main results, a general mean-field linear quadratic control problem in the open-loop case is solved.
△ Less
Submitted 14 January, 2019; v1 submitted 10 January, 2017;
originally announced January 2017.
-
Stochastic Linear Quadratic Optimal Control Problems in Infinite Horizon
Authors:
Jingrui Sun,
Jiongmin Yong
Abstract:
This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with constant coefficients. It is proved that the non-emptiness of the admissible control set for all initial state is equivalent to the $L^2$-stabilizability of the control system, which in turn is equivalent to the existence of a positive solution to an algebraic Riccati equat…
▽ More
This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with constant coefficients. It is proved that the non-emptiness of the admissible control set for all initial state is equivalent to the $L^2$-stabilizability of the control system, which in turn is equivalent to the existence of a positive solution to an algebraic Riccati equation (ARE, for short). Different from the finite horizon case, it is shown that both the open-loop and closed-loop solvabilities of the LQ problem are equivalent to the existence of a static stabilizing solution to the associated generalized ARE. Moreover, any open-loop optimal control admits a closed-loop representation. Finally, the one-dimensional case is worked out completely to illustrate the developed theory.
△ Less
Submitted 17 October, 2016;
originally announced October 2016.
-
Linear Quadratic Stochastic Two-Person Nonzero-Sum Differential Games: Open-Loop and Closed-Loop Nash Equilibria
Authors:
Jingrui Sun,
Jiongmin Yong
Abstract:
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential…
▽ More
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which is different from the outcome of the closed-loop Nash equilibria in general. However, it is found that for the case of zero-sum differential games, the Riccati equation system for the closed-loop representation of open-loop saddle points coincides with that for the closed-loop saddle points, which leads to the conclusion that the closed-loop representation of open-loop saddle points is the outcome of the corresponding closed-loop saddle point as long as both exist. In particular, for linear quadratic optimal control problem, the closed-loop representation of open-loop optimal controls coincides with the outcome of the corresponding closed-loop optimal strategy, provided both exist.
△ Less
Submitted 15 July, 2016;
originally announced July 2016.
-
Time-Inconsistent Recursive Stochastic Optimal Control Problems
Authors:
Qingmeng Wei,
Jiongmin Yong,
Zhiyong Yu
Abstract:
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a corresponding equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation is established to characterize the associated equilibrium value function. Moreover, a well-p…
▽ More
In this paper, we study a time-inconsistent stochastic optimal control problem with a recursive cost functional by a multi-person hierarchical differential game approach. An equilibrium strategy of this problem is constructed and a corresponding equilibrium Hamilton-Jacobi-Bellman (HJB, for short) equation is established to characterize the associated equilibrium value function. Moreover, a well-posedness result of the equilibrium HJB equation is established under certain conditions.
△ Less
Submitted 10 June, 2016;
originally announced June 2016.
-
Exact Controllability of Linear Stochastic Differential Equations and Related Problems
Authors:
Yanqing Wang,
Donghui Yang,
Jiongmin Yong,
Zhiyong Yu
Abstract:
A notion of $L^p$-exact controllability is introduced for linear controlled (forward) stochastic differential equations, for which several sufficient conditions are established. Further, it is proved that the $L^p$-exact controllability, the validity of an observability inequality for the adjoint equation, the solvability of an optimization problem, and the solvability of an $L^p$-type norm optima…
▽ More
A notion of $L^p$-exact controllability is introduced for linear controlled (forward) stochastic differential equations, for which several sufficient conditions are established. Further, it is proved that the $L^p$-exact controllability, the validity of an observability inequality for the adjoint equation, the solvability of an optimization problem, and the solvability of an $L^p$-type norm optimal control problem are all equivalent.
△ Less
Submitted 24 March, 2016;
originally announced March 2016.
-
Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Closed-Loop Solvability
Authors:
Xun Li,
Jingrui Sun,
Jiongmin Yong
Abstract:
An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In thi…
▽ More
An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled two (generalized) Riccati equations, together with some constraints on the adapted solution to a linear backward stochastic differential equation and a linear terminal value problem of an ordinary differential equation.
△ Less
Submitted 25 February, 2016;
originally announced February 2016.
-
Forward-Backward Evolution Equations and Applications
Authors:
Jiongmin Yong
Abstract:
Well-posedness is studied for a special system of two-point boundary value problem for evolution equations which is called a forward-backward evolution equation (FBEE, for short). Two approaches are introduced: A decoupling method with some brief discussions, and a method of continuation with some substantial discussions. For the latter, we have introduced Lyapunov operators for FBEEs, whose exist…
▽ More
Well-posedness is studied for a special system of two-point boundary value problem for evolution equations which is called a forward-backward evolution equation (FBEE, for short). Two approaches are introduced: A decoupling method with some brief discussions, and a method of continuation with some substantial discussions. For the latter, we have introduced Lyapunov operators for FBEEs, whose existence leads to some uniform a priori estimates for the mild solutions of FBEEs, which will be sufficient for the well-posedness. For some special cases, Lyapunov operators are constructed. Also, from some given Lyapunov operators, the corresponding solvable FBEEs are identified.
△ Less
Submitted 14 August, 2015;
originally announced August 2015.
-
Open-Loop and Closed-Loop Solvabilities for Stochastic Linear Quadratic Optimal Control Problems
Authors:
Jingrui Sun,
Xun Li,
Jiongmin Yong
Abstract:
This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic c…
▽ More
This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative the control weighting matrix-valued function R(s) can be. Finiteness of the LQ problem is characterized by the convergence of the solutions to a family of Riccati equations. Then, a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. Finally, an illustrative example is presented.
△ Less
Submitted 10 August, 2015;
originally announced August 2015.
-
Optimal Control Problems of Forward-Backward Stochastic Volterra Integral Equations
Authors:
Yufeng Shi,
Tianxiao Wang,
Jiongmin Yong
Abstract:
Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs in short) are formulated and studied. A general duality principle is established for linear backward stochastic integral equation and linear stochastic Fredholm-Volterra integral equation with mean-field. With the help of such a duality principle, together with some other new delicate and subtle skills, Po…
▽ More
Optimal control problems of forward-backward stochastic Volterra integral equations (FBSVIEs in short) are formulated and studied. A general duality principle is established for linear backward stochastic integral equation and linear stochastic Fredholm-Volterra integral equation with mean-field. With the help of such a duality principle, together with some other new delicate and subtle skills, Pontryagin type maximum principles are proved for two optimal control problems of FBSVIEs.
△ Less
Submitted 29 April, 2014;
originally announced April 2014.
-
Linear Quadratic Stochastic Two-Person Zero-Sum Differential Games in an Infinite Horizon
Authors:
Jingrui Sun,
Jiongmin Yong,
Shuguang Zhang
Abstract:
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work…
▽ More
This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.
△ Less
Submitted 28 April, 2014;
originally announced April 2014.
-
Regularity Analysis for an Abstract System of Coupled Hyperbolic and Parabolic Equations
Authors:
Jianghao Hao,
Zhuangyi Liu,
Jiongmin Yong
Abstract:
In this paper, we provide a complete regularity analysis for an abstract system of coupled hyperbolic and parabolic equations in a complex Hilbert space. We are able to decompose the unit square of the parameters into three parts where the semigroup associated with the system is analytic, of specific order Gevrey classes, and non-smoothing, respectively. Moreover, we will show that the orders of G…
▽ More
In this paper, we provide a complete regularity analysis for an abstract system of coupled hyperbolic and parabolic equations in a complex Hilbert space. We are able to decompose the unit square of the parameters into three parts where the semigroup associated with the system is analytic, of specific order Gevrey classes, and non-smoothing, respectively. Moreover, we will show that the orders of Gevrey class are sharp, under proper conditions.
△ Less
Submitted 24 April, 2014;
originally announced April 2014.
-
Linear Quadratic Stochastic Differential Games: Open-Loop and Closed-Loop Saddle Points
Authors:
Jingrui Sun,
Jiongmin Yong
Abstract:
In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in…
▽ More
In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in terms of the solvability of a Riccati differential equation with certain regularity. It is possible that the closed-loop saddle point fails to exist, and at the same time, the corresponding Riccati equation admits a solution (which does not have needed regularity). Also, we will indicate that the solution of the Riccati equation may be non-unique.
△ Less
Submitted 19 January, 2014;
originally announced January 2014.
-
A Deterministic Affine-Quadratic Optimal Control Problem
Authors:
Yuanchang Wang,
Jiongmin Yong
Abstract:
A Deterministic affine quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the value function is differentiable and therefore satisfies the corresponding Hamilton-Jacobi-Bellman equation in the classical sense. Moreover, the so-called quasi-Riccati equation is de…
▽ More
A Deterministic affine quadratic optimal control problem is considered. Due to the nature of the problem, optimal controls exist under some very mild conditions. Further, it is shown that under some assumptions, the value function is differentiable and therefore satisfies the corresponding Hamilton-Jacobi-Bellman equation in the classical sense. Moreover, the so-called quasi-Riccati equation is derived and any optimal control admits a state feedback representation.
△ Less
Submitted 15 May, 2013;
originally announced May 2013.
