Showing 1–21 of 21 results for author: Moriarty, J

Optimally stopping multidimensional Brownian motion

Authors: John Moriarty

Abstract: We solve optimal stopping for multidimensional Brownian motion in a bounded domain, a question raised in Dynkin and Yushkevich (1967), where the one-dimensional case was presented. Taking a geometric approach, under regularity conditions we construct the optimal stopping free boundary in the multidimensional case. We characterise the value function as the pointwise infimum of potentials with recur… ▽ More We solve optimal stopping for multidimensional Brownian motion in a bounded domain, a question raised in Dynkin and Yushkevich (1967), where the one-dimensional case was presented. Taking a geometric approach, under regularity conditions we construct the optimal stopping free boundary in the multidimensional case. We characterise the value function as the pointwise infimum of potentials with recursive extensions dominating the gain function, and obtain its continuity. Explicit examples illustrate the result. △ Less

Submitted 14 May, 2025; originally announced May 2025.

Comments: 16 pages

MSC Class: 60G40; 60J25

The one-shot problem: Solution to an open question of finite-fuel singular control with discretionary stopping

Authors: John Moriarty, Neofytos Rodosthenous

Abstract: We introduce a novel 'one-shot' solution technique resolving an open problem (Karatzas et al., Finite-fuel singular control with discretionary stopping, Stochastics 71:1-2 (2000)). Unexpectedly given the convexity of the latter problem, its waiting region is not necessarily connected. Along a typical sample path, the state process may even spend positive time in both of its connected components. T… ▽ More We introduce a novel 'one-shot' solution technique resolving an open problem (Karatzas et al., Finite-fuel singular control with discretionary stopping, Stochastics 71:1-2 (2000)). Unexpectedly given the convexity of the latter problem, its waiting region is not necessarily connected. Along a typical sample path, the state process may even spend positive time in both of its connected components. The analysis reveals more generally that when fuel is limited, contrary to intuition, the solution without fuel is not necessarily indicative of the solution for small amounts of fuel. To resolve this, we recommend solving the `one-shot' problem, which is one of optimal stopping, prior to employing the usual `guess and verify' solution approach. △ Less

Submitted 6 November, 2024; originally announced November 2024.

Comments: 51 pages, 9 figures

MSC Class: 93E20; 60G40 (Primary) 60J65 (Secondary)

Optimal stopping with nonlinear expectation: geometric and algorithmic solutions

Authors: Tomasz Kosmala, John Moriarty

Abstract: We use the geometry of functions associated with martingales under nonlinear expectations to solve risk-sensitive Markovian optimal stopping problems. Generalising the linear case due to Dynkin and Yushkievich (1969), the value function is the majorant or pointwise infimum of those functions which dominate the gain function. An emphasis is placed on the geometry of the majorant and pathwise argume… ▽ More We use the geometry of functions associated with martingales under nonlinear expectations to solve risk-sensitive Markovian optimal stopping problems. Generalising the linear case due to Dynkin and Yushkievich (1969), the value function is the majorant or pointwise infimum of those functions which dominate the gain function. An emphasis is placed on the geometry of the majorant and pathwise arguments, rather than exploiting convexity, positive homogeneity or related analytical properties. An algorithm is provided to construct the value function at the computational cost of a two-dimensional search. △ Less

Submitted 4 December, 2023; v1 submitted 30 June, 2023; originally announced June 2023.

Comments: 22 pages, 3 figures

MSC Class: 60G40; 91B08; 91B06; 60J25

RangL: A Reinforcement Learning Competition Platform

Authors: Viktor Zobernig, Richard A. Saldanha, Jinke He, Erica van der Sar, Jasper van Doorn, Jia-Chen Hua, Lachlan R. Mason, Aleksander Czechowski, Drago Indjic, Tomasz Kosmala, Alessandro Zocca, Sandjai Bhulai, Jorge Montalvo Arvizu, Claude Klöckl, John Moriarty

Abstract: The RangL project hosted by The Alan Turing Institute aims to encourage the wider uptake of reinforcement learning by supporting competitions relating to real-world dynamic decision problems. This article describes the reusable code repository developed by the RangL team and deployed for the 2022 Pathways to Net Zero Challenge, supported by the UK Net Zero Technology Centre. The winning solutions… ▽ More The RangL project hosted by The Alan Turing Institute aims to encourage the wider uptake of reinforcement learning by supporting competitions relating to real-world dynamic decision problems. This article describes the reusable code repository developed by the RangL team and deployed for the 2022 Pathways to Net Zero Challenge, supported by the UK Net Zero Technology Centre. The winning solutions to this particular Challenge seek to optimize the UK's energy transition policy to net zero carbon emissions by 2050. The RangL repository includes an OpenAI Gym reinforcement learning environment and code that supports both submission to, and evaluation in, a remote instance of the open source EvalAI platform as well as all winning learning agent strategies. The repository is an illustrative example of RangL's capability to provide a reusable structure for future challenges. △ Less

Submitted 28 July, 2022; originally announced August 2022.

Comments: Documents in general and premierly the RangL competition plattform and in particular its 2022's competition "Pathways to Netzero" 10 pages, 2 figures, 1 table, Comments welcome!

Hopping between distant basins

Authors: Maldon Goodridge, John Moriarty, Jure Vogrinc, Alessandro Zocca

Abstract: We present the Basin Hopping with Skipping (BH-S) algorithm for stochastic optimisation, which replaces the perturbation step of basin hopping (BH) with a so-called skipping proposal from the rare-event sampling literature. Empirical results on benchmark optimisation surfaces demonstrate that BH-S can improve performance relative to BH by encouraging non-local exploration, that is, by hopping betw… ▽ More We present the Basin Hopping with Skipping (BH-S) algorithm for stochastic optimisation, which replaces the perturbation step of basin hopping (BH) with a so-called skipping proposal from the rare-event sampling literature. Empirical results on benchmark optimisation surfaces demonstrate that BH-S can improve performance relative to BH by encouraging non-local exploration, that is, by hopping between distant basins. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Comments: 20 pages; 8 figures

MSC Class: 90C26

Large Fluctuations in Locational Marginal Prices

Authors: Tommaso Nesti, John Moriarty, Alessandro Zocca, Bert Zwart

Abstract: This paper investigates large fluctuations of Locational Marginal Prices (LMPs) in wholesale energy markets caused by volatile renewable generation profiles. Specifically, we study events of the form $\mathbb{P} \Big ( \mathbf{LMP} \notin \prod_{i=1}^n [α_i^-, α_i^+] \Big),$ where $\mathbf{LMP}$ is the vector of LMPs at the $n$ power grid nodes, and $\boldsymbolα^-,\boldsymbolα^+\in\mathbb{R}^n$ a… ▽ More This paper investigates large fluctuations of Locational Marginal Prices (LMPs) in wholesale energy markets caused by volatile renewable generation profiles. Specifically, we study events of the form $\mathbb{P} \Big ( \mathbf{LMP} \notin \prod_{i=1}^n [α_i^-, α_i^+] \Big),$ where $\mathbf{LMP}$ is the vector of LMPs at the $n$ power grid nodes, and $\boldsymbolα^-,\boldsymbolα^+\in\mathbb{R}^n$ are vectors of price thresholds specifying undesirable price occurrences. By exploiting the structure of the supply-demand matching mechanism in power grids, we look at LMPs as deterministic piecewise affine, possibly discontinuous functions of the stochastic input process, modeling uncontrollable renewable generation. We utilize techniques from large deviations theory to identify the most likely ways for extreme price spikes to happen, and to rank the nodes of the power grid in terms of their likelihood of experiencing a price spike. Our results are derived in the case of Gaussian fluctuations and are validated numerically on the IEEE 14-bus test case. △ Less

Submitted 26 February, 2020; originally announced February 2020.

Comments: 16 pages, 10 figures

Markov risk mappings and risk-sensitive optimal prediction

Authors: Tomasz Kosmala, Randall Martyr, John Moriarty

Abstract: We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal prediction, where at each stage the recursion depends on the whole future. The property holds for standard measures of risk used in practice, and is formulated in sev… ▽ More We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal prediction, where at each stage the recursion depends on the whole future. The property holds for standard measures of risk used in practice, and is formulated in several equivalent versions including a representation via acceptance sets, a strong version, and a dual representation. △ Less

Submitted 2 September, 2022; v1 submitted 19 January, 2020; originally announced January 2020.

Comments: 21 pages. Improved introduction and a new result on the canonical form for a Markovian family of dynamic risk measures

MSC Class: 60G40; 91B08; 91B06; 90C40

Discrete-time risk-aware optimal switching with non-adapted costs

Authors: Randall Martyr, John Moriarty, Magnus Perninge

Abstract: We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower planning is provided. We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower planning is provided. △ Less

Submitted 12 September, 2021; v1 submitted 9 October, 2019; originally announced October 2019.

Comments: 29 pages including references. Keywords: infinite horizon, optimal switching, risk measures, reflected backward stochastic difference equations, hydropower planning. Fixed some typos; tidied up the presentation; added new numerical example on hydropower planning. To appear in Journal of/Advances in Applied Probability

MSC Class: 60G40; 91B08; 49N30

A Metropolis-class sampler for targets with non-convex support

Authors: John Moriarty, Jure Vogrinc, Alessandro Zocca

Abstract: We aim to improve upon the exploration of the general-purpose random walk Metropolis algorithm when the target has non-convex support $A \subset \mathbb{R}^d$, by reusing proposals in $A^c$ which would otherwise be rejected. The algorithm is Metropolis-class and under standard conditions the chain satisfies a strong law of large numbers and central limit theorem. Theoretical and numerical evidence… ▽ More We aim to improve upon the exploration of the general-purpose random walk Metropolis algorithm when the target has non-convex support $A \subset \mathbb{R}^d$, by reusing proposals in $A^c$ which would otherwise be rejected. The algorithm is Metropolis-class and under standard conditions the chain satisfies a strong law of large numbers and central limit theorem. Theoretical and numerical evidence of improved performance relative to random walk Metropolis are provided. Issues of implementation are discussed and numerical examples, including applications to global optimisation and rare event sampling, are presented. △ Less

Submitted 13 August, 2021; v1 submitted 23 May, 2019; originally announced May 2019.

Comments: 18 pages, 5 figures

MSC Class: 65C05; 62F12 (primary) 60F05; 60J05; 65C40; 90C26; 65K05 (secondary)

Optimal control of a commercial building's thermostatic load for off-peak demand response

Authors: Randall Martyr, John Moriarty, Christian Beck

Abstract: This paper studies the optimal control of a commercial building's thermostatic load during off-peak hours as an ancillary service to the transmission system operator of a power grid. It provides an algorithmic framework which commercial buildings can implement to cost-effectively increase their electricity demand at night while they are unoccupied, instead of using standard inflexible setpoint con… ▽ More This paper studies the optimal control of a commercial building's thermostatic load during off-peak hours as an ancillary service to the transmission system operator of a power grid. It provides an algorithmic framework which commercial buildings can implement to cost-effectively increase their electricity demand at night while they are unoccupied, instead of using standard inflexible setpoint control. Consequently, there is minimal or no impact on user comfort, while the building manager gains an additional income stream from providing the ancillary service, and can benefit further by pre-conditioning the building for later periods. The framework helps determine the amount of flexibility that should be offered for the service, and cost optimized profiles for electricity usage when delivering the service. Numerical results show that there can be an economic incentive to participate even if the payment rate for the ancillary service is less than the price of electricity. △ Less

Submitted 5 June, 2018; originally announced June 2018.

Comments: 20 pages, 12 figures

MSC Class: 49N90; 93C15; 49M37

A probabilistic verification theorem for the finite horizon two-player zero-sum optimal switching game in continuous time

Authors: Said Hamadène, Randall Martyr, John Moriarty

Abstract: In this paper we study continuous-time two-player zero-sum optimal switching games on a finite horizon. Using the theory of doubly reflected BSDEs with interconnected barriers, we show that this game has a value and an equilibrium in the players' switching controls. In this paper we study continuous-time two-player zero-sum optimal switching games on a finite horizon. Using the theory of doubly reflected BSDEs with interconnected barriers, we show that this game has a value and an equilibrium in the players' switching controls. △ Less

Submitted 1 June, 2018; originally announced June 2018.

Comments: 18 pages

MSC Class: 91A15; 91A55; 91A05; 93E20; 60G40; 49N25

Nonzero-sum games of optimal stopping and generalised Nash equilibrium

Authors: Randall Martyr, John Moriarty

Abstract: In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results. In the nonzero-sum setting, we establish a connection between Nash equilibria in games of optimal stopping (Dynkin games) and generalised Nash equilibrium problems (GNEP). In the Dynkin game this reveals novel equilibria of threshold type and of more complex types, and leads to novel uniqueness and stability results. △ Less

Submitted 6 September, 2017; originally announced September 2017.

Comments: 23 pages

MSC Class: 91A05; 91A15; 91B52; 60G40; 60J65

Energy imbalance market call options and the valuation of storage

Authors: John Moriarty, Jan Palczewski

Abstract: The use of energy storage to balance electric grids is increasing and, with it, the importance of operational optimisation from the twin viewpoints of cost and system stability. In this paper we assess the real option value of balancing reserve provided by an energy-limited storage unit. The contractual arrangement is a series of American-style call options in an energy imbalance market (EIM), phy… ▽ More The use of energy storage to balance electric grids is increasing and, with it, the importance of operational optimisation from the twin viewpoints of cost and system stability. In this paper we assess the real option value of balancing reserve provided by an energy-limited storage unit. The contractual arrangement is a series of American-style call options in an energy imbalance market (EIM), physically covered and delivered by the store, and purchased by the power system operator. We take the EIM price as a general regular one-dimensional diffusion and impose natural economic conditions on the option parameters. In this framework we derive the operational strategy of the storage operator by solving two timing problems: when to purchase energy to load the store (to provide physical cover for the option) and when to sell the option to the system operator. We give necessary and sufficient conditions for the finiteness and positivity of the value function -- the total discounted cash flows generated by operation of the storage unit. We also provide a straightforward procedure for the numerical evaluation of the optimal operational strategy (EIM prices at which power should be purchased) and the value function. This is illustrated with an operational and economic analysis using data from the German Amprion EIM. △ Less

Submitted 17 October, 2016; originally announced October 2016.

Comments: 30 pages, 3 figures

MSC Class: 93E20; 60J60; 91G80

Optimal Entry to an Irreversible Investment Plan with Non Convex Costs

Authors: Tiziano De Angelis, Giorgio Ferrari, Randall Martyr, John Moriarty

Abstract: A problem of optimally purchasing electricity at a real-valued spot price (that is, with potentially negative cost) has been recently addressed in De Angelis, Ferrari and Moriarty (2015) [SIAM J. Control Optim. 53(3)]. This problem can be considered one of irreversible investment with a cost functional which is non convex with respect to the control variable. In this paper we study the optimal ent… ▽ More A problem of optimally purchasing electricity at a real-valued spot price (that is, with potentially negative cost) has been recently addressed in De Angelis, Ferrari and Moriarty (2015) [SIAM J. Control Optim. 53(3)]. This problem can be considered one of irreversible investment with a cost functional which is non convex with respect to the control variable. In this paper we study the optimal entry into this investment plan. The optimal entry policy can have an irregular boundary arising from this non convexity, with a kinked shape. △ Less

Submitted 9 February, 2016; originally announced February 2016.

Comments: 30 pages, 3 figures

A geometric answer to an open question of singular control with stopping

Authors: John Moriarty

Abstract: We solve a problem of singular stochastic control with discretionary stopping, suggested as an interesting open problem by Karatzas, Ocone, Wang and Zervos (2000), by providing suitable candidates for the moving boundaries in an unsolved parameter range. We proceed by identifying an optimal stopping problem with similar variational inequalities and inspecting its parameter-dependent geometry (in a… ▽ More We solve a problem of singular stochastic control with discretionary stopping, suggested as an interesting open problem by Karatzas, Ocone, Wang and Zervos (2000), by providing suitable candidates for the moving boundaries in an unsolved parameter range. We proceed by identifying an optimal stopping problem with similar variational inequalities and inspecting its parameter-dependent geometry (in a sense going back to Dynkin (1965)), which reveals a discontinuity not previously exploited. We thus highlight the potential importance of this geometric information in both singular control and parameter-dependent optimal stopping. △ Less

Submitted 28 September, 2015; originally announced September 2015.

Comments: 25 pages, 3 figures

Nash equilibria of threshold type for two-player nonzero-sum games of stopping

Authors: Tiziano De Angelis, Giorgio Ferrari, John Moriarty

Abstract: This paper analyses two-player nonzero-sum games of optimal stopping on a class of linear regular diffusions with not non-singular boundary behaviour (in the sense of Itô and McKean (1974), p.\ 108). We provide sufficient conditions under which Nash equilibria are realised by each player stopping the diffusion at one of the two boundary points of an interval. The boundaries of this interval solve… ▽ More This paper analyses two-player nonzero-sum games of optimal stopping on a class of linear regular diffusions with not non-singular boundary behaviour (in the sense of Itô and McKean (1974), p.\ 108). We provide sufficient conditions under which Nash equilibria are realised by each player stopping the diffusion at one of the two boundary points of an interval. The boundaries of this interval solve a system of algebraic equations. We also provide conditions sufficient for the uniqueness of the equilibrium in this class. △ Less

Submitted 2 August, 2017; v1 submitted 17 August, 2015; originally announced August 2015.

Comments: 29 pages; small edits in the text, updated acknowledgement of funding

MSC Class: 91A05; 91A15; 60G40; 60J60; 35R35

A solvable two-dimensional singular stochastic control problem with non convex costs

Authors: Tiziano De Angelis, Giorgio Ferrari, John Moriarty

Abstract: In this paper we provide a complete theoretical analysis of a two-dimensional degenerate non convex singular stochastic control problem. The optimisation is motivated by a storage-consumption model in an electricity market, and features a stochastic real-valued spot price modelled by Brownian motion. We find analytical expressions for the value function, the optimal control and the boundaries of t… ▽ More In this paper we provide a complete theoretical analysis of a two-dimensional degenerate non convex singular stochastic control problem. The optimisation is motivated by a storage-consumption model in an electricity market, and features a stochastic real-valued spot price modelled by Brownian motion. We find analytical expressions for the value function, the optimal control and the boundaries of the action and inaction regions. The optimal policy is characterised in terms of two monotone and discontinuous repelling free boundaries, although part of one boundary is constant and the smooth fit condition holds there. △ Less

Submitted 28 October, 2015; v1 submitted 10 November, 2014; originally announced November 2014.

Comments: 25 pages, 3 figures. Improved exposition of the results. This is a revised version of "A solvable two-dimensional degenerate singular stochastic control problem with non convex costs"

MSC Class: 91B70; 93E20; 60G40; 49L20

A Non Convex Singular Stochastic Control Problem and its Related Optimal Stopping Boundaries

Authors: Tiziano De Angelis, Giorgio Ferrari, John Moriarty

Abstract: Equivalences are known between problems of singular stochastic control (SSC) with convex performance criteria and related questions of optimal stopping, see for example Karatzas and Shreve [SIAM J. Control Optim. 22 (1984)]. The aim of this paper is to investigate how far connections of this type generalise to a non convex problem of purchasing electricity. Where the classical equivalence breaks d… ▽ More Equivalences are known between problems of singular stochastic control (SSC) with convex performance criteria and related questions of optimal stopping, see for example Karatzas and Shreve [SIAM J. Control Optim. 22 (1984)]. The aim of this paper is to investigate how far connections of this type generalise to a non convex problem of purchasing electricity. Where the classical equivalence breaks down we provide alternative connections to optimal stopping problems. We consider a non convex infinite time horizon SSC problem whose state consists of an uncontrolled diffusion representing a real-valued commodity price, and a controlled increasing bounded process representing an inventory. We analyse the geometry of the action and inaction regions by characterising their (optimal) boundaries. Unlike the case of convex SSC problems we find that the optimal boundaries may be both reflecting and repelling and it is natural to interpret the problem as one of SSC with discretionary stopping. △ Less

Submitted 12 November, 2014; v1 submitted 10 May, 2014; originally announced May 2014.

Comments: 24 pages, 2 figures

MSC Class: 91B70; 93E20; 60G40; 49L20

Reflected Brownian motion in a wedge: sum-of-exponential stationary densities

Authors: A. B. Dieker, J. Moriarty

Abstract: We give necessary and sufficient conditions for the stationary density of semimartingale reflected Brownian motion in a wedge to be written as a finite sum of terms of exponential product form. Relying on geometric ideas reminiscent of the reflection principle, we give an explicit formula for the density in such cases. We give necessary and sufficient conditions for the stationary density of semimartingale reflected Brownian motion in a wedge to be written as a finite sum of terms of exponential product form. Relying on geometric ideas reminiscent of the reflection principle, we give an explicit formula for the density in such cases. △ Less

Submitted 22 December, 2008; v1 submitted 5 December, 2007; originally announced December 2007.

MSC Class: 60J65; 60J60; 60K25

Journal ref: Electronic Communications in Probability, 14, p. 1-16, 2009

Exit problems associated with affine reflection groups

Authors: Yan Doumerc, John Moriarty

Abstract: We obtain a formula for the distribution of the first exit time of Brownian motion from the alcove of an affine Weyl group. In most cases the formula is expressed compactly, in terms of Pfaffians. Expected exit times are derived in the type à case. The results extend to other Markov processes. We also give formulas for the real eigenfunctions of the Dirichlet and Neumann Laplacians on alcoves, o… ▽ More We obtain a formula for the distribution of the first exit time of Brownian motion from the alcove of an affine Weyl group. In most cases the formula is expressed compactly, in terms of Pfaffians. Expected exit times are derived in the type à case. The results extend to other Markov processes. We also give formulas for the real eigenfunctions of the Dirichlet and Neumann Laplacians on alcoves, observing that the `Hot Spots' conjecture of J. Rauch is true for alcoves. △ Less

Submitted 25 May, 2008; v1 submitted 13 July, 2007; originally announced July 2007.

Comments: 34 pages. Rewritten in some places to improve presentation. To appear in Probability Theory and Related Fields

MSC Class: 20F55; 60J65

Journal ref: Probability Theory and Related Fields 145 (3-4), pp 351-383 (2009)

On the free energy of a directed polymer in a Brownian environment

Authors: John Moriarty, Neil O'Connell

Abstract: We prove a formula conjectured in O'Connell and Yor (2001) for the free energy density of a directed polymer in a Brownian environment in 1+1 dimensions. We prove a formula conjectured in O'Connell and Yor (2001) for the free energy density of a directed polymer in a Brownian environment in 1+1 dimensions. △ Less

Submitted 13 June, 2006; originally announced June 2006.

Comments: 15 pages. To appear in Markov Processes and Related Filelds (J.T. Lewis special edition)

MSC Class: 60K37; 82D30; 60K25; 60J65

Journal ref: Markov Processes and Related Fields 13, 251-266 (2007)