-
Deep differentiable reinforcement learning and optimal trading
Authors:
Thibault Jaisson
Abstract:
In many reinforcement learning applications, the underlying environment reward and transition functions are explicitly known differentiable functions. This enables us to use recent research which applies machine learning tools to stochastic control to find optimal action functions. In this paper, we define differentiable reinforcement learning as a particular case of this research. We find that in…
▽ More
In many reinforcement learning applications, the underlying environment reward and transition functions are explicitly known differentiable functions. This enables us to use recent research which applies machine learning tools to stochastic control to find optimal action functions. In this paper, we define differentiable reinforcement learning as a particular case of this research. We find that incorporating deep learning in this framework leads to more accurate and stable solutions than those obtained from more generic actor critic algorithms. We apply this deep differentiable reinforcement learning (DDRL) algorithm to the problem of one asset optimal trading strategies in various environments where the market dynamics are known. Thanks to the stability of this method, we are able to efficiently find optimal strategies for complex multi-scale market models. We also extend these methods to simultaneously find optimal action functions for a wide range of environment parameters. This makes it applicable to real life financial signals and portfolio optimization where the expected return has multiple time scales. In the case of a slow and a fast alpha signal, we find that the optimal trading strategy consists in using the fast signal to time the trades associated to the slow signal.
△ Less
Submitted 7 April, 2022; v1 submitted 6 December, 2021;
originally announced December 2021.
-
Liquidity and Impact in Fair Markets
Authors:
Thibault Jaisson
Abstract:
We develop a theory which applies to any market dynamics that satisfy a fair market assumption on the nullity of the average profit of simple market making strategies. We show that for any such fair market, there exists a martingale fair price which corresponds to the average liquidation value (at the ask or the bid) of an infinitesimal quantity of stock. We show that this fair price is a natural…
▽ More
We develop a theory which applies to any market dynamics that satisfy a fair market assumption on the nullity of the average profit of simple market making strategies. We show that for any such fair market, there exists a martingale fair price which corresponds to the average liquidation value (at the ask or the bid) of an infinitesimal quantity of stock. We show that this fair price is a natural reference price to compute the ex post gain of limit orders. Using only the fair market assumption, we link the spread to the impact of market orders on the fair price. We use our definition of the fair price to build empirical tests of the relevance of this notion whose results are consistent with our theoretical predictions.
△ Less
Submitted 8 June, 2015;
originally announced June 2015.
-
Rough fractional diffusions as scaling limits of nearly unstable heavy tailed Hawkes processes
Authors:
Thibault Jaisson,
Mathieu Rosenbaum
Abstract:
We investigate the asymptotic behavior as time goes to infinity of Hawkes processes whose regression kernel has $L^1$ norm close to one and power law tail of the form $x^{-(1+α)}$, with $α\in(0,1)$. We in particular prove that when $α\in(1/2,1)$, after suitable rescaling, their law converges to that of a kind of integrated fractional Cox-Ingersoll-Ross process, with associated Hurst parameter…
▽ More
We investigate the asymptotic behavior as time goes to infinity of Hawkes processes whose regression kernel has $L^1$ norm close to one and power law tail of the form $x^{-(1+α)}$, with $α\in(0,1)$. We in particular prove that when $α\in(1/2,1)$, after suitable rescaling, their law converges to that of a kind of integrated fractional Cox-Ingersoll-Ross process, with associated Hurst parameter $H=α-1/2$. This result is in contrast to the case of a regression kernel with light tail, where a classical Brownian CIR process is obtained at the limit. Interestingly, it shows that persistence properties in the point process can lead to an irregular behavior of the limiting process. This theoretical result enables us to give an agent-based foundation to some recent findings about the rough nature of volatility in financial markets.
△ Less
Submitted 13 April, 2015;
originally announced April 2015.
-
Estimation of slowly decreasing Hawkes kernels: Application to high frequency order book modelling
Authors:
Emmanuel Bacry,
Thibault Jaisson,
Jean-Francois Muzy
Abstract:
We present a modified version of the non parametric Hawkes kernel estimation procedure studied in arXiv:1401.0903 that is adapted to slowly decreasing kernels. We show on numerical simulations involving a reasonable number of events that this method allows us to estimate faithfully a power-law decreasing kernel over at least 6 decades. We then propose a 8-dimensional Hawkes model for all events as…
▽ More
We present a modified version of the non parametric Hawkes kernel estimation procedure studied in arXiv:1401.0903 that is adapted to slowly decreasing kernels. We show on numerical simulations involving a reasonable number of events that this method allows us to estimate faithfully a power-law decreasing kernel over at least 6 decades. We then propose a 8-dimensional Hawkes model for all events associated with the first level of some asset order book. Applying our estimation procedure to this model, allows us to uncover the main properties of the coupled dynamics of trade, limit and cancel orders in relationship with the mid-price variations.
△ Less
Submitted 22 December, 2014;
originally announced December 2014.
-
Volatility is rough
Authors:
Jim Gatheral,
Thibault Jaisson,
Mathieu Rosenbaum
Abstract:
Estimating volatility from recent high frequency data, we revisit the question of the smoothness of the volatility process. Our main result is that log-volatility behaves essentially as a fractional Brownian motion with Hurst exponent H of order 0.1, at any reasonable time scale. This leads us to adopt the fractional stochastic volatility (FSV) model of Comte and Renault. We call our model Rough F…
▽ More
Estimating volatility from recent high frequency data, we revisit the question of the smoothness of the volatility process. Our main result is that log-volatility behaves essentially as a fractional Brownian motion with Hurst exponent H of order 0.1, at any reasonable time scale. This leads us to adopt the fractional stochastic volatility (FSV) model of Comte and Renault. We call our model Rough FSV (RFSV) to underline that, in contrast to FSV, H<1/2. We demonstrate that our RFSV model is remarkably consistent with financial time series data; one application is that it enables us to obtain improved forecasts of realized volatility. Furthermore, we find that although volatility is not long memory in the RFSV model, classical statistical procedures aiming at detecting volatility persistence tend to conclude the presence of long memory in data generated from it. This sheds light on why long memory of volatility has been widely accepted as a stylized fact. Finally, we provide a quantitative market microstructure-based foundation for our findings, relating the roughness of volatility to high frequency trading and order splitting.
△ Less
Submitted 13 October, 2014;
originally announced October 2014.
-
Market impact as anticipation of the order flow imbalance
Authors:
Thibault Jaisson
Abstract:
In this paper, we assume that the permanent market impact of metaorders is linear and that the price is a martingale. Those two hypotheses enable us to derive the evolution of the price from the dynamics of the flow of market orders. For example, if the market order flow is assumed to follow a nearly unstable Hawkes process, we retrieve the apparent long memory of the flow together with a power la…
▽ More
In this paper, we assume that the permanent market impact of metaorders is linear and that the price is a martingale. Those two hypotheses enable us to derive the evolution of the price from the dynamics of the flow of market orders. For example, if the market order flow is assumed to follow a nearly unstable Hawkes process, we retrieve the apparent long memory of the flow together with a power law impact function which is consistent with the celebrated square root law. We also link the long memory exponent of the sign of market orders with the impact function exponent. One of the originalities of our approach is that our results are derived without assuming that market participants are able to detect the beginning of metaorders.
△ Less
Submitted 6 February, 2014;
originally announced February 2014.
-
Limit theorems for nearly unstable Hawkes processes
Authors:
Thibault Jaisson,
Mathieu Rosenbaum
Abstract:
Because of their tractability and their natural interpretations in term of market quantities, Hawkes processes are nowadays widely used in high-frequency finance. However, in practice, the statistical estimation results seem to show that very often, only nearly unstable Hawkes processes are able to fit the data properly. By nearly unstable, we mean that the $L^1$ norm of their kernel is close to u…
▽ More
Because of their tractability and their natural interpretations in term of market quantities, Hawkes processes are nowadays widely used in high-frequency finance. However, in practice, the statistical estimation results seem to show that very often, only nearly unstable Hawkes processes are able to fit the data properly. By nearly unstable, we mean that the $L^1$ norm of their kernel is close to unity. We study in this work such processes for which the stability condition is almost violated. Our main result states that after suitable rescaling, they asymptotically behave like integrated Cox-Ingersoll-Ross models. Thus, modeling financial order flows as nearly unstable Hawkes processes may be a good way to reproduce both their high and low frequency stylized facts. We then extend this result to the Hawkes-based price model introduced by Bacry et al. [Quant. Finance 13 (2013) 65-77]. We show that under a similar criticality condition, this process converges to a Heston model. Again, we recover well-known stylized facts of prices, both at the microstructure level and at the macroscopic scale.
△ Less
Submitted 12 March, 2015; v1 submitted 8 October, 2013;
originally announced October 2013.
