-
A new look on large deviations and concentration inequalities for the Ewens-Pitman model
Authors:
Bernard Bercu,
Stefano Favaro
Abstract:
The Ewens-Pitman model is a probability distribution for random partitions of the set $[n]=\{1,\ldots,n\}$, parameterized by $α\in[0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The goal of this paper is to provide an alternative and concise proof of the Feng-Hoppe large deviation principle for the number $K_{n}$ of partition sets in the Ewens-Pitman model wi…
▽ More
The Ewens-Pitman model is a probability distribution for random partitions of the set $[n]=\{1,\ldots,n\}$, parameterized by $α\in[0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The goal of this paper is to provide an alternative and concise proof of the Feng-Hoppe large deviation principle for the number $K_{n}$ of partition sets in the Ewens-Pitman model with $α\in(0,1)$ and $θ>-α$. Our approach leverages an integral representation of the moment-generating function of $K_{n}$ in terms of the (one-parameter) Mittag-Leffler function, along with a sharp asymptotic expansion of it. This approach significantly simplifies the original proof of Feng-Hoppe large deviation principle, as it avoids all the technical difficulties arising from a continuity argument with respect to rational and non-rational values of $α$. Beyond large deviations for $K_{n}$, our approach allows to establish a sharp concentration inequality for $K_n$ involving the rate function of the large deviation principle.
△ Less
Submitted 9 March, 2025;
originally announced March 2025.
-
On the multidimensional elephant random walk with stops
Authors:
Bernard Bercu
Abstract:
The goal of this paper is to investigate the asymptotic behavior of the multidimensional elephant random walk with stops (MERWS). In contrast with the standard elephant random walk, the elephant is allowed to stay on his own position. We prove that the Gram matrix associated with the MERWS, properly normalized, converges almost surely to the product of a deterministic matrix, related to the axes o…
▽ More
The goal of this paper is to investigate the asymptotic behavior of the multidimensional elephant random walk with stops (MERWS). In contrast with the standard elephant random walk, the elephant is allowed to stay on his own position. We prove that the Gram matrix associated with the MERWS, properly normalized, converges almost surely to the product of a deterministic matrix, related to the axes on which the MERWS moves uniformly, and a Mittag-Leffler distribution. It allows us to extend all the results previously established for the one-dimensional elephant random walk with stops. More precisely, in the diffusive and critical regimes, we prove the almost sure convergence of the MERWS. In the superdiffusive regime, we establish the almost sure convergence of the MERWS, properly normalized, to a nondegenerate random vector. We also study the self-normalized asymptotic normality of the MERWS.
△ Less
Submitted 24 January, 2025;
originally announced January 2025.
-
On the SAGA algorithm with decreasing step
Authors:
Luis Fredes,
Bernard Bercu,
Eméric Gbaguidi
Abstract:
Stochastic optimization naturally appear in many application areas, including machine learning. Our goal is to go further in the analysis of the Stochastic Average Gradient Accelerated (SAGA) algorithm. To achieve this, we introduce a new $λ$-SAGA algorithm which interpolates between the Stochastic Gradient Descent ($λ=0$) and the SAGA algorithm ($λ=1$). Firstly, we investigate the almost sure co…
▽ More
Stochastic optimization naturally appear in many application areas, including machine learning. Our goal is to go further in the analysis of the Stochastic Average Gradient Accelerated (SAGA) algorithm. To achieve this, we introduce a new $λ$-SAGA algorithm which interpolates between the Stochastic Gradient Descent ($λ=0$) and the SAGA algorithm ($λ=1$). Firstly, we investigate the almost sure convergence of this new algorithm with decreasing step which allows us to avoid the restrictive strong convexity and Lipschitz gradient hypotheses associated to the objective function. Secondly, we establish a central limit theorem for the $λ$-SAGA algorithm. Finally, we provide the non-asymptotic $\mathbb{L}^p$ rates of convergence.
△ Less
Submitted 2 October, 2024;
originally announced October 2024.
-
Sharp Large Deviations for the Number of Descents and the Major Index in a Random Permutation
Authors:
Bernard Bercu,
Michel Bonnefont,
Luis Fredes,
Adrien Richou
Abstract:
The aim of this paper is to improve the large deviation principle for the number of descents in a random permutation by establishing a sharp large deviation principle of any order. We shall also prove a sharp large deviation principle of any order for the major index in a random permutation.
The aim of this paper is to improve the large deviation principle for the number of descents in a random permutation by establishing a sharp large deviation principle of any order. We shall also prove a sharp large deviation principle of any order for the major index in a random permutation.
△ Less
Submitted 8 July, 2024;
originally announced July 2024.
-
Sharp analysis on the joint distribution of the number of descents and inverse descents in a random permutation
Authors:
Luis Fredes,
Bernard Bercu,
Michel Bonnefont,
Adrien Richou
Abstract:
Chatteerjee and Diaconis have recently shown the asymptotic normality for the joint distribution of the number of descents and inverse descents in a random permutation. A noteworthy point of their results is that the asymptotic variance of the normal distribution is diagonal, which means that the number of descents and inverse descents are asymptotically uncorrelated.The goal of this paper is to g…
▽ More
Chatteerjee and Diaconis have recently shown the asymptotic normality for the joint distribution of the number of descents and inverse descents in a random permutation. A noteworthy point of their results is that the asymptotic variance of the normal distribution is diagonal, which means that the number of descents and inverse descents are asymptotically uncorrelated.The goal of this paper is to go further in this analysis by proving a large deviation principlefor the joint distribution. We shall show that the rate function of the joint distributionis the sum of the rate functions of the marginal distributions, which also means that the number of descents and inverse descents are asymptotically independent at the large deviation level. However,we are going to prove that they are finely dependent at the sharp large deviation level.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
A martingale approach to Gaussian fluctuations and laws of iterated logarithm for Ewens-Pitman model
Authors:
Bernard Bercu,
Stefano Favaro
Abstract:
The Ewens-Pitman model refers to a distribution for random partitions of $[n]=\{1,\ldots,n\}$, which is indexed by a pair of parameters $α\in [0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The large $n$ asymptotic properties of the Ewens-Pitman model have been the subject of numerous studies, with the focus being on the number $K_{n}$ of partition sets and t…
▽ More
The Ewens-Pitman model refers to a distribution for random partitions of $[n]=\{1,\ldots,n\}$, which is indexed by a pair of parameters $α\in [0,1)$ and $θ>-α$, with $α=0$ corresponding to the Ewens model in population genetics. The large $n$ asymptotic properties of the Ewens-Pitman model have been the subject of numerous studies, with the focus being on the number $K_{n}$ of partition sets and the number $K_{r,n}$ of partition subsets of size $r$, for $r=1,\ldots,n$. While for $α=0$ asymptotic results have been obtained in terms of almost-sure convergence and Gaussian fluctuations, for $α\in(0,1)$ only almost-sure convergences are available, with the proof for $K_{r,n}$ being given only as a sketch. In this paper, we make use of martingales to develop a unified and comprehensive treatment of the large $n$ asymptotic behaviours of $K_{n}$ and $K_{r,n}$ for $α\in(0,1)$, providing alternative, and rigorous, proofs of the almost-sure convergences of $K_{n}$ and $K_{r,n}$, and covering the gap of Gaussian fluctuations. We also obtain new laws of the iterated logarithm for $K_{n}$ and $K_{r,n}$.
△ Less
Submitted 26 August, 2024; v1 submitted 11 April, 2024;
originally announced April 2024.
-
Monge-Kantorovich superquantiles and expected shortfalls with applications to multivariate risk measurements
Authors:
Bernard Bercu,
Jeremie Bigot,
Gauthier Thurin
Abstract:
We propose center-outward superquantile and expected shortfall functions, with applications to multivariate risk measurements, extending the standard notion of value at risk and conditional value at risk from the real line to $\mathbb{R}^d$. Our new concepts are built upon the recent definition of Monge-Kantorovich quantiles based on the theory of optimal transport, and they provide a natural way…
▽ More
We propose center-outward superquantile and expected shortfall functions, with applications to multivariate risk measurements, extending the standard notion of value at risk and conditional value at risk from the real line to $\mathbb{R}^d$. Our new concepts are built upon the recent definition of Monge-Kantorovich quantiles based on the theory of optimal transport, and they provide a natural way to characterize multivariate tail probabilities and central areas of point clouds. They preserve the univariate interpretation of a typical observation that lies beyond or ahead a quantile, but in a meaningful multivariate way. We show that they characterize random vectors and their convergence in distribution, which underlines their importance. Our new concepts are illustrated on both simulated and real datasets.
△ Less
Submitted 23 August, 2024; v1 submitted 4 July, 2023;
originally announced July 2023.
-
Stochastic optimal transport in Banach Spaces for regularized estimation of multivariate quantiles
Authors:
Bernard Bercu,
Jérémie Bigot,
Gauthier Thurin
Abstract:
We introduce a new stochastic algorithm for solving entropic optimal transport (EOT) between two absolutely continuous probability measures $μ$ and $ν$. Our work is motivated by the specific setting of Monge-Kantorovich quantiles where the source measure $μ$ is either the uniform distribution on the unit hypercube or the spherical uniform distribution. Using the knowledge of the source measure, we…
▽ More
We introduce a new stochastic algorithm for solving entropic optimal transport (EOT) between two absolutely continuous probability measures $μ$ and $ν$. Our work is motivated by the specific setting of Monge-Kantorovich quantiles where the source measure $μ$ is either the uniform distribution on the unit hypercube or the spherical uniform distribution. Using the knowledge of the source measure, we propose to parametrize a Kantorovich dual potential by its Fourier coefficients. In this way, each iteration of our stochastic algorithm reduces to two Fourier transforms that enables us to make use of the Fast Fourier Transform (FFT) in order to implement a fast numerical method to solve EOT. We study the almost sure convergence of our stochastic algorithm that takes its values in an infinite-dimensional Banach space. Then, using numerical experiments, we illustrate the performances of our approach on the computation of regularized Monge-Kantorovich quantiles. In particular, we investigate the potential benefits of entropic regularization for the smooth estimation of multivariate quantiles using data sampled from the target measure $ν$.
△ Less
Submitted 19 February, 2024; v1 submitted 2 February, 2023;
originally announced February 2023.
-
Sharp large deviations and concentration inequalities for the number of descents in a random permutation
Authors:
Bernard Bercu,
Michel Bonnefont,
Adrien Richou
Abstract:
The goal of this paper is to go further in the analysis of the behavior of the number of descents in a random permutation. Via two different approaches relying on a suitable martingale decomposition or on the Irwin-Hall distribution, we prove that the number of descents satisfies a sharp large deviation principle. A very precise concentration inequality involving the rate function in the large dev…
▽ More
The goal of this paper is to go further in the analysis of the behavior of the number of descents in a random permutation. Via two different approaches relying on a suitable martingale decomposition or on the Irwin-Hall distribution, we prove that the number of descents satisfies a sharp large deviation principle. A very precise concentration inequality involving the rate function in the large deviation principle is also provided.
△ Less
Submitted 6 October, 2023; v1 submitted 19 October, 2022;
originally announced October 2022.
-
On the elephant random walk with stops playing hide and seek with the Mittag-Leffler distribution
Authors:
Bernard Bercu
Abstract:
The aim of this paper is to investigate the asymptotic behavior of the so-called elephant random walk with stops (ERWS). In contrast with the standard elephant random walk, the elephant is allowed to be lazy by staying on his own position. We prove that the number of ones of the ERWS, properly normalized, converges almost surely to a Mittag-Leffler distribution. It allows us to carry out a sharp a…
▽ More
The aim of this paper is to investigate the asymptotic behavior of the so-called elephant random walk with stops (ERWS). In contrast with the standard elephant random walk, the elephant is allowed to be lazy by staying on his own position. We prove that the number of ones of the ERWS, properly normalized, converges almost surely to a Mittag-Leffler distribution. It allows us to carry out a sharp analysis on the asymptotic behavior of the ERWS. In the diffusive and critical regimes, we establish the almost sure convergence of the ERWS. We also show that it is necessary to self-normalized the position of the ERWS by the random number of ones in order to prove the asymptotic normality. In the superdiffusive regime, we establish the almost sure convergence of the ERWS, properly normalized, to a nondegenerate random variable. Moreover, we also show that the fluctuation of the ERWS around its limiting random variable is still Gaussian.
△ Less
Submitted 8 March, 2022;
originally announced March 2022.
-
How to estimate the memory of the Elephant Random Walk
Authors:
Bernard Bercu,
Lucile Laulin
Abstract:
We introduce an original way to estimate the memory parameter of the elephant random walk, a fascinating discrete time random walk on integers having a complete memory of its entire history. Our estimator is nothing more than a quasi-maximum likelihood estimator, based on a second order Taylor approximation of the log-likelihood function. We show the almost sure convergence of our estimate in the…
▽ More
We introduce an original way to estimate the memory parameter of the elephant random walk, a fascinating discrete time random walk on integers having a complete memory of its entire history. Our estimator is nothing more than a quasi-maximum likelihood estimator, based on a second order Taylor approximation of the log-likelihood function. We show the almost sure convergence of our estimate in the diffusive, critical and superdiffusive regimes. The local asymptotic normality of our statistical procedure is established in the diffusive regime, while the local asymptotic mixed normality is proven in the superdiffusive regime. Asymptotic and exact confidence intervals as well as statistical tests are also provided. All our analysis relies on asymptotic results for martingales and the quadratic variations associated.
△ Less
Submitted 20 December, 2021;
originally announced December 2021.
-
New insights on the minimal random walk
Authors:
Bernard Bercu,
Víctor Hugo Vázquez Guevara
Abstract:
The aim of this paper is to deepen the analysis of the asymptotic behavior of the so-called minimal random walk (MRW) using a new martingale approach. The MRW is a discrete-time random walk with infinite memory that has three regimes depending on the location of its two parameters. In the diffusive and critical regimes, we establish new results on the almost sure asymptotic behavior of the MRW, su…
▽ More
The aim of this paper is to deepen the analysis of the asymptotic behavior of the so-called minimal random walk (MRW) using a new martingale approach. The MRW is a discrete-time random walk with infinite memory that has three regimes depending on the location of its two parameters. In the diffusive and critical regimes, we establish new results on the almost sure asymptotic behavior of the MRW, such as the quadratic strong law and the law of the iterated logarithm. In the superdiffusive regime, we prove the almost sure convergence of the MRW, properly normalized, to a nondegenerate random variable. Moreover, we show that the fluctuation of the MRW around its limiting random variable is still Gaussian.
△ Less
Submitted 17 December, 2021;
originally announced December 2021.
-
Asymptotic analysis of random walks on ice and graphite
Authors:
Bernard Bercu,
Fabien Montégut
Abstract:
The purpose of this paper is to investigate the asymptotic behavior of random walks on three-dimensional crystal structures. We focus our attention on the 1h structure of the ice and the 2h structure of graphite. We establish the strong law of large numbers and the asymptotic normality for both random walks on ice and graphite. All our analysis relies on asymptotic results for multi-dimensional ma…
▽ More
The purpose of this paper is to investigate the asymptotic behavior of random walks on three-dimensional crystal structures. We focus our attention on the 1h structure of the ice and the 2h structure of graphite. We establish the strong law of large numbers and the asymptotic normality for both random walks on ice and graphite. All our analysis relies on asymptotic results for multi-dimensional martingales.
△ Less
Submitted 17 September, 2021;
originally announced September 2021.
-
A stochastic Gauss-Newton algorithm for regularized semi-discrete optimal transport
Authors:
Bernard Bercu,
Jérémie Bigot,
Sébastien Gadat,
Emilia Siviero
Abstract:
We introduce a new second order stochastic algorithm to estimate the entropically regularized optimal transport cost between two probability measures. The source measure can be arbitrary chosen, either absolutely continuous or discrete, while the target measure is assumed to be discrete. To solve the semi-dual formulation of such a regularized and semi-discrete optimal transportation problem, we p…
▽ More
We introduce a new second order stochastic algorithm to estimate the entropically regularized optimal transport cost between two probability measures. The source measure can be arbitrary chosen, either absolutely continuous or discrete, while the target measure is assumed to be discrete. To solve the semi-dual formulation of such a regularized and semi-discrete optimal transportation problem, we propose to consider a stochastic Gauss-Newton algorithm that uses a sequence of data sampled from the source measure. This algorithm is shown to be adaptive to the geometry of the underlying convex optimization problem with no important hyperparameter to be accurately tuned. We establish the almost sure convergence and the asymptotic normality of various estimators of interest that are constructed from this stochastic Gauss-Newton algorithm. We also analyze their non-asymptotic rates of convergence for the expected quadratic risk in the absence of strong convexity of the underlying objective function. The results of numerical experiments from simulated data are also reported to illustrate the finite sample properties of this Gauss-Newton algorithm for stochastic regularized optimal transport, and to show its advantages over the use of the stochastic gradient descent, stochastic Newton and ADAM algorithms.
△ Less
Submitted 2 March, 2022; v1 submitted 12 July, 2021;
originally announced July 2021.
-
Stochastic approximation algorithms for superquantiles estimation
Authors:
Bernard Bercu,
Manon Costa,
Sébastien Gadat
Abstract:
This paper is devoted to two different two-time-scale stochastic approximation algorithms for superquantile estimation. We shall investigate the asymptotic behavior of a Robbins-Monro estimator and its convexified version. Our main contribution is to establish the almost sure convergence, the quadratic strong law and the law of iterated logarithm for our estimates via a martingale approach. A join…
▽ More
This paper is devoted to two different two-time-scale stochastic approximation algorithms for superquantile estimation. We shall investigate the asymptotic behavior of a Robbins-Monro estimator and its convexified version. Our main contribution is to establish the almost sure convergence, the quadratic strong law and the law of iterated logarithm for our estimates via a martingale approach. A joint asymptotic normality is also provided. Our theoretical analysis is illustrated by numerical experiments on real datasets.
△ Less
Submitted 29 July, 2020;
originally announced July 2020.
-
On the center of mass of the elephant random walk
Authors:
Bernard Bercu,
Lucile Laulin
Abstract:
Our goal is to investigate the asymptotic behavior of the center of mass of the elephant random walk, which is a discrete-time random walk on integers with a complete memory of its whole history. In the diffusive and critical regimes, we establish the almost sure convergence, the law of iterated logarithm and the quadratric strong law for the center of mass of the elephant random walk. The asympto…
▽ More
Our goal is to investigate the asymptotic behavior of the center of mass of the elephant random walk, which is a discrete-time random walk on integers with a complete memory of its whole history. In the diffusive and critical regimes, we establish the almost sure convergence, the law of iterated logarithm and the quadratric strong law for the center of mass of the elephant random walk. The asymptotic normality of the center of mass, properly normalized, is also provided. Finally, we prove a strong limit theorem for the center of mass in the superdiffusive regime. All our analysis relies on asymptotic results for multi-dimensional martingales.
△ Less
Submitted 8 April, 2020; v1 submitted 12 November, 2019;
originally announced November 2019.
-
An efficient stochastic Newton algorithm for parameter estimation in logistic regressions
Authors:
Bernard Bercu,
Antoine Godichon-Baggioni,
Bruno Portier
Abstract:
Logistic regression is a well-known statistical model which is commonly used in the situation where the output is a binary random variable. It has a wide range of applications including machine learning, public health, social sciences, ecology and econometry. In order to estimate the unknown parameters of logistic regression with data streams arriving sequentially and at high speed, we focus our a…
▽ More
Logistic regression is a well-known statistical model which is commonly used in the situation where the output is a binary random variable. It has a wide range of applications including machine learning, public health, social sciences, ecology and econometry. In order to estimate the unknown parameters of logistic regression with data streams arriving sequentially and at high speed, we focus our attention on a recursive stochastic algorithm. More precisely, we investigate the asymptotic behavior of a new stochastic Newton algorithm. It enables to easily update the estimates when the data arrive sequentially and to have research steps in all directions. We establish the almost sure convergence of our stochastic Newton algorithm as well as its asymptotic normality. All our theoretical results are illustrated by numerical experiments.
△ Less
Submitted 16 April, 2019;
originally announced April 2019.
-
Hypergeometric identities arising from the elephant random walk
Authors:
Bernard Bercu,
Marie-Line Chabanol,
Jean-Jacques Ruch
Abstract:
A probabilistic approach is provided to establish new hypergeometric identities. It is based on the calculation of moments of the limiting distribution of the position of the elephant random walk in the superdiffusive regime.
A probabilistic approach is provided to establish new hypergeometric identities. It is based on the calculation of moments of the limiting distribution of the position of the elephant random walk in the superdiffusive regime.
△ Less
Submitted 28 February, 2019;
originally announced February 2019.
-
Asymptotic distribution and convergence rates of stochastic algorithms for entropic optimal transportation between probability measures
Authors:
Bernard Bercu,
Jérémie Bigot
Abstract:
This paper is devoted to the stochastic approximation of entropically regularized Wasserstein distances between two probability measures, also known as Sinkhorn divergences. The semi-dual formulation of such regularized optimal transportation problems can be rewritten as a non-strongly concave optimisation problem. It allows to implement a Robbins-Monro stochastic algorithm to estimate the Sinkhor…
▽ More
This paper is devoted to the stochastic approximation of entropically regularized Wasserstein distances between two probability measures, also known as Sinkhorn divergences. The semi-dual formulation of such regularized optimal transportation problems can be rewritten as a non-strongly concave optimisation problem. It allows to implement a Robbins-Monro stochastic algorithm to estimate the Sinkhorn divergence using a sequence of data sampled from one of the two distributions. Our main contribution is to establish the almost sure convergence and the asymptotic normality of a new recursive estimator of the Sinkhorn divergence between two probability measures in the discrete and semi-discrete settings. We also study the rate of convergence of the expected excess risk of this estimator in the absence of strong concavity of the objective function. Numerical experiments on synthetic and real datasets are also provided to illustrate the usefulness of our approach for data analysis.
△ Less
Submitted 9 December, 2024; v1 submitted 21 December, 2018;
originally announced December 2018.
-
On the almost sure central limit theorem for ARX processes in adaptive tracking
Authors:
Bernard Bercu,
Victor Vazquez
Abstract:
The goal of this paper is to highlight the almost sure central limit theorem for martingales to the control community and to show the usefulness of this result for the system identification of controllable ARX(p,q) process in adaptive tracking. We also provide strongly consistent estimators of the even moments of the driven noise of a controllable ARX(p,q) process as well as quadratic strong laws…
▽ More
The goal of this paper is to highlight the almost sure central limit theorem for martingales to the control community and to show the usefulness of this result for the system identification of controllable ARX(p,q) process in adaptive tracking. We also provide strongly consistent estimators of the even moments of the driven noise of a controllable ARX(p,q) process as well as quadratic strong laws for the average costs and estimation errors sequences. Our theoretical results are illustrated by numerical experiments.
△ Less
Submitted 21 November, 2018;
originally announced November 2018.
-
New insights on concentration inequalities for self-normalized martingales
Authors:
Bernard Bercu,
Taieb Touati
Abstract:
We propose new concentration inequalities for self-normalized martingales. The main idea is to introduce a suitable weighted sum of the predictable quadratic variation and the total quadratic variation of the martingale. It offers much more flexibility and allows us to improve previous concentration inequalities. Statistical applications on autoregressive process, internal diffusion-limited aggreg…
▽ More
We propose new concentration inequalities for self-normalized martingales. The main idea is to introduce a suitable weighted sum of the predictable quadratic variation and the total quadratic variation of the martingale. It offers much more flexibility and allows us to improve previous concentration inequalities. Statistical applications on autoregressive process, internal diffusion-limited aggregation process, and online statistical learning are also provided.
△ Less
Submitted 14 June, 2019; v1 submitted 24 October, 2018;
originally announced October 2018.
-
On the multi-dimensional elephant random walk
Authors:
Bernard Bercu,
Lucile Laulin
Abstract:
The purpose of this paper is to investigate the asymptotic behavior of the multi-dimensional elephant random walk (MERW). It is a non-Markovian random walk which has a complete memory of its entire history. A wide range of literature is available on the one-dimensional ERW. Surprisingly, no references are available on the MERW. The goal of this paper is to fill the gap by extending the results on…
▽ More
The purpose of this paper is to investigate the asymptotic behavior of the multi-dimensional elephant random walk (MERW). It is a non-Markovian random walk which has a complete memory of its entire history. A wide range of literature is available on the one-dimensional ERW. Surprisingly, no references are available on the MERW. The goal of this paper is to fill the gap by extending the results on the one-dimensional ERW to the MERW. In the diffusive and critical regimes, we establish the almost sure convergence, the law of iterated logarithm and the quadratic strong law for the MERW. The asymptotic normality of the MERW, properly normalized, is also provided. In the superdiffusive regime, we prove the almost sure convergence as well as the mean square convergence of the MERW. All our analysis relies on asymptotic results for multi-dimensional martingales.
△ Less
Submitted 21 September, 2017;
originally announced September 2017.
-
A martingale approach for the elephant random walk
Authors:
Bernard Bercu
Abstract:
The purpose of this paper is to establish, via a martingale approach, some refinements on the asymptotic behavior of the one-dimensional elephant random walk (ERW). The asymptotic behavior of the ERW mainly depends on a memory parameter $p$ which lies between zero and one. This behavior is totally different in the diffusive regime $0 \leq p <3/4$, the critical regime $p=3/4$, and the superdiffusiv…
▽ More
The purpose of this paper is to establish, via a martingale approach, some refinements on the asymptotic behavior of the one-dimensional elephant random walk (ERW). The asymptotic behavior of the ERW mainly depends on a memory parameter $p$ which lies between zero and one. This behavior is totally different in the diffusive regime $0 \leq p <3/4$, the critical regime $p=3/4$, and the superdiffusive regime $3/4<p \leq 1$. Notwithstanding of this trichotomy, we provide some new results on the almost sure convergence and the asymptotic normality of the ERW.
△ Less
Submitted 6 July, 2017;
originally announced July 2017.
-
Nonparametric estimation of the derivative of the regression function: application to sea shores water quality
Authors:
Bernard Bercu,
Sami Capderou,
Gilles Durrieu
Abstract:
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonpar…
▽ More
This paper is devoted to the nonparametric estimation of the derivative of the regression function in a nonparametric regression model. We implement a very efficient and easy to handle statistical procedure based on the derivative of the recursive Nadaraya-Watson estimator. We establish the almost sure convergence as well as the asymptotic normality for our estimates. We also illustrate our nonparametric estimation procedure on simulated and real life data associated with sea shores water quality and valvometry.
△ Less
Submitted 20 June, 2016;
originally announced June 2016.
-
Large deviations and concentration inequalities for the Ornstein-Uhlenbeck process without tears
Authors:
Bernard Bercu,
Adrien Richou
Abstract:
Our goal is to establish large deviations and concentration inequalities for the maximum likelihood estimator of the drift parameter of the Ornstein-Uhlenbeck process without tears. We propose a new strategy to establish large deviation results which allows us, via a suitable transformation, to circumvent the classical difficulty of non-steepness. Our approach holds in the stable case where the pr…
▽ More
Our goal is to establish large deviations and concentration inequalities for the maximum likelihood estimator of the drift parameter of the Ornstein-Uhlenbeck process without tears. We propose a new strategy to establish large deviation results which allows us, via a suitable transformation, to circumvent the classical difficulty of non-steepness. Our approach holds in the stable case where the process is positive recurrent as well as in the unstable and explosive cases where the process is respectively null recurrent and transient. Notwithstanding of this trichotomy, we also provide new concentration inequalities for the maximum likelihood estimator.
△ Less
Submitted 8 February, 2016; v1 submitted 5 February, 2016;
originally announced February 2016.
-
A Durbin-Watson serial correlation test for ARX processes via excited adaptive tracking
Authors:
Bernard Bercu,
Bruno Portier,
Victor Vazquez
Abstract:
We propose a new statistical test for the residual autocorrelation in ARX adaptive tracking. The introduction of a persistent excitation in the adaptive tracking control allows us to build a bilateral statistical test based on the well-known Durbin-Watson statistic. We establish the almost sure convergence and the asymptotic normality for the Durbin-Watson statistic leading to a powerful serial co…
▽ More
We propose a new statistical test for the residual autocorrelation in ARX adaptive tracking. The introduction of a persistent excitation in the adaptive tracking control allows us to build a bilateral statistical test based on the well-known Durbin-Watson statistic. We establish the almost sure convergence and the asymptotic normality for the Durbin-Watson statistic leading to a powerful serial correlation test. Numerical experiments illustrate the good performances of our statistical test procedure.
△ Less
Submitted 15 July, 2014;
originally announced July 2014.
-
Large deviations for the Ornstein-Uhlenbeck process with shift
Authors:
Bernard Bercu,
Adrien Richou
Abstract:
We investigate the large deviation properties of the maximum likelihood estimators for the Ornstein-Uhlenbeck process with shift. We estimate simultaneously the drift and shift parameters. On the one hand, we establish a large deviation principle for the maximum likelihood estimates of the drift and shift parameters. Surprisingly, we find that the drift estimator shares the same large deviation pr…
▽ More
We investigate the large deviation properties of the maximum likelihood estimators for the Ornstein-Uhlenbeck process with shift. We estimate simultaneously the drift and shift parameters. On the one hand, we establish a large deviation principle for the maximum likelihood estimates of the drift and shift parameters. Surprisingly, we find that the drift estimator shares the same large deviation principle as the one previously established for the Ornstein-Uhlenbeck process without shift. Sharp large deviation principles are also provided. On the other hand, we show that the maximum likelihood estimator of the shift parameter satisfies a large deviation principle with a very unusual implicit rate function.
△ Less
Submitted 4 September, 2014; v1 submitted 27 November, 2013;
originally announced November 2013.
-
On Ornstein-Uhlenbeck driven by Ornstein-Uhlenbeck processes
Authors:
Bernard Bercu,
Frederic Proia,
Nicolas Savy
Abstract:
We investigate the asymptotic behavior of the maximum likelihood estimators of the unknown parameters of positive recurrent Ornstein-Uhlenbeck processes driven by Ornstein-Uhlenbeck processes.
We investigate the asymptotic behavior of the maximum likelihood estimators of the unknown parameters of positive recurrent Ornstein-Uhlenbeck processes driven by Ornstein-Uhlenbeck processes.
△ Less
Submitted 12 December, 2012;
originally announced December 2012.
-
A Rademacher-Menchov approach for random coefficient bifurcating autoregressive processes
Authors:
Bernard Bercu,
Vassili Blandin
Abstract:
We investigate the asymptotic behavior of the least squares estimator of the unknown parameters of random coefficient bifurcating autoregressive processes. Under suitable assumptions on inherited and environmental effects, we establish the almost sure convergence of our estimates. In addition, we also prove a quadratic strong law and central limit theorems. Our approach mainly relies on asymptotic…
▽ More
We investigate the asymptotic behavior of the least squares estimator of the unknown parameters of random coefficient bifurcating autoregressive processes. Under suitable assumptions on inherited and environmental effects, we establish the almost sure convergence of our estimates. In addition, we also prove a quadratic strong law and central limit theorems. Our approach mainly relies on asymptotic results for vector-valued martingales together with the well-known Rademacher-Menchov theorem.
△ Less
Submitted 22 October, 2012;
originally announced October 2012.
-
On the asymptotic behavior of the Durbin-Watson statistic for ARX processes in adaptive tracking
Authors:
Bernard Bercu,
Bruno Portier,
Victor Vazquez
Abstract:
A wide literature is available on the asymptotic behavior of the Durbin-Watson statistic for autoregressive models. However, it is impossible to find results on the Durbin-Watson statistic for autoregressive models with adaptive control. Our purpose is to fill the gap by establishing the asymptotic behavior of the Durbin Watson statistic for ARX models in adaptive tracking. On the one hand, we sho…
▽ More
A wide literature is available on the asymptotic behavior of the Durbin-Watson statistic for autoregressive models. However, it is impossible to find results on the Durbin-Watson statistic for autoregressive models with adaptive control. Our purpose is to fill the gap by establishing the asymptotic behavior of the Durbin Watson statistic for ARX models in adaptive tracking. On the one hand, we show the almost sure convergence as well as the asymptotic normality of the least squares estimators of the unknown parameters of the ARX models. On the other hand, we establish the almost sure convergence of the Durbin-Watson statistic and its asymptotic normality. Finally, we propose a bilateral statistical test for residual autocorrelation in adaptive tracking.
△ Less
Submitted 20 April, 2012;
originally announced April 2012.
-
On the asymptotic behavior of the Nadaraya-Watson estimator associated with the recursive SIR method
Authors:
Bernard Bercu,
Thi Mong Ngoc Nguyen,
Jerome Saracco
Abstract:
We investigate the asymptotic behavior of the Nadaraya-Watson estimator for the estimation of the regression function in a semiparametric regression model. On the one hand, we make use of the recursive version of the sliced inverse regression method for the estimation of the unknown parameter of the model. On the other hand, we implement a recursive Nadaraya-Watson procedure for the estimation of…
▽ More
We investigate the asymptotic behavior of the Nadaraya-Watson estimator for the estimation of the regression function in a semiparametric regression model. On the one hand, we make use of the recursive version of the sliced inverse regression method for the estimation of the unknown parameter of the model. On the other hand, we implement a recursive Nadaraya-Watson procedure for the estimation of the regression function which takes into account the previous estimation of the parameter of the semiparametric regression model. We establish the almost sure convergence as well as the asymptotic normality for our Nadaraya-Watson estimator. We also illustrate our semiparametric estimation procedure on simulated data.
△ Less
Submitted 24 February, 2012;
originally announced February 2012.
-
Fluctuations of Interacting Markov Chain Monte Carlo Methods
Authors:
Bernard Bercu,
Pierre Del Moral,
Arnaud Doucet
Abstract:
We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the class of nonlinear Markov chains interacting with their empirical occupation measures. We develop an original theoretical analysis based on resolvent operators and…
▽ More
We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the class of nonlinear Markov chains interacting with their empirical occupation measures. We develop an original theoretical analysis based on resolvent operators and semigroup techniques to analyze the fluctuations of their occupation measures around their limiting values.
△ Less
Submitted 2 January, 2012;
originally announced January 2012.
-
Sharp large deviations for the non-stationary Ornstein-Uhlenbeck process
Authors:
Bernard Bercu,
Laure Coutin,
Nicolas Savy
Abstract:
For the Ornstein-Uhlenbeck process, the asymptotic behavior of the maximum likelihood estimator of the drift parameter is totally different in the stable, unstable, and explosive cases. Notwithstanding of this trichotomy, we investigate sharp large deviation principles for this estimator in the three situations. In the explosive case, we exhibit a very unusual rate function with a shaped flat vall…
▽ More
For the Ornstein-Uhlenbeck process, the asymptotic behavior of the maximum likelihood estimator of the drift parameter is totally different in the stable, unstable, and explosive cases. Notwithstanding of this trichotomy, we investigate sharp large deviation principles for this estimator in the three situations. In the explosive case, we exhibit a very unusual rate function with a shaped flat valley and an abrupt discontinuity point at its minimum.
△ Less
Submitted 25 November, 2011;
originally announced November 2011.
-
A sharp analysis on the asymptotic behavior of the Durbin-Watson statistic for the first-order autoregressive process
Authors:
Bernard Bercu,
Frederic Proia
Abstract:
The purpose of this paper is to provide a sharp analysis on the asymptotic behavior of the Durbin-Watson statistic. We focus our attention on the first-order autoregressive process where the driven noise is also given by a first-order autoregressive process. We establish the almost sure convergence and the asymptotic normality for both the least squares estimator of the unknown parameter of the au…
▽ More
The purpose of this paper is to provide a sharp analysis on the asymptotic behavior of the Durbin-Watson statistic. We focus our attention on the first-order autoregressive process where the driven noise is also given by a first-order autoregressive process. We establish the almost sure convergence and the asymptotic normality for both the least squares estimator of the unknown parameter of the autoregressive process as well as for the serial correlation estimator associated to the driven noise. In addition, the almost sure rates of convergence of our estimates are also provided. It allows us to establish the almost sure convergence and the asymptotic normality for the Durbin-Watson statistic. Finally, we propose a new bilateral statistical test for residual autocorrelation.
△ Less
Submitted 17 April, 2011;
originally announced April 2011.
-
A Robbins-Monro procedure for estimation in semiparametric regression models
Authors:
Bernard Bercu,
Philippe Fraysse
Abstract:
This paper is devoted to the parametric estimation of a shift together with the nonparametric estimation of a regression function in a semiparametric regression model. We implement a very efficient and easy to handle Robbins-Monro procedure. On the one hand, we propose a stochastic algorithm similar to that of Robbins-Monro in order to estimate the shift parameter. A preliminary evaluation of the…
▽ More
This paper is devoted to the parametric estimation of a shift together with the nonparametric estimation of a regression function in a semiparametric regression model. We implement a very efficient and easy to handle Robbins-Monro procedure. On the one hand, we propose a stochastic algorithm similar to that of Robbins-Monro in order to estimate the shift parameter. A preliminary evaluation of the regression function is not necessary to estimate the shift parameter. On the other hand, we make use of a recursive Nadaraya-Watson estimator for the estimation of the regression function. This kernel estimator takes into account the previous estimation of the shift parameter. We establish the almost sure convergence for both Robbins-Monro and Nadaraya--Watson estimators. The asymptotic normality of our estimates is also provided. Finally, we illustrate our semiparametric estimation procedure on simulated and real data.
△ Less
Submitted 4 June, 2012; v1 submitted 4 January, 2011;
originally announced January 2011.
-
Almost sure central limit theorems on the Wiener space
Authors:
Bernard Bercu,
Ivan Nourdin,
Murad Taqqu
Abstract:
In this paper, we study almost sure central limit theorems for sequences of functionals of general Gaussian fields. We apply our result to non-linear functions of stationary Gaussian sequences. We obtain almost sure central limit theorems for these non-linear functions when they converge in law to a normal distribution.
In this paper, we study almost sure central limit theorems for sequences of functionals of general Gaussian fields. We apply our result to non-linear functions of stationary Gaussian sequences. We obtain almost sure central limit theorems for these non-linear functions when they converge in law to a normal distribution.
△ Less
Submitted 12 December, 2009;
originally announced December 2009.
-
A multiple stochastic integral criterion for almost sure limit theorems
Authors:
Bernard Bercu,
Ivan Nourdin,
Murad S. Taqqu
Abstract:
In this paper, we study almost sure central limit theorems for multiple stochastic integrals and provide a criterion based on the kernel of these multiple integrals. We apply our result to normalized partial sums of Hermite polynomials of increments of fractional Brownian motion. We obtain almost sure central limit theorems for these normalized sums when they converge in law to a normal distribu…
▽ More
In this paper, we study almost sure central limit theorems for multiple stochastic integrals and provide a criterion based on the kernel of these multiple integrals. We apply our result to normalized partial sums of Hermite polynomials of increments of fractional Brownian motion. We obtain almost sure central limit theorems for these normalized sums when they converge in law to a normal distribution.
△ Less
Submitted 14 April, 2009;
originally announced April 2009.
-
On the usefulness of persistent excitation in ARX adaptive tracking
Authors:
Bernard Bercu,
Victor Vazquez
Abstract:
The usefulness of persistent excitation is well-known in the control community. Thanks to a persistently excited adaptive tracking control, we show that it is possible to avoid the strong controllability assumption recently proposed in the multidimensional ARX framework. We establish the almost sure convergence for both least squares and weighted least squares estimators of the unknown parameter…
▽ More
The usefulness of persistent excitation is well-known in the control community. Thanks to a persistently excited adaptive tracking control, we show that it is possible to avoid the strong controllability assumption recently proposed in the multidimensional ARX framework. We establish the almost sure convergence for both least squares and weighted least squares estimators of the unknown parameters. A central limit theorem and a law of iterated logarithm are also provided. All this asymptotical analysis is related to the Schur complement of a suitable limiting matrix.
△ Less
Submitted 14 March, 2009;
originally announced March 2009.
-
On the Almost Sure Central Limit Theorem for Vector Martingales: Convergence of Moments and Statistical Applications
Authors:
Bernard Bercu,
Peggy Cénac,
Guy Fayolle
Abstract:
We investigate the almost sure asymptotic properties of vector martingale transforms. Assuming some appropriate regularity conditions both on the increasing process and on the moments of the martingale, we prove that normalized moments of any even order converge in the almost sure cental limit theorem for martingales. A conjecture about almost sure upper bounds under wider hypotheses is formulat…
▽ More
We investigate the almost sure asymptotic properties of vector martingale transforms. Assuming some appropriate regularity conditions both on the increasing process and on the moments of the martingale, we prove that normalized moments of any even order converge in the almost sure cental limit theorem for martingales. A conjecture about almost sure upper bounds under wider hypotheses is formulated. The theoretical results are supported by examples borrowed from statistical applications, including linear autoregressive models and branching processes with immigration, for which new asymptotic properties are established on estimation and prediction errors.
△ Less
Submitted 18 December, 2008;
originally announced December 2008.
-
Sharp large deviations for the fractional Ornstein-Uhlenbeck process
Authors:
Bernard Bercu,
Laure Coutin,
Nicolas Savy
Abstract:
We investigate the sharp large deviation properties of the energy and the maximum likelihood estimator for the Ornstein-Uhlenbeck process driven by a fractional Brownian motion with Hurst index greater than one half.
We investigate the sharp large deviation properties of the energy and the maximum likelihood estimator for the Ornstein-Uhlenbeck process driven by a fractional Brownian motion with Hurst index greater than one half.
△ Less
Submitted 24 October, 2008;
originally announced October 2008.
-
Almost Sure Stabilization for Adaptive Controls of Regime-switching LQ Systems with A Hidden Markov Chain
Authors:
Bernard Bercu,
Francois Dufour,
G. George Yin
Abstract:
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were conside…
▽ More
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear growth in the continuous component, the resulting process is regular. Moreover, by appropriate choice of the Lyapunov functions, it is shown that the adaptive system is stabilizable almost surely. As a by-product, it is also established that the controlled process is positive recurrent.
△ Less
Submitted 9 July, 2008;
originally announced July 2008.
-
Asymptotic analysis for bifurcating autoregressive processes via a martingale approach
Authors:
Bernard Bercu,
Benoite de Saporta,
Anne Gegout-Petit
Abstract:
We study the asymptotic behavior of the least squares estimators of the unknown parameters of bifurcating autoregressive processes. Under very weak assumptions on the driven noise of the process, namely conditional pair-wise independence and suitable moment conditions, we establish the almost sure convergence of our estimators together with the quadratic strong law and the central limit theorem.…
▽ More
We study the asymptotic behavior of the least squares estimators of the unknown parameters of bifurcating autoregressive processes. Under very weak assumptions on the driven noise of the process, namely conditional pair-wise independence and suitable moment conditions, we establish the almost sure convergence of our estimators together with the quadratic strong law and the central limit theorem. All our analysis relies on non-standard asymptotic results for martingales.
△ Less
Submitted 29 June, 2009; v1 submitted 3 July, 2008;
originally announced July 2008.
-
A new concept of strong controllability via the Schur complement in adaptive tracking
Authors:
Bernard Bercu,
Victor Vazquez
Abstract:
We propose a new concept of strong controllability associated with the Schur complement of a suitable limiting matrix. This concept allows us to extend the previous results associated with multidimensional ARX models. On the one hand, we carry out a sharp analysis of the almost sure convergence for both least squares and weighted least squares algorithms. On the other hand, we also provide a cen…
▽ More
We propose a new concept of strong controllability associated with the Schur complement of a suitable limiting matrix. This concept allows us to extend the previous results associated with multidimensional ARX models. On the one hand, we carry out a sharp analysis of the almost sure convergence for both least squares and weighted least squares algorithms. On the other hand, we also provide a central limit theorem and a law of iterated logarithm for these two stochastic algorithms. Our asymptotic results are illustrated by numerical simulations.
△ Less
Submitted 18 January, 2008;
originally announced January 2008.
-
Spectrum of the product of Toeplitz matrices with application in probability
Authors:
Bernard Bercu,
Jean-Francois Bony,
Vincent Bruneau
Abstract:
We study the spectrum of the product of two Toeplitz operators. Assume that the symbols of these operators are continuous and real-valued and that one of them is non-negative. We prove that the spectrum of the product of finite section Toeplitz matrices converges to the spectrum of the product of the semi-infinite Toeplitz operators. We give an example showing that the supremum of this set is no…
▽ More
We study the spectrum of the product of two Toeplitz operators. Assume that the symbols of these operators are continuous and real-valued and that one of them is non-negative. We prove that the spectrum of the product of finite section Toeplitz matrices converges to the spectrum of the product of the semi-infinite Toeplitz operators. We give an example showing that the supremum of this set is not always the supremum of the product of the two symbols. Finally, we provide an application in probability which is the first motivation of this study. More precisely, we obtain a large deviation principle for Gaussian quadratic forms.
△ Less
Submitted 10 December, 2007;
originally announced December 2007.
-
Exponential inequalities for self-normalized martingales with applications
Authors:
Bernard Bercu,
Abderrahmen Touati
Abstract:
We propose several exponential inequalities for self-normalized martingales similar to those established by De la Peña. The keystone is the introduction of a new notion of random variable heavy on left or right. Applications associated with linear regressions, autoregressive and branching processes are also provided.
We propose several exponential inequalities for self-normalized martingales similar to those established by De la Peña. The keystone is the introduction of a new notion of random variable heavy on left or right. Applications associated with linear regressions, autoregressive and branching processes are also provided.
△ Less
Submitted 14 November, 2008; v1 submitted 25 July, 2007;
originally announced July 2007.
-
Asymptotic results for empirical measures of weighted sums of independent random variables
Authors:
Bernard Bercu,
Wlodzimierz Bryc
Abstract:
We prove that if a rectangular matrix with uniformly small entries and approximately orthogonal rows is applied to the independent standardized random variables with uniformly bounded third moments, then the empirical CDF of the resulting partial sums converges to the normal CDF with probability one. This implies almost sure convergence of empirical periodograms, almost sure convergence of spect…
▽ More
We prove that if a rectangular matrix with uniformly small entries and approximately orthogonal rows is applied to the independent standardized random variables with uniformly bounded third moments, then the empirical CDF of the resulting partial sums converges to the normal CDF with probability one. This implies almost sure convergence of empirical periodograms, almost sure convergence of spectra of circulant and reverse circulant matrices, and almost sure convergence of the CDF's generated from independent random variables by independent random orthogonal matrices.
For special trigonometric matrices, the speed of the almost sure convergence is described by the normal approximation and by the large deviation principle.
△ Less
Submitted 26 July, 2006;
originally announced July 2006.
