Showing 1–9 of 9 results for author: Broniatowski, M

A Unifying Framework for Some Directed Distances in Statistics

Authors: Michel Broniatowski, Wolfgang Stummer

Abstract: Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likeliho… ▽ More Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine learning. Prominent examples are the Kullback-Leibler information distance (relative entropy) which e.g. is closely connected to the omnipresent maximum likelihood estimation method, and Pearson's chisquare-distance which e.g. is used for the celebrated chisquare goodness-of-fit test. Another line of statistical inference is built upon distribution-function-based divergences such as e.g. the prominent (weighted versions of) Cramer-von Mises test statistics respectively Anderson-Darling test statistics which are frequently applied for goodness-of-fit investigations; some more recent methods deal with (other kinds of) cumulative paired divergences and closely related concepts. In this paper, we provide a general framework which covers in particular both the above-mentioned density-based and distribution-function-based divergence approaches; the dissimilarity of quantiles respectively of other statistical functionals will be included as well. From this framework, we structurally extract numerous classical and also state-of-the-art (including new) procedures. Furthermore, we deduce new concepts of dependence between random variables, as alternatives to the celebrated mutual information. Some variational representations are discussed, too. △ Less

Submitted 1 March, 2022; originally announced March 2022.

MSC Class: 62-02; 94A17; 62B10; 62FXX; 62GXX; 62NXX; 62B11; 49Q25

A sequential design for extreme quantiles estimation under binary sampling

Authors: Michel Broniatowski, Emilie Miranda

Abstract: We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of dichotomic data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists in estimating a failure quantile from trials whose outcomes are reduced to indicators of whether the specimen have failed at the tested stress… ▽ More We propose a sequential design method aiming at the estimation of an extreme quantile based on a sample of dichotomic data corresponding to peaks over a given threshold. This study is motivated by an industrial challenge in material reliability and consists in estimating a failure quantile from trials whose outcomes are reduced to indicators of whether the specimen have failed at the tested stress levels. The solution proposed is a sequential design making use of a splitting approach, decomposing the target probability level into a product of probabilities of conditional events of higher order. The method consists in gradually targeting the tail of the distribution and sampling under truncated distributions. The model is GEV or Weibull, and sequential estimation of its parameters involves an improved maximum likelihood procedure for binary data, due to the large uncertainty associated with such a restricted information. △ Less

Submitted 3 April, 2020; originally announced April 2020.

SAFIP: a streaming algorithm for inverse problems

Authors: Maeva Biret, Michel Broniatowski

Abstract: This paper presents a new algorithm which aims at the resolution of inverse problems of the form f(x) = 0, for x a vector of dimension d and f an arbitrary function with mild regularity condition. The set of solutions S may be infinite. This algorithm produces a good coverage of S, with a limited number of evaluations of the function f. It is therefore appropriate for complex problems where those… ▽ More This paper presents a new algorithm which aims at the resolution of inverse problems of the form f(x) = 0, for x a vector of dimension d and f an arbitrary function with mild regularity condition. The set of solutions S may be infinite. This algorithm produces a good coverage of S, with a limited number of evaluations of the function f. It is therefore appropriate for complex problems where those evaluations are costly. Various examples are presented, with d varying from 2 to 10. Proofs of convergence and of coverage of S are presented. △ Less

Submitted 27 September, 2016; originally announced September 2016.

Two Iterative Proximal-Point Algorithms for the Calculus of Divergence-based Estimators with Application to Mixture Models

Authors: Diaa Al Mohamad, Michel Broniatowski

Abstract: Estimators derived from an EM algorithm are not robust since they are based on the maximization of the likelihood function. We propose a proximal-point algorithm based on the EM algorithm which aim to minimize a divergence criterion. Resulting estimators are generally robust against outliers and misspecification. An EM-type proximal-point algorithm is also introduced in order to produce robust est… ▽ More Estimators derived from an EM algorithm are not robust since they are based on the maximization of the likelihood function. We propose a proximal-point algorithm based on the EM algorithm which aim to minimize a divergence criterion. Resulting estimators are generally robust against outliers and misspecification. An EM-type proximal-point algorithm is also introduced in order to produce robust estimators for mixture models. Convergence properties of the two algorithms are treated. We relax an identifiability condition imposed on the proximal term in the literature; a condition which is generally not fulfilled by mixture models. The convergence of the introduced algorithms is discussed on a two-component Weibull mixture and a two-component Gaussian mixture entailing a condition on the initialization of the EM algorithm in order for the later to converge. Simulations on mixture models using different statistical divergences are provided to confirm the validity of our work and the robustness of the resulting estimators against outliers in comparison to the EM algorithm. △ Less

Submitted 7 July, 2016; originally announced July 2016.

Comments: Article submitted to IEEE Transactions on Information Theory so that copy writes may be transfered if accepted. arXiv admin note: substantial text overlap with arXiv:1603.07117

A Proximal Point Algorithm for Minimum Divergence Estimators with Application to Mixture Models

Authors: Diaa Al Mohamad, Michel Broniatowski

Abstract: Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual representation of $\varphi$--divergences. We present in this paper an iterative proximal point algorithm which permits to calculate such estimator. This algorithm cont… ▽ More Estimators derived from a divergence criterion such as $\varphi-$divergences are generally more robust than the maximum likelihood ones. We are interested in particular in the so-called MD$\varphi$DE, an estimator built using a dual representation of $\varphi$--divergences. We present in this paper an iterative proximal point algorithm which permits to calculate such estimator. This algorithm contains by its construction the well-known EM algorithm. Our work is based on the paper of \citep{Tseng} on the likelihood function. We provide several convergence properties of the sequence generated by the algorithm, and improve the existing results by relaxing the identifiability condition on the proximal term, a condition which is not verified for most mixture models and hard to be verified for non mixture ones. Since convergence analysis uses regularity conditions (continuity and differentiability) of the objective function, which has a supremal form, we find it useful to present some analytical approaches for studying such functions. Convergence of the EM algorithm is discussed here again in a Gaussian and Weibull mixtures in the spirit of our approach. Simulations are provided to confirm the validity of our work and the robustness of the resulting estimators against outliers. △ Less

Submitted 12 June, 2016; v1 submitted 23 March, 2016; originally announced March 2016.

Comments: 19 pages. Article submitted to Journal Entropy, special issue : Diffierential Geometrical Theory of Statistics

Weighted sampling, Maximum Likelihood and minimum divergence estimators

Authors: Michel Broniatowski, Zhansheng Cao

Abstract: This paper explores Maximum Likelihood in parametric models in the context of Sanov type Large Deviation Probabilities. MLE in parametric models under weighted sampling is shown to be associated with the minimization of a specific divergence criterion defined with respect to the distribution of the weights. Some properties of the resulting inferential procedure are presented; Bahadur efficiency of… ▽ More This paper explores Maximum Likelihood in parametric models in the context of Sanov type Large Deviation Probabilities. MLE in parametric models under weighted sampling is shown to be associated with the minimization of a specific divergence criterion defined with respect to the distribution of the weights. Some properties of the resulting inferential procedure are presented; Bahadur efficiency of tests are also considered in this context. △ Less

Submitted 27 July, 2012; originally announced July 2012.

Conditional inference in parametric models

Authors: Michel Broniatowski, Virgile Caron

Abstract: This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density… ▽ More This paper presents a new approach to conditional inference, based on the simulation of samples conditioned by a statistics of the data. Also an explicit expression for the approximation of the conditional likelihood of long runs of the sample given the observed statistics is provided. It is shown that when the conditioning statistics is sufficient for a given parameter, the approximating density is still invariant with respect to the parameter. A new Rao-Blackwellisation procedure is proposed and simulation shows that Lehmann Scheffé Theorem is valid for this approximation. Conditional inference for exponential families with nuisance parameter is also studied, leading to Monte carlo tests. Finally the estimation of the parameter of interest through conditional likelihood is considered. Comparison with the parametric bootstrap method is discussed. △ Less

Submitted 5 February, 2012; originally announced February 2012.

Minimum divergence estimators, maximum likelihood and exponential families

Authors: Michel Broniatowski

Abstract: In this note we prove the dual representation formula of the divergence between two distributions in a parametric model. Resulting estimators for the divergence as for the parameter are derived. These estimators do not make use of any grouping nor smoothing. It is proved that all differentiable divergences induce the same estimator of the parameter on any regular exponential family, which is nothi… ▽ More In this note we prove the dual representation formula of the divergence between two distributions in a parametric model. Resulting estimators for the divergence as for the parameter are derived. These estimators do not make use of any grouping nor smoothing. It is proved that all differentiable divergences induce the same estimator of the parameter on any regular exponential family, which is nothing else but the MLE. △ Less

Submitted 20 August, 2011; v1 submitted 3 August, 2011; originally announced August 2011.

Comments: Submitted

Importance Sampling for rare events and conditioned random walks

Authors: Michel Broniatowski, Ya'Acov Ritov

Abstract: This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical mean of real i.i.d. summands. It is based on a sharp approximation of the density of long runs extracted from a random walk conditioned on its end value. This paper introduces a new Importance Sampling scheme, called Adaptive Twisted Importance Sampling, which is adequate for the improved estimation of rare event probabilities in he range of moderate deviations pertaining to the empirical mean of real i.i.d. summands. It is based on a sharp approximation of the density of long runs extracted from a random walk conditioned on its end value. △ Less

Submitted 9 October, 2009; originally announced October 2009.