Showing 1–4 of 4 results for author: Gayen, A

Generalized Fisher-Darmois-Koopman-Pitman Theorem and Rao-Blackwell Type Estimators for Power-Law Distributions

Authors: Atin Gayen, M. Ashok Kumar

Abstract: This paper generalizes the notion of sufficiency for estimation problems beyond maximum likelihood. In particular, we consider estimation problems based on Jones et al. and Basu et al. likelihood functions that are popular among distance-based robust inference methods. We first characterize the probability distributions that always have a fixed number of sufficient statistics (independent of sampl… ▽ More This paper generalizes the notion of sufficiency for estimation problems beyond maximum likelihood. In particular, we consider estimation problems based on Jones et al. and Basu et al. likelihood functions that are popular among distance-based robust inference methods. We first characterize the probability distributions that always have a fixed number of sufficient statistics (independent of sample size) with respect to these likelihood functions. These distributions are power-law extensions of the usual exponential family and contain Student distributions as a special case. We then extend the notion of minimal sufficient statistics and compute it for these power-law families. Finally, we establish a Rao-Blackwell-type theorem for finding the best estimators for a power-law family. This helps us establish Cramér-Rao-type lower bounds for power-law families. △ Less

Submitted 28 February, 2024; v1 submitted 1 May, 2022; originally announced May 2022.

Comments: 33 pages, Added an interpretation of resampling using generalized sufficient statistics, Minor modifications made, Published in IEEE Transactions on Information Theory, December 2023

MSC Class: Primary 62F10; 94A17; Secondary 62F35; 94A15

Journal ref: IEEE Transactions on Information Theory 69 (12) 7565 - 7583 (2023)

Projection Theorems and Estimating Equations for Power-Law Models

Authors: Atin Gayen, M. Ashok Kumar

Abstract: We extend projection theorems concerning Hellinger and Jones et al. divergences to the continuous case. These projection theorems reduce certain estimation problems on generalized exponential models to linear problems. We introduce the notion of regularity for generalized exponential models and show that the projection theorems in this case are similar to the ones in discrete and canonical case. W… ▽ More We extend projection theorems concerning Hellinger and Jones et al. divergences to the continuous case. These projection theorems reduce certain estimation problems on generalized exponential models to linear problems. We introduce the notion of regularity for generalized exponential models and show that the projection theorems in this case are similar to the ones in discrete and canonical case. We also apply these ideas to solve certain estimation problems concerning Student and Cauchy distributions. △ Less

Submitted 26 January, 2021; v1 submitted 4 May, 2019; originally announced May 2019.

Comments: New simulation results added stating the applicability of the generalized estimators: (1) comparing the robust estimators for Student distributions with maximum likelihood estimators, (2) Hellinger estimators for Cauchy distributions using two kernel density estimates for the sample empirical measure

MSC Class: Primary 62F10; 94A17; Secondary 62F35; 94A15

Generalized Estimating Equation for the Student-t Distributions

Authors: Atin Gayen, M. Ashok Kumar

Abstract: In \cite{KumarS15J2}, it was shown that a generalized maximum likelihood estimation problem on a (canonical) $α$-power-law model ($\mathbb{M}^{(α)}$-family) can be solved by solving a system of linear equations. This was due to an orthogonality relationship between the $\mathbb{M}^{(α)}$-family and a linear family with respect to the relative $α$-entropy (or the $\mathscr{I}_α$-divergence). Relati… ▽ More In \cite{KumarS15J2}, it was shown that a generalized maximum likelihood estimation problem on a (canonical) $α$-power-law model ($\mathbb{M}^{(α)}$-family) can be solved by solving a system of linear equations. This was due to an orthogonality relationship between the $\mathbb{M}^{(α)}$-family and a linear family with respect to the relative $α$-entropy (or the $\mathscr{I}_α$-divergence). Relative $α$-entropy is a generalization of the usual relative entropy (or the Kullback-Leibler divergence). $\mathbb{M}^{(α)}$-family is a generalization of the usual exponential family. In this paper, we first generalize the $\mathbb{M}^{(α)}$-family including the multivariate, continuous case and show that the Student-t distributions fall in this family. We then extend the above stated result of \cite{KumarS15J2} to the general $\mathbb{M}^{(α)}$-family. Finally we apply this result to the Student-t distribution and find generalized estimators for its parameters. △ Less

Submitted 27 January, 2018; originally announced January 2018.

Comments: 6 pages, Submitted to ISIT 2018

Projection Theorems of Divergences and Likelihood Maximization Methods

Authors: Atin Gayen, M. Ashok Kumar

Abstract: Projection theorems of divergences enable us to find reverse projection of a divergence on a specific statistical model as a forward projection of the divergence on a different but rather "simpler" statistical model, which, in turn, results in solving a system of linear equations. Reverse projection of divergences are closely related to various estimation methods such as the maximum likelihood est… ▽ More Projection theorems of divergences enable us to find reverse projection of a divergence on a specific statistical model as a forward projection of the divergence on a different but rather "simpler" statistical model, which, in turn, results in solving a system of linear equations. Reverse projection of divergences are closely related to various estimation methods such as the maximum likelihood estimation or its variants in robust statistics. We consider projection theorems of three parametric families of divergences that are widely used in robust statistics, namely the Rényi divergences (or the Cressie-Reed power divergences), density power divergences, and the relative $α$-entropy (or the logarithmic density power divergences). We explore these projection theorems from the usual likelihood maximization approach and from the principle of sufficiency. In particular, we show the equivalence of solving the estimation problems by the projection theorems of the respective divergences and by directly solving the corresponding estimating equations. We also derive the projection theorem for the density power divergences. △ Less

Submitted 16 June, 2017; v1 submitted 28 May, 2017; originally announced May 2017.

Comments: 28 pages, added projection theorem for density power divergences, minor corrections made, submitted to Bernoulli