-
The least favorable noise
Authors:
Philip A. Ernst,
Abram M. Kagan,
L. C. G. Rogers
Abstract:
Suppose that a random variable $X$ of interest is observed perturbed by independent additive noise $Y$. This paper concerns the "the least favorable perturbation" $\hat Y_\ep$, which maximizes the prediction error $E(X-E(X|X+Y))^2$ in the class of $Y$ with $ \var (Y)\leq \ep$. We find a characterization of the answer to this question, and show by example that it can be surprisingly complicated. Ho…
▽ More
Suppose that a random variable $X$ of interest is observed perturbed by independent additive noise $Y$. This paper concerns the "the least favorable perturbation" $\hat Y_\ep$, which maximizes the prediction error $E(X-E(X|X+Y))^2$ in the class of $Y$ with $ \var (Y)\leq \ep$. We find a characterization of the answer to this question, and show by example that it can be surprisingly complicated. However, in the special case where $X$ is infinitely divisible, the solution is complete and simple. We also explore the conjecture that noisier $Y$ makes prediction worse.
△ Less
Submitted 17 March, 2021;
originally announced March 2021.
-
Statistical Meaning of Mean Functions
Authors:
Abram M. Kagan,
Paul J. Smith
Abstract:
The basic properties of the Fisher information allow to reveal the statistical meaning of classical inequalities between mean functions. The properties applied to scale mixtures of Gaussian distributions lead to a new mean function of purely statistical origin, unrelated to the classical arithmetic, geometric, and harmonic means. We call it the informational mean and show that when the arguments o…
▽ More
The basic properties of the Fisher information allow to reveal the statistical meaning of classical inequalities between mean functions. The properties applied to scale mixtures of Gaussian distributions lead to a new mean function of purely statistical origin, unrelated to the classical arithmetic, geometric, and harmonic means. We call it the informational mean and show that when the arguments of the mean functions are Hermitian positive definite matrices, not necessarily commuting, the informational mean lies between the arithmetic and harmonic means, playing, in a sense, the role of the geometric mean that cannot be correctly defined in case of non-commuting matrices.\\ Surprisingly the monotonicity and additivity properties of the Fisher information lead to a new generalization of the classical inequality between the arithmetic and harmonic means.
△ Less
Submitted 6 April, 2019;
originally announced April 2019.
-
Calibrating dependence between random elements
Authors:
Abram M. Kagan,
Gabor J. Székely
Abstract:
Attempts to quantify dependence between random elements X and Y via maximal correlation go back to Gebelein (1941) and Rényi (1959). After summarizing properties (including some new) of the Rényi measure of dependence, a calibrated scale of dependence is introduced. It is based on the ``complexity`` of approximating functions of X by functions of Y.
Attempts to quantify dependence between random elements X and Y via maximal correlation go back to Gebelein (1941) and Rényi (1959). After summarizing properties (including some new) of the Rényi measure of dependence, a calibrated scale of dependence is introduced. It is based on the ``complexity`` of approximating functions of X by functions of Y.
△ Less
Submitted 11 March, 2019;
originally announced March 2019.
-
Efficiency requires innovation
Authors:
Abram M. Kagan
Abstract:
In estimation a parameter $θ\in{\mathbb R}$ from a sample $(x_1,\ldots,x_n)$ from a population $P_θ$ a simple way of incorporating a new observation $x_{n+1}$ into an estimator $\tildeθ_{n} = \tildeθ_{n}(x_1,\ldots,x_n)$ is transforming $\tildeθ_n$ to what we call the {\it jackknife extension} $\tildeθ_{n+1}^{(e)} = \tildeθ_{n+1}^{(e)}(x_1,\ldots,x_n,x_{n+1})$, \[\tildeθ_{n+1}^{(e)} = \{\tildeθ_n…
▽ More
In estimation a parameter $θ\in{\mathbb R}$ from a sample $(x_1,\ldots,x_n)$ from a population $P_θ$ a simple way of incorporating a new observation $x_{n+1}$ into an estimator $\tildeθ_{n} = \tildeθ_{n}(x_1,\ldots,x_n)$ is transforming $\tildeθ_n$ to what we call the {\it jackknife extension} $\tildeθ_{n+1}^{(e)} = \tildeθ_{n+1}^{(e)}(x_1,\ldots,x_n,x_{n+1})$, \[\tildeθ_{n+1}^{(e)} = \{\tildeθ_n (x_1 ,\ldots,x_n)+ \tildeθ_n (x_{n+1},x_2 ,\ldots,x_n) + \ldots + \tildeθ_n (x_1 ,\ldots,x_{n-1},x_{n+1})\}/(n+1).\] Though $\tildeθ_{n+1}^{(e)}$ lacks an innovation the statistician could expect from a larger data set, it is still better than $\tildeθ_n$, \[{\rm var}(\tildeθ_{n+1}^{(e)})\leq\frac{n}{n+1} {\rm var}(\tildeθ_n).\] However, an estimator obtained by jackknife extension for all $n$ is asymptotically efficient only for samples from exponential families. For a general $P_θ$, asymptotically efficient estimators require innovation when a new observation is added to the data. Some examples illustrate the concept.
△ Less
Submitted 18 February, 2019;
originally announced February 2019.
-
The KLR-theorem revisited
Authors:
Abram M. Kagan
Abstract:
For independent random variables $X_1,\ldots, X_n;Y_1,\ldots, Y_n$ with all $X_i$ identically distributed and same for $Y_j$, we study the relation \[E\{a\bar X + b\bar Y|X_1 -\bar X +Y_1 -\bar Y,\ldots,X_n -\bar X +Y_n -\bar Y\}={\rm const}\] with $a, b$ some constants. It is proved that for $n\geq 3$ and $ab>0$ the relation holds iff $X_i$ and $Y_j$ are Gaussian.\\ A new characterization arises…
▽ More
For independent random variables $X_1,\ldots, X_n;Y_1,\ldots, Y_n$ with all $X_i$ identically distributed and same for $Y_j$, we study the relation \[E\{a\bar X + b\bar Y|X_1 -\bar X +Y_1 -\bar Y,\ldots,X_n -\bar X +Y_n -\bar Y\}={\rm const}\] with $a, b$ some constants. It is proved that for $n\geq 3$ and $ab>0$ the relation holds iff $X_i$ and $Y_j$ are Gaussian.\\ A new characterization arises in case of $a=1, b= -1$. In this case either $X_i$ or $Y_j$ or both have a Gaussian component. It is the first (at least known to the author) case when presence of a Gaussian component is a characteristic property.
△ Less
Submitted 18 February, 2019;
originally announced February 2019.
-
On the structure of UMVUEs
Authors:
Abram M. Kagan,
Yaakov Malinovsky
Abstract:
In all setups when the structure of UMVUEs is known, there exists a subalgebra $\cal U$ (MVE-algebra) of the basic $σ$-algebra such that all $\cal U$-measurable statistics with finite second moments are UMVUEs. It is shown that MVE-algebras are, in a sense, similar to the subalgebras generated by complete sufficient statistics. Examples are given when these subalgebras differ, in these cases a new…
▽ More
In all setups when the structure of UMVUEs is known, there exists a subalgebra $\cal U$ (MVE-algebra) of the basic $σ$-algebra such that all $\cal U$-measurable statistics with finite second moments are UMVUEs. It is shown that MVE-algebras are, in a sense, similar to the subalgebras generated by complete sufficient statistics. Examples are given when these subalgebras differ, in these cases a new statistical structure arises.
△ Less
Submitted 9 July, 2015;
originally announced July 2015.
-
Partially complete sufficient statistics are jointly complete
Authors:
Abram M. Kagan,
Yaakov Malinovsky,
Lutz Mattner
Abstract:
The theory of the basic statistical concept of (Lehmann-Scheffé-)completeness is perfected by providing the theorem indicated in the title and previously overlooked for several decades. Relations to earlier results are discussed and illustrating examples are presented.
Of the two proofs offered for the main result, the first is direct and short, following the prototypical example of Landers and…
▽ More
The theory of the basic statistical concept of (Lehmann-Scheffé-)completeness is perfected by providing the theorem indicated in the title and previously overlooked for several decades. Relations to earlier results are discussed and illustrating examples are presented.
Of the two proofs offered for the main result, the first is direct and short, following the prototypical example of Landers and Rogge (1976), and the second is very short and purely statistical, utilizing the basic theory of optimal unbiased estimation in the little known version completed by Schmetterer and Strasser (1974).
△ Less
Submitted 23 May, 2014; v1 submitted 13 July, 2013;
originally announced July 2013.
-
Contribution to the theory of Pitman estimators
Authors:
Abram M. Kagan,
Tinghui Yu,
Andrew Barron,
Mokshay Madiman
Abstract:
New inequalities are proved for the variance of the Pitman estimators (minimum variance equivariant estimators) of θconstructed from samples of fixed size from populations F(x-θ). The inequalities are closely related to the classical Stam inequality for the Fisher information, its analog in small samples, and a powerful variance drop inequality. The only condition required is finite variance of F;…
▽ More
New inequalities are proved for the variance of the Pitman estimators (minimum variance equivariant estimators) of θconstructed from samples of fixed size from populations F(x-θ). The inequalities are closely related to the classical Stam inequality for the Fisher information, its analog in small samples, and a powerful variance drop inequality. The only condition required is finite variance of F; even the absolute continuity of F is not assumed. As corollaries of the main inequalities for small samples, one obtains alternate proofs of known properties of the Fisher information, as well as interesting new observations like the fact that the variance of the Pitman estimator based on a sample of size n scaled by n monotonically decreases in n. Extensions of the results to the polynomial versions of the Pitman estimators and a multivariate location parameter are given. Also, the search for characterization of equality conditions for one of the inequalities leads to a Cauchy-type functional equation for independent random variables, and an interesting new behavior of its solutions is described.
△ Less
Submitted 13 February, 2013;
originally announced February 2013.
-
On the Nile Problem by Sir Ronald Fisher
Authors:
Abram M. Kagan,
Yaakov Malinovsky
Abstract:
The Nile problem by Ronald Fisher may be interpreted as the problem of making statistical inference for a special curved exponential family when the minimal sufficient statistic is incomplete. The problem itself and its versions for general curved exponential families pose a mathematical-statistical challenge: studying the subalgebras of ancillary statistics within the $σ$-algebra of the (incomple…
▽ More
The Nile problem by Ronald Fisher may be interpreted as the problem of making statistical inference for a special curved exponential family when the minimal sufficient statistic is incomplete. The problem itself and its versions for general curved exponential families pose a mathematical-statistical challenge: studying the subalgebras of ancillary statistics within the $σ$-algebra of the (incomplete) minimal sufficient statistics and closely related questions of the structure of UMVUEs.
In this paper a new method is developed that, in particular, proves that in the classical Nile problem no statistic subject to mild natural conditions is a UMVUE. The method almost solves an old problem of the existence of UMVUEs. The method is purely statistical (vs. analytical) and works for any family possessing an ancillary statistic. It complements an analytical method that uses only the first order ancillarity (and thus works when the existence of ancillary subalgebras is an open problem) and works for curved exponential families with polynomial constraints on the canonical parameters of which the Nile problem is a special case.
△ Less
Submitted 5 July, 2013; v1 submitted 4 February, 2013;
originally announced February 2013.
-
Monotonicity in the Sample Size of the Length of Classical Confidence Intervals
Authors:
Abram M. Kagan,
Yaakov Malinovsky
Abstract:
It is proved that the average length of standard confidence intervals for parameters of gamma and normal distributions monotonically decrease with the sample size. The proofs are based on fine properties of the classical gamma function.
It is proved that the average length of standard confidence intervals for parameters of gamma and normal distributions monotonically decrease with the sample size. The proofs are based on fine properties of the classical gamma function.
△ Less
Submitted 28 August, 2012; v1 submitted 28 February, 2012;
originally announced February 2012.
