-
Large Sample Properties of Entropy Balancing Estimators of Average Causal Effects
Authors:
David Källberg,
Ingeborg Waernbaum
Abstract:
Weighting methods are used in observational studies to adjust for covariate imbalances between treatment and control groups. Entropy balancing (EB) is an alternative to inverse probability weighting with an estimated propensity score. The EB weights are constructed to satisfy balance constraints and optimized towards stability. We describe large sample properties of EB estimators of the average ca…
▽ More
Weighting methods are used in observational studies to adjust for covariate imbalances between treatment and control groups. Entropy balancing (EB) is an alternative to inverse probability weighting with an estimated propensity score. The EB weights are constructed to satisfy balance constraints and optimized towards stability. We describe large sample properties of EB estimators of the average causal treatment effect, based on the Kullback-Leibler and quadratic Rényi relative entropies. Additionally, we propose estimators of their asymptotic variances. Even though the objective of EB is to reduce model dependence, the estimators are generally not consistent unless implicit parametric assumptions for the propensity score or conditional outcomes are met. The finite sample properties of the estimators are investigated through a simulation study. In an application with observational data from the Swedish Childhood Diabetes Register, we estimate the average effect of school achievements on hospitalization due to acute complications of type 1 diabetes mellitus.
△ Less
Submitted 22 April, 2022;
originally announced April 2022.
-
Estimation of quadratic density functionals under m-dependence
Authors:
David Källberg,
Oleg Seleznjev
Abstract:
In this paper, we study estimation of certain integral functionals of one or two densities with samples from stationary m-dependent sequences. We consider two types of U-statistic estimators for these functionals that are functions of the number of epsilon-close vector observations in the samples. We show that the estimators are consistent and obtain their rates of convergence under weak distribut…
▽ More
In this paper, we study estimation of certain integral functionals of one or two densities with samples from stationary m-dependent sequences. We consider two types of U-statistic estimators for these functionals that are functions of the number of epsilon-close vector observations in the samples. We show that the estimators are consistent and obtain their rates of convergence under weak distributional assumptions. In particular, we propose estimators based on incomplete U-statistics which have favorable consistency properties even when m-dependence is the only dependence condition that can be imposed on the stationary sequences. The results can be used for divergence and entropy estimation, and thus find many applications in statistics and applied sciences.
△ Less
Submitted 19 September, 2013;
originally announced September 2013.
-
Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence
Authors:
David Källberg,
Nikolaj Leonenko,
Oleg Seleznjev
Abstract:
The Rényi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number…
▽ More
The Rényi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of epsilon-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g., consistency, asymptotic normality, Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g., epsilon-keys in time series databases, distribution identification problems for dependent samples).
△ Less
Submitted 7 March, 2013;
originally announced March 2013.
-
Estimation of entropy-type integral functionals
Authors:
David Källberg,
Oleg Seleznjev
Abstract:
Entropy-type integral functionals of densities are widely used in mathematical statistics, information theory, and computer science. Examples include measures of closeness between distributions (e.g., density power divergence) and uncertainty characteristics for a random variable (e.g., Rényi entropy). In this paper, we study U-statistic estimators for a class of such functionals. The estimators a…
▽ More
Entropy-type integral functionals of densities are widely used in mathematical statistics, information theory, and computer science. Examples include measures of closeness between distributions (e.g., density power divergence) and uncertainty characteristics for a random variable (e.g., Rényi entropy). In this paper, we study U-statistic estimators for a class of such functionals. The estimators are based on epsilon-close vector observations in the corresponding independent and identically distributed samples. We prove asymptotic properties of the estimators (consistency and asymptotic normality) under mild integrability and smoothness conditions for the densities. The results can be applied in diverse problems in mathematical statistics and computer science (e.g., distribution identification problems, approximate matching for random databases, two-sample problems).
△ Less
Submitted 7 March, 2013; v1 submitted 12 September, 2012;
originally announced September 2012.
-
Statistical Inference for Rényi Entropy Functionals
Authors:
David Källberg,
Nikolaj Leonenko,
Oleg Seleznjev
Abstract:
Numerous entropy-type characteristics (functionals) generalizing Rényi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and distribution identification problems. We consider estimators of some entropy (integral) functionals for discrete and continuous distributions based on the number…
▽ More
Numerous entropy-type characteristics (functionals) generalizing Rényi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and distribution identification problems. We consider estimators of some entropy (integral) functionals for discrete and continuous distributions based on the number of epsilon-close vector records in the corresponding independent and identically distributed samples from two distributions. The estimators form a triangular scheme of generalized U-statistics. We show the asymptotic properties of these estimators (e.g., consistency and asymptotic normality). The results can be applied in various problems in computer science and mathematical statistics (e.g., approximate matching for random databases, record linkage, image matching).
△ Less
Submitted 25 March, 2011;
originally announced March 2011.
