Showing 1–11 of 11 results for author: Bulinski, A

Statistical estimation of the Kullback-Leibler divergence

Authors: Alexander Bulinski, Denis Dimitrov

Abstract: Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These estimates are constructed by means of two independent collections of i.i.d. observations and involve the specified k-nearest neighbor statistics. In particular, the es… ▽ More Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These estimates are constructed by means of two independent collections of i.i.d. observations and involve the specified k-nearest neighbor statistics. In particular, the established results are valid for estimates of the Kullback-Leibler divergence between any two Gaussian measures in R^d with nondegenerate covariance matrices. As a byproduct we obtain new statements concerning the Kozachenko-Leonenko estimators of the Shannon differential entropy. △ Less

Submitted 29 June, 2019; originally announced July 2019.

MSC Class: 60F25; 62G20; 62H12

Statistical Estimation of Conditional Shannon Entropy

Authors: Alexander Bulinski, Alexey Kozhevin

Abstract: The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the mixed-pair model (X,Y) is considered where X and Y take values in R^d and an arbitrary finite set, respectively. Such models include, for instance, the famous log… ▽ More The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the mixed-pair model (X,Y) is considered where X and Y take values in R^d and an arbitrary finite set, respectively. Such models include, for instance, the famous logistic regression. In contrast to the well-known Kozachenko -- Leonenko estimates of unconditional entropy the proposed estimates are constructed by means of the certain spacial order statistics (or k-nearest neighbor statistics where k=k_n depends on amount of observations n) and a random number of i.i.d. observations contained in the balls of specified random radii. The asymptotic unbiasedness and L^2-consistency of the new estimates are established under simple conditions. The obtained results can be applied to the feature selection problem which is important, e.g., for medical and biological investigations. △ Less

Submitted 23 April, 2018; originally announced April 2018.

MSC Class: 60F25; 62G20; 62H12

Statistical estimation of the Shannon entropy

Authors: Alexander Bulinski, Denis Dimitrov

Abstract: The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates are established. The conditions employed involve the analogues of the Hardy - Littlewood maximal function. It is shown that the results are valid in particular f… ▽ More The behavior of the Kozachenko - Leonenko estimates for the (differential) Shannon entropy is studied when the number of i.i.d. vector-valued observations tends to infinity. The asymptotic unbiasedness and L^2-consistency of the estimates are established. The conditions employed involve the analogues of the Hardy - Littlewood maximal function. It is shown that the results are valid in particular for the entropy estimation of any nondegenerate Gaussian vector. △ Less

Submitted 6 January, 2018; originally announced January 2018.

MSC Class: 60F25; 62G20; 62H12

Modification of the MDR-EFE method for stratified samples

Authors: Alexander Bulinski, Alexey Kozhevin

Abstract: The MDR-EFE method of performing identification of relevant factors within a given collection X_1,...,X_n is developed for stratified samples in the case of binary response variable Y. We establish a criterion of strong consistency of estimates (involving K-cross-validation procedure and penalty) for a specified prediction error function. The cost approach is proposed to compare experiments with r… ▽ More The MDR-EFE method of performing identification of relevant factors within a given collection X_1,...,X_n is developed for stratified samples in the case of binary response variable Y. We establish a criterion of strong consistency of estimates (involving K-cross-validation procedure and penalty) for a specified prediction error function. The cost approach is proposed to compare experiments with random and nonrandom number of observations. Analytic results and simulations demonstrate advantages of the method introduced for stratified samples over that employed for i.i.d. learning sample. △ Less

Submitted 21 June, 2016; originally announced June 2016.

MSC Class: 62G05; 62G20

Simulation and analytical approach to the identification of significant factors

Authors: Alexander V. Bulinski, Alexander S. Rakitko

Abstract: We develop our previous works concerning the identification of the collection of significant factors determining some, in general, non-binary random response variable. Such identification is important, e.g., in biological and medical studies. Our approach is to examine the quality of response variable prediction by functions in (certain part of) the factors. The prediction error estimation require… ▽ More We develop our previous works concerning the identification of the collection of significant factors determining some, in general, non-binary random response variable. Such identification is important, e.g., in biological and medical studies. Our approach is to examine the quality of response variable prediction by functions in (certain part of) the factors. The prediction error estimation requires some cross-validation procedure, certain prediction algorithm and estimation of the penalty function. Using simulated data we demonstrate the efficiency of our method. We prove a new central limit theorem for introduced regularized estimates under some natural conditions for arrays of exchangeable random variables. Keywords: nonbinary random response; identification of significant factors; regularized estimates of prediction error; exchangeable random variables; central limit theorem. △ Less

Submitted 4 June, 2014; originally announced June 2014.

Comments: 25 pages, 6 tables, 3 figures

MSC Class: 62G05; 62E20

Central limit theorem related to MDR-method

Authors: Alexander Bulinski

Abstract: In many medical and biological investigations, including genetics, it is typical to handle high dimensional data which can be viewed as a set of values of some factors and a binary response variable. For instance, the response variable can describe the state of a patient health and one often assumes that it depends only on some part of factors. An important problem is to determine collections of s… ▽ More In many medical and biological investigations, including genetics, it is typical to handle high dimensional data which can be viewed as a set of values of some factors and a binary response variable. For instance, the response variable can describe the state of a patient health and one often assumes that it depends only on some part of factors. An important problem is to determine collections of significant factors. In this regard we turn to the MDR-method introduced by M.Ritchie and coauthors. Our recent paper provided the necessary and sufficient conditions for strong consistency of estimates of the prediction error employing the K-fold cross-validation and an arbitrary penalty function. Here we introduce the regularized versions of the mentioned estimates and prove for them the multidimensional CLT. Statistical variants of the CLT involving self-normalization are discussed as well. Keywords and phrases: binary response variable, significant factors, penalty function, cross-validation, MDR-method, SLLN for arrays, strong consistency, regularized estimates, multidimensional CLT, self-normalization. △ Less

Submitted 28 January, 2013; originally announced January 2013.

MSC Class: 60F05; 60F15; 62P10

Statistical methods of SNP data analysis with applications

Authors: Alexander Bulinski, Oleg Butkovsky, Alexey Shashkin, Pavel Yaskov

Abstract: Various statistical methods important for genetic analysis are considered and developed. Namely, we concentrate on the multifactor dimensionality reduction, logic regression, random forests and stochastic gradient boosting. These methods and their new modifications, e.g., the MDR method with "independent rule", are used to study the risk of complex diseases such as cardiovascular ones. The roles o… ▽ More Various statistical methods important for genetic analysis are considered and developed. Namely, we concentrate on the multifactor dimensionality reduction, logic regression, random forests and stochastic gradient boosting. These methods and their new modifications, e.g., the MDR method with "independent rule", are used to study the risk of complex diseases such as cardiovascular ones. The roles of certain combinations of single nucleotide polymorphisms and external risk factors are examined. To perform the data analysis concerning the ischemic heart disease and myocardial infarction the supercomputer SKIF "Chebyshev" of the Lomonosov Moscow State University was employed. △ Less

Submitted 24 June, 2011; originally announced June 2011.

On the Newman Conjecture

Authors: Alexander Bulinski

Abstract: We consider a random field, defined on an integer-valued d-dimensional lattice, with covariance function satisfying a condition more general than summability. Such condition appeared in the well-known Newman's conjecture concerning the central limit theorem (CLT) for stationary associated random fields. As was demonstrated by Herrndorf and Shashkin, the conjecture fails already for d=1. In the pre… ▽ More We consider a random field, defined on an integer-valued d-dimensional lattice, with covariance function satisfying a condition more general than summability. Such condition appeared in the well-known Newman's conjecture concerning the central limit theorem (CLT) for stationary associated random fields. As was demonstrated by Herrndorf and Shashkin, the conjecture fails already for d=1. In the present paper, we show the validity of modified conjecture leaving intact the mentioned condition on covariance function. Thus we establish, for any positive integer d, a criterion of the CLT validity for the wider class of positively associated stationary fields. The uniform integrability for the squares of normalized partial sums, taken over growing parallelepipeds or cubes, plays the key role in deriving their asymptotic normality. So our result extends the Lewis theorem proved for sequences of random variables. A representation of variances of partial sums of a field using the slowly varying functions in several arguments is employed in essential way. △ Less

Submitted 21 April, 2011; originally announced April 2011.

Central limit theorems for the excursion set volumes of weakly dependent random fields

Authors: Alexander Bulinski, Evgeny Spodarev, Florian Timmermann

Abstract: The multivariate central limit theorems (CLT) for the volumes of excursion sets of stationary quasi-associated random fields on $\mathbb{R}^d$ are proved. Special attention is paid to Gaussian and shot noise fields. Formulae for the covariance matrix of the limiting distribution are provided. A statistical version of the CLT is considered as well. Some numerical results are also discussed. The multivariate central limit theorems (CLT) for the volumes of excursion sets of stationary quasi-associated random fields on $\mathbb{R}^d$ are proved. Special attention is paid to Gaussian and shot noise fields. Formulae for the covariance matrix of the limiting distribution are provided. A statistical version of the CLT is considered as well. Some numerical results are also discussed. △ Less

Submitted 1 March, 2012; v1 submitted 4 May, 2010; originally announced May 2010.

Comments: Published in at http://dx.doi.org/10.3150/10-BEJ339 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

Report number: IMS-BEJ-BEJ339

Journal ref: Bernoulli 2012, Vol. 18, No. 1, 100-118

Strong invariance principle for dependent random fields

Authors: Alexander Bulinski, Alexey Shashkin

Abstract: A strong invariance principle is established for random fields which satisfy dependence conditions more general than positive or negative association. We use the approach of Csörgő and Révész applied recently by Balan to associated random fields. The key step in our proof combines new moment and maximal inequalities, established by the authors for partial sums of multiindexed random variables, w… ▽ More A strong invariance principle is established for random fields which satisfy dependence conditions more general than positive or negative association. We use the approach of Csörgő and Révész applied recently by Balan to associated random fields. The key step in our proof combines new moment and maximal inequalities, established by the authors for partial sums of multiindexed random variables, with the estimate of the convergence rate in the CLT for random fields under consideration. △ Less

Submitted 10 August, 2006; originally announced August 2006.

Comments: Published at http://dx.doi.org/10.1214/074921706000000167 in the IMS Lecture Notes--Monograph Series (http://www.imstat.org/publications/lecnotes.htm) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-LNMS48-LNMS4813 MSC Class: 60F15; 60F17 (Primary)

Journal ref: IMS Lecture Notes--Monograph Series 2006, Vol. 48, 128-143

Generalization of the Critical Volume NTCP Model in the Radiobiology

Authors: Alexander Bulinski, Andrei Khrennikov

Abstract: A generalization of the well known critical volume NTCP model is proposed to take into account dependence of the functional subunits of irradiated organ (or tissue). A new statistical version of the CLT is established to analyze the corresponding random fields. A generalization of the well known critical volume NTCP model is proposed to take into account dependence of the functional subunits of irradiated organ (or tissue). A new statistical version of the CLT is established to analyze the corresponding random fields. △ Less

Submitted 11 April, 2005; originally announced April 2005.

MSC Class: AMS: 60F05; 62E20; 62G15; 62P10