Showing 1–15 of 15 results for author: Grendar, M

Multinomial and empirical likelihood under convex constraints: directions of recession, Fenchel duality, perturbations

Authors: Marian Grendár, Vladimír Špitalský

Abstract: The primal problem of multinomial likelihood maximization restricted to a convex closed subset of the probability simplex is studied. Contrary to widely held belief, a solution of this problem may assign a positive mass to an outcome with zero count. Related flaws in the simplified Lagrange and Fenchel dual problems, which arise because the recession directions are ignored, are identified and corr… ▽ More The primal problem of multinomial likelihood maximization restricted to a convex closed subset of the probability simplex is studied. Contrary to widely held belief, a solution of this problem may assign a positive mass to an outcome with zero count. Related flaws in the simplified Lagrange and Fenchel dual problems, which arise because the recession directions are ignored, are identified and corrected. A solution of the primal problem can be obtained by the PP (perturbed primal) algorithm, that is, as the limit of a sequence of solutions of perturbed primal problems. The PP algorithm may be implemented by the simplified Fenchel dual. The results permit us to specify linear sets and data such that the empirical likelihood-maximizing distribution exists and is the same as the multinomial likelihood-maximizing distribution. The multinomial likelihood ratio reaches, in general, a different conclusion than the empirical likelihood ratio. Implications for minimum discrimination information, compositional data analysis, Lindsay geometry, bootstrap with auxiliary information, and Lagrange multiplier tests are discussed. △ Less

Submitted 24 August, 2014; originally announced August 2014.

MSC Class: 62H12; 62H17 (Primary) 90C46 (Secondary)

Journal ref: Electron. J. Statist. 11 (2017), no. 1, 2547--2612

Strong laws for recurrence quantification analysis

Authors: Marian Grendár, Jana Majerová, Vladimír Špitalský

Abstract: The recurrence rate and determinism are two of the basic complexity measures studied in the recurrence quantification analysis. In this paper, the recurrence rate and determinism are expressed in terms of the correlation sums, and strong laws of large numbers are given for them. The recurrence rate and determinism are two of the basic complexity measures studied in the recurrence quantification analysis. In this paper, the recurrence rate and determinism are expressed in terms of the correlation sums, and strong laws of large numbers are given for them. △ Less

Submitted 7 February, 2013; originally announced February 2013.

MSC Class: 37A50; 60F15 (Primary) 37N99; 60G10 (Secondary)

Is the p-value a good measure of evidence? An asymptotic consistency criterion

Authors: M. Grendar

Abstract: What are the criteria that a measure of statistical evidence should satisfy? It is argued that a measure of evidence should be consistent. Consistency is an asymptotic criterion: the probability that if a measure of evidence in data strongly testifies against a hypothesis H, then H is indeed not true, should go to one, as more and more data appear. The p-value is not consistent, while the ratio of… ▽ More What are the criteria that a measure of statistical evidence should satisfy? It is argued that a measure of evidence should be consistent. Consistency is an asymptotic criterion: the probability that if a measure of evidence in data strongly testifies against a hypothesis H, then H is indeed not true, should go to one, as more and more data appear. The p-value is not consistent, while the ratio of likelihoods is. △ Less

Submitted 21 November, 2011; originally announced November 2011.

Comments: 4 pages

Asymptotic equivalence of empirical likelihood and Bayesian MAP

Authors: Marian Grendár, George Judge

Abstract: In this paper we are interested in empirical likelihood (EL) as a method of estimation, and we address the following two problems: (1) selecting among various empirical discrepancies in an EL framework and (2) demonstrating that EL has a well-defined probabilistic interpretation that would justify its use in a Bayesian context. Using the large deviations approach, a Bayesian law of large numbers… ▽ More In this paper we are interested in empirical likelihood (EL) as a method of estimation, and we address the following two problems: (1) selecting among various empirical discrepancies in an EL framework and (2) demonstrating that EL has a well-defined probabilistic interpretation that would justify its use in a Bayesian context. Using the large deviations approach, a Bayesian law of large numbers is developed that implies that EL and the Bayesian maximum a posteriori probability (MAP) estimators are consistent under misspecification and that EL can be viewed as an asymptotic form of MAP. Estimators based on other empirical discrepancies are, in general, inconsistent under misspecification. △ Less

Submitted 24 August, 2009; originally announced August 2009.

Comments: Published in at http://dx.doi.org/10.1214/08-AOS645 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS645 MSC Class: 62G05; 62C10 (Primary); 60F10 (Secondary)

Journal ref: Annals of Statistics 2009, Vol. 37, No. 5A, 2445-2457

Maximum Probability and Relative Entropy Maximization. Bayesian Maximum Probability and Empirical Likelihood

Authors: M. Grendar

Abstract: Works, briefly surveyed here, are concerned with two basic methods: Maximum Probability and Bayesian Maximum Probability; as well as with their asymptotic instances: Relative Entropy Maximization and Maximum Non-parametric Likelihood. Parametric and empirical extensions of the latter methods - Empirical Maximum Maximum Entropy and Empirical Likelihood - are also mentioned. The methods are viewed… ▽ More Works, briefly surveyed here, are concerned with two basic methods: Maximum Probability and Bayesian Maximum Probability; as well as with their asymptotic instances: Relative Entropy Maximization and Maximum Non-parametric Likelihood. Parametric and empirical extensions of the latter methods - Empirical Maximum Maximum Entropy and Empirical Likelihood - are also mentioned. The methods are viewed as tools for solving certain ill-posed inverse problems, called Pi-problem, Phi-problem, respectively. Within the two classes of problems, probabilistic justification and interpretation of the respective methods are discussed. △ Less

Submitted 24 April, 2008; originally announced April 2008.

Comments: Intnl. Workshop on Applied Probability 2008, Compiegne, France

MSC Class: 62F10; 62G10

L-Divergence Consistency for a Discrete Prior

Authors: M. Grendar

Abstract: Posterior distribution over a countable set M of continuous data-sampling distributions piles up at L-projection of the true distribution r on M, provided that the L-projection is unique. If there are several L-projections of r on M, then the posterior probability splits among them equally. Posterior distribution over a countable set M of continuous data-sampling distributions piles up at L-projection of the true distribution r on M, provided that the L-projection is unique. If there are several L-projections of r on M, then the posterior probability splits among them equally. △ Less

Submitted 10 October, 2007; v1 submitted 27 October, 2006; originally announced October 2006.

Comments: Typos corrected

MSC Class: 60F10; 60F15

Effective support size

Authors: M. Grendar

Abstract: Notion of effective size of support (Ess) of a random variable is introduced. A small set of natural requirements that a measure of Ess should satisfy is presented. The measure with prescribed properties is in a direct (exp-) relationship to the family of Renyi's $α$-entropies. Question of selecting the value of $α$ most appropriate for the purpose of measuring Ess is briefly addressed. Notion of effective size of support (Ess) of a random variable is introduced. A small set of natural requirements that a measure of Ess should satisfy is presented. The measure with prescribed properties is in a direct (exp-) relationship to the family of Renyi's $α$-entropies. Question of selecting the value of $α$ most appropriate for the purpose of measuring Ess is briefly addressed. △ Less

Submitted 5 May, 2006; originally announced May 2006.

MSC Class: 94A17

Conditioning by rare sources

Authors: M. Grendar

Abstract: In this paper we study the exponential decay of posterior probability of a set of sources and conditioning by rare sources for both uniform and general prior distributions of sources. The decay rate is determined by $L$-divergence and rare sources from a convex, closed set asymptotically conditionally concentrate on an $L$-projection. $L$-projection on a linear family of sources belongs to… ▽ More In this paper we study the exponential decay of posterior probability of a set of sources and conditioning by rare sources for both uniform and general prior distributions of sources. The decay rate is determined by $L$-divergence and rare sources from a convex, closed set asymptotically conditionally concentrate on an $L$-projection. $L$-projection on a linear family of sources belongs to $\varLambda$-family of distributions. The results parallel those of Large Deviations for Empirical Measures (Sanov's Theorem and Conditional Limit Theorem). △ Less

Submitted 12 January, 2007; v1 submitted 3 January, 2006; originally announced January 2006.

Comments: Identical with version 1. Replaced by an error

MSC Class: 60F10; 94A15

Conditional Equi-concentration of Types

Authors: Marian Grendar

Abstract: Conditional Equi-concentration of Types on I-projections is presented. It provides an extension of Conditional Weak Law of Large Numbers to the case of several I-projections. Also a multiple I-projections extension of Gibbs Conditioning Principle is developed. mu-projection variants of the probabilistic laws are stated. Implications of the results for Relative Entropy Maximization, Maximum Proba… ▽ More Conditional Equi-concentration of Types on I-projections is presented. It provides an extension of Conditional Weak Law of Large Numbers to the case of several I-projections. Also a multiple I-projections extension of Gibbs Conditioning Principle is developed. mu-projection variants of the probabilistic laws are stated. Implications of the results for Relative Entropy Maximization, Maximum Probability, Maximum Entropy in the Mean and Maximum Renyi-Tsallis Entropy methods are discussed. △ Less

Submitted 12 January, 2007; v1 submitted 7 July, 2004; originally announced July 2004.

Comments: Published in this form

MSC Class: 60F05; 60F10

Journal ref: Grendar, M., Conditional Equi-concentration of Types, in Focus on Probability Theory, L. R. Velle (ed.), NSP (NY), pp. 73-89, 2006

Chernoff's bound forms

Authors: M. Grendar, Jr., M. Grendar

Abstract: Chernoff's bound binds a tail probability (ie. $Pr(X \ge a)$, where $a \ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler distance) for $I$-projection $\hat P$ of $Q$ on a set $\mathcal{H} \triangleq \{P: E_PX = a\}$. Here, Chernoff's bound is related to Maximum Likelihood on exponent… ▽ More Chernoff's bound binds a tail probability (ie. $Pr(X \ge a)$, where $a \ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler distance) for $I$-projection $\hat P$ of $Q$ on a set $\mathcal{H} \triangleq \{P: E_PX = a\}$. Here, Chernoff's bound is related to Maximum Likelihood on exponential form and consequently implications for the notion of complementarity are discussed. Moreover, a novel form of the bound is proposed, which expresses the value of the Chernoff's bound directly in terms of the $I$-projection (or generalized $I$-projection). △ Less

Submitted 23 June, 2003; originally announced June 2003.

Comments: M. Grendar, Jr. and M. Grendar, ``Chernoff's bound forms,'' in Bayesian inference and Maximum Entropy methods in Science and Engineering, edited by Ch. Williams, AIP Conference Proceedings 659, Melville, New York, 2003, pp. 67-72

MSC Class: 60E15; 62B10

Journal ref: In: Bayesian inference and Maximum Entropy methods in Science and Engineering, Ch. Williams (ed.), AIP (Melville), 67-72, 2003

Randomness as an Equilibrium. Potential and Probability Density

Authors: M. Grendar, Jr., M. Grendar

Abstract: Randomness is viewed through an analogy between a physical quantity, density of gas, and a mathematical construct -- probability density. Boltzmann's deduction of equilibrium distribution of ideal gas placed in an external potential field than provides a way of viewing probability density from a perspective of forces/potentials, hidden behind it. Randomness is viewed through an analogy between a physical quantity, density of gas, and a mathematical construct -- probability density. Boltzmann's deduction of equilibrium distribution of ideal gas placed in an external potential field than provides a way of viewing probability density from a perspective of forces/potentials, hidden behind it. △ Less

Submitted 2 December, 2002; originally announced December 2002.

Comments: 4 pages, MaxEnt 2001 workshop

MSC Class: 60A99

Journal ref: In: Bayesian inference and Maximum Entropy methods in Science and Engineering, R. L. Fry (ed.), AIP (Melville), 405-410, 2002

Why Maximum Entropy? A Non-axiomatic Approach

Authors: M. Grendar, Jr., M. Grendar

Abstract: Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are frequently solved by Shannon's entropy maximization (MaxEnt). Several axiomatizations were proposed to justify the MaxEnt method (also) in this context. The main aim… ▽ More Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are frequently solved by Shannon's entropy maximization (MaxEnt). Several axiomatizations were proposed to justify the MaxEnt method (also) in this context. The main aim of the presented work is two-fold: 1) to view the concept of complementarity of MaxEnt and Maximum Likelihood (ML) tasks from a geometric perspective, and consequently 2) to provide an intuitive and non-axiomatic answer to the 'Why MaxEnt?' question. △ Less

Submitted 2 December, 2002; originally announced December 2002.

Comments: 4 pages, MaxEnt 2001

MSC Class: 62B10; 94A17

Journal ref: In: Bayesian inference and Maximum Entropy methods in Science and Engineering, R. L. Fry (ed.), AIP (Melville), 375-379, 2002

Minimax Entropy and Maximum Likelihood. Complementarity of tasks, identity of solutions

Authors: Marian Grendar, Jr, Marian Grendar

Abstract: Concept of exponential family is generalized by simple and general exponential form. Simple and general potential are introduced. Maximum Entropy and Maximum Likelihood tasks are defined. ML task on the simple exponential form and ME task on the simple potentials are proved to be complementary in set-up and identical in solutions. ML task on the general exponential form and ME task on the genera… ▽ More Concept of exponential family is generalized by simple and general exponential form. Simple and general potential are introduced. Maximum Entropy and Maximum Likelihood tasks are defined. ML task on the simple exponential form and ME task on the simple potentials are proved to be complementary in set-up and identical in solutions. ML task on the general exponential form and ME task on the general potentials are weakly complementary, leading to the same necessary conditions. A hypothesis about complementarity of ML and MiniMax Entropy tasks and identity of their solutions, brought up by a special case analytical as well as several numerical investigations, is suggested in this case. MiniMax Ent can be viewed as a generalization of MaxEnt for parametric linear inverse problems, and its complementarity with ML as yet another argument in favor of Shannon's entropy criterion. △ Less

Submitted 13 September, 2000; originally announced September 2000.

Comments: Presented at 'MaxEnt 2000', CNRS, Gif sur Yvette, France, July 8-13 2000

MSC Class: 62A10; 62A99

Journal ref: In: Bayesian inference and Maximum Entropy methods in Science and Engineering, A. Mohammad-Djafari (ed.), AIP (Melville), pp. 49-61, 2001

What is the question that MaxEnt answers? A probabilistic interpretation

Authors: Marian Grendar Jr, Marian Grendar

Abstract: The Boltzmann-Wallis-Jaynes' multiplicity argument is taken up and elaborated. MaxEnt is proved and demonstrated to be just an asymptotic case of looking for such a vector of absolute frequencies in a feasible set, which has maximal probability of being generated by a uniform prior generator/pmf. The Boltzmann-Wallis-Jaynes' multiplicity argument is taken up and elaborated. MaxEnt is proved and demonstrated to be just an asymptotic case of looking for such a vector of absolute frequencies in a feasible set, which has maximal probability of being generated by a uniform prior generator/pmf. △ Less

Submitted 13 September, 2000; originally announced September 2000.

Comments: Presented at 'MaxEnt 2000', CNRS, Gif-sur-Yvette, France, July 8-13, 2000

MSC Class: 60C05 (primary); 60F99 (secondary); 62A15 (secondary)

Journal ref: In: Bayesian inference and Maximum Entropy methods in Science and Engineering, A. Mohammad-Djafari (ed.), AIP (Melville), pp. 83-94, 2001

On the probabilistic rationale of I-divergence and J-divergence minimization

Authors: Marian Grendar Jr, Marian Grendar

Abstract: A probabilistic rationale for I-divergence minimization (relative entropy maximization), non-parametric likelihood maximization and J-divergence minimization (Jeffres' entropy maximization) criteria is provided. A probabilistic rationale for I-divergence minimization (relative entropy maximization), non-parametric likelihood maximization and J-divergence minimization (Jeffres' entropy maximization) criteria is provided. △ Less

Submitted 4 August, 2000; originally announced August 2000.

Report number: IMS SAS 2/2000 MSC Class: 60C05; 60F99