-
Properties of the generalized inverse Gaussian with applications to Monte Carlo simulation and distribution function evaluation
Authors:
Victor Peña,
Michael Jauch
Abstract:
The generalized inverse Gaussian, denoted $\mathrm{GIG}(p, a, b)$, is a flexible family of distributions that includes the gamma, inverse gamma, and inverse Gaussian distributions as special cases. In addition to its applications in statistical modeling and its theoretical interest, the GIG often arises in computational statistics, especially in Markov chain Monte Carlo (MCMC) algorithms for poste…
▽ More
The generalized inverse Gaussian, denoted $\mathrm{GIG}(p, a, b)$, is a flexible family of distributions that includes the gamma, inverse gamma, and inverse Gaussian distributions as special cases. In addition to its applications in statistical modeling and its theoretical interest, the GIG often arises in computational statistics, especially in Markov chain Monte Carlo (MCMC) algorithms for posterior inference. This article introduces two mixture representations for the GIG: one that expresses the distribution as a continuous mixture of inverse Gaussians and another that reveals a recursive relationship between GIGs with different values of $p$. The former representation forms the basis for a data augmentation scheme that leads to a geometrically ergodic Gibbs sampler for the GIG. This simple Gibbs sampler, which alternates between gamma and inverse Gaussian conditional distributions, can be incorporated within an encompassing MCMC algorithm when simulation from a GIG is required. The latter representation leads to algorithms for exact, rejection-free sampling as well as CDF evaluation for the GIG with half-integer $p.$ We highlight computational examples from the literature where these new algorithms could be applied.
△ Less
Submitted 26 January, 2025; v1 submitted 1 January, 2024;
originally announced January 2024.
-
Prediction and estimation of random variables with infinite mean or variance
Authors:
Victor de la Pena,
Henryk Gzyl,
Silvia Mayoral,
Haolin Zou,
Demissie Alemayehu
Abstract:
In this paper we propose an optimal predictor of a random variable that has either an infinite mean or an infinite variance. The method consists of transforming the random variable such that the transformed variable has a finite mean and finite variance. The proposed predictor is a generalized arithmetic mean which is similar to the notion of certainty price in utility theory. Typically, the trans…
▽ More
In this paper we propose an optimal predictor of a random variable that has either an infinite mean or an infinite variance. The method consists of transforming the random variable such that the transformed variable has a finite mean and finite variance. The proposed predictor is a generalized arithmetic mean which is similar to the notion of certainty price in utility theory. Typically, the transformation consists of a parametric family of bijections, in which case the parameter might be chosen to minimize the prediction error in the transformed coordinates. The statistical properties of the estimator of the proposed predictor are studied, and confidence intervals are provided. The performance of the procedure is illustrated using simulated and real data.
△ Less
Submitted 26 March, 2023;
originally announced March 2023.
-
Mixture representations and Bayesian nonparametric inference for likelihood ratio ordered distributions
Authors:
Michael Jauch,
Andrés F. Barrientos,
Víctor Peña,
David S. Matteson
Abstract:
In this article, we introduce mixture representations for likelihood ratio ordered distributions. Essentially, the ratio of two probability densities, or mass functions, is monotone if and only if one can be expressed as a mixture of one-sided truncations of the other. To illustrate the practical value of the mixture representations, we address the problem of density estimation for likelihood rati…
▽ More
In this article, we introduce mixture representations for likelihood ratio ordered distributions. Essentially, the ratio of two probability densities, or mass functions, is monotone if and only if one can be expressed as a mixture of one-sided truncations of the other. To illustrate the practical value of the mixture representations, we address the problem of density estimation for likelihood ratio ordered distributions. In particular, we propose a nonparametric Bayesian solution which takes advantage of the mixture representations. The prior distribution is constructed from Dirichlet process mixtures and has large support on the space of pairs of densities satisfying the monotone ratio constraint. Posterior consistency holds under reasonable conditions on the prior specification and the true unknown densities. To our knowledge, this is the first posterior consistency result in the literature on order constrained inference. With a simple modification to the prior distribution, we can test the equality of two distributions against the alternative of likelihood ratio ordering. We develop a Markov chain Monte Carlo algorithm for posterior inference and demonstrate the method in a biomedical application.
△ Less
Submitted 26 October, 2023; v1 submitted 10 October, 2021;
originally announced October 2021.
-
A Dynamic Taylor's Law
Authors:
Victor De la Pena,
Paul Doukhan,
Yahia Salhi
Abstract:
Taylor's power law (or fluctuation scaling) states that on comparable populations, the variance of each sample is approximately proportional to a power of the mean of the population. It has been shown to hold by empirical observations in a broad class of disciplines including demography, biology, economics, physics and mathematics.
In particular, it has been observed in the problems involving po…
▽ More
Taylor's power law (or fluctuation scaling) states that on comparable populations, the variance of each sample is approximately proportional to a power of the mean of the population. It has been shown to hold by empirical observations in a broad class of disciplines including demography, biology, economics, physics and mathematics.
In particular, it has been observed in the problems involving population dynamics, market trading, thermodynamics and number theory.
For this many authors consider panel data in order to obtain laws of large numbers and the possibility to fit those expressions; essentially we aim at considering ergodic behaviors without independence. Thus we restrict the study to stationary time series and we develop different Taylor exponents in this setting.
From a theoretic point of view, there has been a growing interest on the study of the behavior of such a phenomenon. Most of these works focused on the so-called static Taylor related to independent samples. In this paper, we introduce a dynamic Taylor's law for dependent samples using self-normalised expressions involving Bernstein blocks. A central limit theorem (CLT) is proved under either weak dependence or strong mixing assumptions for the marginal process. The limit behavior of such a new index involves the series of covariances unlike the classic framework where the limit behavior involves the marginal variance. We also provide an asymptotic result for for a goodness-of-fit testing suited to check whether the corresponding dynamical Taylor's law holds in empirical studies. Moreover, we also obtain a consistent estimation of the Taylor's exponent.
△ Less
Submitted 20 October, 2020;
originally announced October 2020.
-
On the relationship between beta-Bartlett and Uhlig extended processes
Authors:
Víctor Peña,
Kaoru Irie
Abstract:
Stochastic volatility processes are used in multivariate time-series analysis to track time-varying patterns in covariance matrices. Uhlig extended and beta-Bartlett processes are especially convenient for analyzing high-dimensional time-series because they are conjugate with Wishart likelihoods. In this article, we show that Uhlig extended and beta-Bartlett are closely related, but not equivalent…
▽ More
Stochastic volatility processes are used in multivariate time-series analysis to track time-varying patterns in covariance matrices. Uhlig extended and beta-Bartlett processes are especially convenient for analyzing high-dimensional time-series because they are conjugate with Wishart likelihoods. In this article, we show that Uhlig extended and beta-Bartlett are closely related, but not equivalent: their hyperparameters can be matched so that they have the same forward-filtered posteriors and one-step ahead forecasts, but different joint (smoothed) posterior distributions. Under this circumstance, Bayes factors can't discriminate the models and alternative approaches to model comparison are needed. We illustrate these issues in a retrospective analysis of volatilities of returns of foreign exchange rates. Additionally, we provide a backward sampling algorithm for the beta-Bartlett process, for which retrospective analysis had not been developed.
△ Less
Submitted 4 May, 2021; v1 submitted 24 June, 2020;
originally announced June 2020.
-
Sharp Concentration Results for Heavy-Tailed Distributions
Authors:
Milad Bakhshizadeh,
Arian Maleki,
Victor H. de la Pena
Abstract:
We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $\mathbb{P}(X>t) \leq {\rm e}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow α\in [0, \infty)$ as…
▽ More
We obtain concentration and large deviation for the sums of independent and identically distributed random variables with heavy-tailed distributions. Our concentration results are concerned with random variables whose distributions satisfy $\mathbb{P}(X>t) \leq {\rm e}^{- I(t)}$, where $I: \mathbb{R} \rightarrow \mathbb{R}$ is an increasing function and $I(t)/t \rightarrow α\in [0, \infty)$ as $t \rightarrow \infty$. Our main theorem can not only recover some of the existing results, such as the concentration of the sum of subWeibull random variables, but it can also produce new results for the sum of random variables with heavier tails. We show that the concentration inequalities we obtain are sharp enough to offer large deviation results for the sums of independent random variables as well. Our analyses which are based on standard truncation arguments simplify, unify and generalize the existing results on the concentration and large deviation of heavy-tailed random variables.
△ Less
Submitted 25 July, 2022; v1 submitted 30 March, 2020;
originally announced March 2020.
-
A note on recent criticisms to Birnbaum's theorem
Authors:
Víctor Peña,
James O. Berger
Abstract:
In this note, we provide critical commentary on two articles that cast doubt on the validity and implications of Birnbaum's theorem: Evans (2013) and Mayo (2014). In our view, the proof is correct and the consequences of the theorem are alive and well.
In this note, we provide critical commentary on two articles that cast doubt on the validity and implications of Birnbaum's theorem: Evans (2013) and Mayo (2014). In our view, the proof is correct and the consequences of the theorem are alive and well.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
Restricted type II maximum likelihood priors on regression coefficients
Authors:
Víctor Peña,
James O. Berger
Abstract:
In Bayesian hypothesis testing and model selection, prior distributions must be chosen carefully. For example, setting arbitrarily large prior scales for location parameters, which is common practice in estimation problems, can lead to undesirable behavior in testing (Lindley's paradox). We study the properties of some restricted type II maximum likelihood (type II ML) priors on regression coeffic…
▽ More
In Bayesian hypothesis testing and model selection, prior distributions must be chosen carefully. For example, setting arbitrarily large prior scales for location parameters, which is common practice in estimation problems, can lead to undesirable behavior in testing (Lindley's paradox). We study the properties of some restricted type II maximum likelihood (type II ML) priors on regression coefficients. In type II ML, hyperparameters are "estimated" by maximizing the marginal likelihood of a model. In this article, we define priors by estimating their variances or covariance matrices, adding restrictions which ensure that the resulting priors are at least as vague as conventional proper priors for model uncertainty. We find that these type II ML priors typically yield results that are close to answers obtained with the Bayesian Information Criterion (BIC).
△ Less
Submitted 21 November, 2019; v1 submitted 21 November, 2017;
originally announced November 2017.
-
On the Ubiquity of Information Inconsistency for Conjugate Priors
Authors:
Joris Mulder,
James O. Berger,
Víctor Peña,
M. J. Bayarri
Abstract:
Informally, "Information Inconsistency" is the property that has been observed in many Bayesian hypothesis testing and model selection procedures whereby the Bayesian conclusion does not become definitive when the data seems to become definitive. An example is that, when performing a t-test using standard conjugate priors, the Bayes factor of the alternative hypothesis to the null hypothesis remai…
▽ More
Informally, "Information Inconsistency" is the property that has been observed in many Bayesian hypothesis testing and model selection procedures whereby the Bayesian conclusion does not become definitive when the data seems to become definitive. An example is that, when performing a t-test using standard conjugate priors, the Bayes factor of the alternative hypothesis to the null hypothesis remains bounded as the t statistic grows to infinity. This paper shows that information inconsistency is ubiquitous in Bayesian hypothesis testing under conjugate priors. Yet the title does not fully describe the paper, since we also show that theoretically recommended priors, including scale mixtures of conjugate priors and adaptive priors, are information consistent. Hence the paper is simply a forceful warning that use of conjugate priors in testing and model selection is highly problematical, and should be replaced by the information consistent alternatives.
△ Less
Submitted 26 October, 2017;
originally announced October 2017.
-
From Boundary Crossing of Non-Random Functions to Boundary Crossing of Stochastic Processes
Authors:
Mark Brown,
Victor de la Pena,
Tony Sit
Abstract:
One problem of wide interest involves estimating expected crossing-times. Several tools have been developed to solve this problem beginning with the works of Wald and the theory of sequential analysis. An extension of his approach is provided by the optional sampling theorem in conjunction with martingale inequalities. Deriving the explicit close form solution for the expected crossing times may b…
▽ More
One problem of wide interest involves estimating expected crossing-times. Several tools have been developed to solve this problem beginning with the works of Wald and the theory of sequential analysis. An extension of his approach is provided by the optional sampling theorem in conjunction with martingale inequalities. Deriving the explicit close form solution for the expected crossing times may be difficult. In this paper, we provide a framework that can be used to estimate expected crossing times of arbitrary stochastic processes. Our key assumption is the knowledge of the average behavior of the supremum of the process. Our results include a universal sharp lower bound on the expected crossing times.
△ Less
Submitted 11 December, 2012; v1 submitted 4 December, 2012;
originally announced December 2012.
-
Pseudo-maximization and self-normalized processes
Authors:
Victor H. de la Peña,
Michael J. Klass,
Tze Leung Lai
Abstract:
Self-normalized processes are basic to many probabilistic and statistical studies. They arise naturally in the the study of stochastic integrals, martingale inequalities and limit theorems, likelihood-based methods in hypothesis testing and parameter estimation, and Studentized pivots and bootstrap-$t$ methods for confidence intervals. In contrast to standard normalization, large values of the o…
▽ More
Self-normalized processes are basic to many probabilistic and statistical studies. They arise naturally in the the study of stochastic integrals, martingale inequalities and limit theorems, likelihood-based methods in hypothesis testing and parameter estimation, and Studentized pivots and bootstrap-$t$ methods for confidence intervals. In contrast to standard normalization, large values of the observations play a lesser role as they appear both in the numerator and its self-normalized denominator, thereby making the process scale invariant and contributing to its robustness. Herein we survey a number of results for self-normalized processes in the case of dependent variables and describe a key method called ``pseudo-maximization'' that has been used to derive these results. In the multivariate case, self-normalization consists of multiplying by the inverse of a positive definite matrix (instead of dividing by a positive random variable as in the scalar case) and is ubiquitous in statistical applications, examples of which are given.
△ Less
Submitted 10 October, 2007; v1 submitted 14 September, 2007;
originally announced September 2007.
-
Characterizations of joint distributions, copulas, information, dependence and decoupling, with applications to time series
Authors:
Victor H. de la Peña,
Rustam Ibragimov,
Shaturgun Sharakhmetov
Abstract:
In this paper, we obtain general representations for the joint distributions and copulas of arbitrary dependent random variables absolutely continuous with respect to the product of given one-dimensional marginal distributions. The characterizations obtained in the paper represent joint distributions of dependent random variables and their copulas as sums of $U$-statistics in independent random…
▽ More
In this paper, we obtain general representations for the joint distributions and copulas of arbitrary dependent random variables absolutely continuous with respect to the product of given one-dimensional marginal distributions. The characterizations obtained in the paper represent joint distributions of dependent random variables and their copulas as sums of $U$-statistics in independent random variables. We show that similar results also hold for expectations of arbitrary statistics in dependent random variables. As a corollary of the results, we obtain new representations for multivariate divergence measures as well as complete characterizations of important classes of dependent random variables that give, in particular, methods for constructing new copulas and modeling different dependence structures. The results obtained in the paper provide a device for reducing the analysis of convergence in distribution of a sum of a double array of dependent random variables to the study of weak convergence for a double array of their independent copies. Weak convergence in the dependent case is implied by similar asymptotic results under independence together with convergence to zero of one of a series of dependence measures including the multivariate extension of Pearson's correlation, the relative entropy or other multivariate divergence measures. A closely related result involves conditions for convergence in distribution of $m$-dimensional statistics $h(X_t,X_{t+1},...,X_{t+m-1})$ of time series $\{X_t\}$ in terms of weak convergence of $h(ξ_t,ξ_{t+1},...,ξ_{t+m-1})$, where $\{ξ_t\}$ is a sequence of independent copies of $X_t'$s, and convergence to zero of measures of intertemporal dependence in $\{X_t\}$. The tools used include new sharp estimates for the distance between the distribution function of an arbitrary statistic in dependent random variables and the distribution function of the statistic in independent copies of the random variables in terms of the measures of dependence of the random variables. Furthermore, we obtain new sharp complete decoupling moment and probability inequalities for dependent random variables in terms of their dependence characteristics.
△ Less
Submitted 7 November, 2006;
originally announced November 2006.
-
Inverse problems for random walks on trees: network tomography
Authors:
Victor de la Pena,
Henryk Gzyl,
Patrick McDonald
Abstract:
Let $G$ be a finite tree with root $r$ and associate to the internal vertices of $G$ a collection of transition probabilities for a simple nondegenerate Markov chain. Embedd $G$ into a graph $G^\prime$ constructed by gluing finite linear chains of length at least 2 to the terminal vertices of $G.$ Then $G^\prime$ admits distinguished boundary layers and the transition probabilities associated to…
▽ More
Let $G$ be a finite tree with root $r$ and associate to the internal vertices of $G$ a collection of transition probabilities for a simple nondegenerate Markov chain. Embedd $G$ into a graph $G^\prime$ constructed by gluing finite linear chains of length at least 2 to the terminal vertices of $G.$ Then $G^\prime$ admits distinguished boundary layers and the transition probabilities associated to the internal vertices of $G$ can be augmented to define a simple nondegenerate Markov chain $X$ on the vertices of $G^\prime.$ We show that the transition probabilities of $X$ can be recovered from the joint distribution of first hitting time and first hitting place of $X$ started at the root $r$ for the distinguished boundary layers of $G^\prime.$
△ Less
Submitted 26 October, 2006;
originally announced October 2006.
-
Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws
Authors:
Victor H. de la Pena,
Michael J. Klass,
Tze Leung Lai
Abstract:
Self-normalized processes arise naturally in statistical applications.
Being unit free, they are not affected by scale changes. Moreover, self-normalization often eliminates or weakens moment assumptions. In this paper we present several exponential and moment inequalities, particularly those related to laws of the iterated logarithm, for self-normalized random variables including martingales.…
▽ More
Self-normalized processes arise naturally in statistical applications.
Being unit free, they are not affected by scale changes. Moreover, self-normalization often eliminates or weakens moment assumptions. In this paper we present several exponential and moment inequalities, particularly those related to laws of the iterated logarithm, for self-normalized random variables including martingales. Tail probability bounds are also derived. For random variables B_t>0 and A_t, let Y_t(λ)=\exp{λA_t-λ^2B_t^2/2}. We develop inequalities for the moments of A_t/B_{t} or sup_{t\geq 0}A_t/{B_t(\log \log B_{t})^{1/2}} and variants thereof, when EY_t(λ)\leq 1 or when Y_t(λ) is a supermartingale, for all λbelonging to some interval. Our results are valid for a wide class of random processes including continuous martingales with A_t=M_t and B_t=\sqrt < M>_t, and sums of conditionally symmetric variables d_i with A_t=\sum_{i=1}^td_i and B_t=\sqrt\sum_{i=1}^td_i^2. A sharp maximal inequality for conditionally symmetric random variables and for continuous local martingales with values in R^m, m\ge 1, is also established. Another development in this paper is a bounded law of the iterated logarithm for general adapted sequences that are centered at certain truncated conditional expectations and self-normalized by the square root of the sum of squares. The key ingredient in this development is a new exponential supermartingale involving \sum_{i=1}^td_i and \sum_{i=1}^td_i^2.
△ Less
Submitted 5 October, 2004;
originally announced October 2004.
-
Contraction and decoupling inequalities for multilinear forms and u-statistics
Authors:
V. de la Pena,
Stephen J. Montgomery-Smith,
Jerzy Szulga
Abstract:
We prove decoupling inequalities for random polynomials in independent random variables with coefficients in vector space. We use various means of comparison, including rearrangement invariant norms (e.g., Orlicz and Lorentz norms), tail distributions, tightness, hypercontractivity, etc.
We prove decoupling inequalities for random polynomials in independent random variables with coefficients in vector space. We use various means of comparison, including rearrangement invariant norms (e.g., Orlicz and Lorentz norms), tail distributions, tightness, hypercontractivity, etc.
△ Less
Submitted 6 December, 1999; v1 submitted 7 June, 1994;
originally announced June 1994.
-
Decoupling Inequalities for the Tail Probabilities of Multivariate U-statistics
Authors:
Victor H. de la Peña,
Stephen J. Montgomery-Smith
Abstract:
In this paper the following result, which allows one to decouple U-Statistics in tail probability, is proved in full generality.
Theorem 1. Let $X_i$ be a sequence of independent random variables taking values in a measure space $S$, and let $f_{i_1...i_k}$ be measurable functions from $S^k$ to a Banach space $B$. Let $(X_i^{(j)})$ be independent copies of $(X_i)$. The following inequality hol…
▽ More
In this paper the following result, which allows one to decouple U-Statistics in tail probability, is proved in full generality.
Theorem 1. Let $X_i$ be a sequence of independent random variables taking values in a measure space $S$, and let $f_{i_1...i_k}$ be measurable functions from $S^k$ to a Banach space $B$. Let $(X_i^{(j)})$ be independent copies of $(X_i)$. The following inequality holds for all $t \ge 0$ and all $n\ge 2$, $$ P(||\sum_{1\le i_1 \ne ... \ne i_k \le n} f_{i_1 ... i_k}(X_{i_1},...,X_{i_k}) || \ge t) \qquad\qquad$$ $$ \qquad\qquad\le C_k P(C_k||\sum_{1\le i_1 \ne ... \ne i_k \le n} f_{i_1 ... i_k}(X_{i_1}^{(1)},...,X_{i_k}^{(k)}) || \ge t) .$$ Furthermore, the reverse inequality also holds in the case that the functions $\{f_{i_1... i_k}\}$ satisfy the symmetry condition $$ f_{i_1 ... i_k}(X_{i_1},...,X_{i_k}) = f_{i_{π(1)} ... i_{π(k)}}(X_{i_{π(1)}},...,X_{i_{π(k)}}) $$ for all permutations $π$ of $\{1,...,k\}$. Note that the expression $i_1 \ne ... \ne i_k$ means that $i_r \ne i_s$ for $r\ne s$. Also, $C_k$ is a constant that depends only on $k$.
△ Less
Submitted 6 December, 1999; v1 submitted 13 September, 1993;
originally announced September 1993.
-
Bounds on the tail probability of U-statistics and quadratic forms
Authors:
Victor H. de la Peña,
Stephen J. Montgomery-Smith
Abstract:
The authors announce a general tail estimate, called a decoupling inequality, for a symmetrized sum of non-linear $k$-correlations of $n>k$ independent random variables.
The authors announce a general tail estimate, called a decoupling inequality, for a symmetrized sum of non-linear $k$-correlations of $n>k$ independent random variables.
△ Less
Submitted 12 September, 1993;
originally announced September 1993.
