-
Necessary and sufficient conditions for posterior propriety for generalized linear mixed models
Authors:
Yalin Rao,
Vivekananda Roy
Abstract:
Generalized linear mixed models (GLMMs) are commonly used to analyze correlated discrete or continuous response data. In Bayesian GLMMs, the often-used improper priors may yield undesirable improper posterior distributions. Thus, verifying posterior propriety is crucial for valid applications of Bayesian GLMMs with improper priors. Here, we consider the popular improper uniform prior on the regres…
▽ More
Generalized linear mixed models (GLMMs) are commonly used to analyze correlated discrete or continuous response data. In Bayesian GLMMs, the often-used improper priors may yield undesirable improper posterior distributions. Thus, verifying posterior propriety is crucial for valid applications of Bayesian GLMMs with improper priors. Here, we consider the popular improper uniform prior on the regression coefficients and several proper or improper priors, including the widely used gamma and power priors on the variance components of the random effects. We also construct an approximate Jeffreys' prior for objective Bayesian analysis of GLMMs. For the two most widely used GLMMs, namely, the binomial and Poisson GLMMs, we provide easily verifiable sufficient conditions compared to the currently available results. We also derive the necessary conditions for posterior propriety for the general exponential family GLMMs. Finally, we use examples involving one-way and two-way random effects models to demonstrate the theoretical results derived here.
△ Less
Submitted 16 January, 2025; v1 submitted 1 February, 2023;
originally announced February 2023.
-
Posterior Impropriety of some Sparse Bayesian Learning Models
Authors:
Anand Dixit,
Vivekananda Roy
Abstract:
Sparse Bayesian learning models are typically used for prediction in datasets with significantly greater number of covariates than observations. Such models often take a reproducing kernel Hilbert space (RKHS) approach to carry out the task of prediction and can be implemented using either proper or improper priors. In this article we show that a few sparse Bayesian learning models in the literatu…
▽ More
Sparse Bayesian learning models are typically used for prediction in datasets with significantly greater number of covariates than observations. Such models often take a reproducing kernel Hilbert space (RKHS) approach to carry out the task of prediction and can be implemented using either proper or improper priors. In this article we show that a few sparse Bayesian learning models in the literature, when implemented using improper priors, lead to improper posteriors.
△ Less
Submitted 19 June, 2021; v1 submitted 1 August, 2020;
originally announced August 2020.
-
Geometric ergodicity of Polya-Gamma Gibbs sampler for Bayesian logistic regression with a flat prior
Authors:
Xin Wang,
Vivekananda Roy
Abstract:
The logistic regression model is the most popular model for analyzing binary data. In the absence of any prior information, an improper flat prior is often used for the regression coefficients in Bayesian logistic regression models. The resulting intractable posterior density can be explored by running Polson et al.'s (2013) data augmentation (DA) algorithm. In this paper, we establish that the Ma…
▽ More
The logistic regression model is the most popular model for analyzing binary data. In the absence of any prior information, an improper flat prior is often used for the regression coefficients in Bayesian logistic regression models. The resulting intractable posterior density can be explored by running Polson et al.'s (2013) data augmentation (DA) algorithm. In this paper, we establish that the Markov chain underlying Polson et al.'s (2013) DA algorithm is geometrically ergodic. Proving this theoretical result is practically important as it ensures the existence of central limit theorems (CLTs) for sample averages under a finite second moment condition. The CLT in turn allows users of the DA algorithm to calculate standard errors for posterior estimates.
△ Less
Submitted 1 July, 2018; v1 submitted 17 February, 2018;
originally announced February 2018.
-
Analysis of the Polya-Gamma block Gibbs sampler for Bayesian logistic linear mixed models
Authors:
Xin Wang,
Vivekananda Roy
Abstract:
In this article, we construct a two-block Gibbs sampler using Polson et al. (2013) data augmentation technique with Polya-Gamma latent variables for Bayesian logistic linear mixed models under proper priors. Furthermore, we prove the uniform ergodicity of this Gibbs sampler, which guarantees the existence of the central limit theorems for MCMC based estimators.
In this article, we construct a two-block Gibbs sampler using Polson et al. (2013) data augmentation technique with Polya-Gamma latent variables for Bayesian logistic linear mixed models under proper priors. Furthermore, we prove the uniform ergodicity of this Gibbs sampler, which guarantees the existence of the central limit theorems for MCMC based estimators.
△ Less
Submitted 17 November, 2017; v1 submitted 31 July, 2017;
originally announced August 2017.
-
Convergence analysis of the block Gibbs sampler for Bayesian probit linear mixed models with improper priors
Authors:
Xin Wang,
Vivekananda Roy
Abstract:
In this article, we consider Markov chain Monte Carlo(MCMC) algorithms for exploring the intractable posterior density associated with Bayesian probit linear mixed models under improper priors on the regression coefficients and variance components. In particular, we construct the two-block Gibbs sampler using the data augmentation (DA) techniques. Furthermore, we prove geometric ergodicity of the…
▽ More
In this article, we consider Markov chain Monte Carlo(MCMC) algorithms for exploring the intractable posterior density associated with Bayesian probit linear mixed models under improper priors on the regression coefficients and variance components. In particular, we construct the two-block Gibbs sampler using the data augmentation (DA) techniques. Furthermore, we prove geometric ergodicity of the Gibbs sampler, which is the foundation for building central limit theorems for MCMC based estimators and subsequent inferences. The conditions for geometric convergence are similar to those guaranteeing posterior propriety. We also provide conditions for posterior propriety when the design matrices take commonly observed forms. In general, the Haar parameter expansion for DA (PX- DA) algorithm is an improvement of the DA algorithm and it has been shown that it is theoretically at least as good as the DA algorithm. Here we construct a Haar PX-DA algorithm, which has essentially the same computational cost as the two-block Gibbs sampler.
△ Less
Submitted 22 November, 2018; v1 submitted 6 June, 2017;
originally announced June 2017.
-
Estimating standard errors for importance sampling estimators with multiple Markov chains
Authors:
Vivekananda Roy,
Aixin Tan,
James M. Flegal
Abstract:
The naive importance sampling estimator, based on samples from a single importance density, can be numerically unstable. Instead, we consider generalized importance sampling estimators where samples from more than one probability distribution are combined. We study this problem in the Markov chain Monte Carlo context, where independent samples are replaced with Markov chain samples. If the chains…
▽ More
The naive importance sampling estimator, based on samples from a single importance density, can be numerically unstable. Instead, we consider generalized importance sampling estimators where samples from more than one probability distribution are combined. We study this problem in the Markov chain Monte Carlo context, where independent samples are replaced with Markov chain samples. If the chains converge to their respective target distributions at a polynomial rate, then under two finite moment conditions, we show a central limit theorem holds for the generalized estimators. Further, we develop an easy to implement method to calculate valid asymptotic standard errors based on batch means. We also provide a batch means estimator for calculating asymptotically valid standard errors of Geyer(1994) reverse logistic estimator. We illustrate the method using a Bayesian variable selection procedure in linear regression. In particular, the generalized importance sampling estimator is used to perform empirical Bayes variable selection and the batch means estimator is used to obtain standard errors in a high-dimensional setting where current methods are not applicable.
△ Less
Submitted 11 August, 2016; v1 submitted 21 September, 2015;
originally announced September 2015.
-
Local times in a Brownian excursion
Authors:
Krishna B. Athreya,
Raoul Normand,
Vivekananda Roy,
Sheng-Jhih Wu
Abstract:
Let $\{B(t), t \geq 0\}$ be a standard Brownian motion in $\mathbb{R}$. Let $T$ be the first return time to 0 after hitting 1, and $\{L(T,x), x \in \mathbb{R}\}$ be the local time process at time $T$ and level $x$. The distribution of $L(T,x)$ for each $x \in \mathbb{R}$ is determined. This is applied to the estimation of a $L^1$ integral on $\mathbb{R}$.
Let $\{B(t), t \geq 0\}$ be a standard Brownian motion in $\mathbb{R}$. Let $T$ be the first return time to 0 after hitting 1, and $\{L(T,x), x \in \mathbb{R}\}$ be the local time process at time $T$ and level $x$. The distribution of $L(T,x)$ for each $x \in \mathbb{R}$ is determined. This is applied to the estimation of a $L^1$ integral on $\mathbb{R}$.
△ Less
Submitted 17 October, 2014;
originally announced October 2014.
