-
Robust Bayesian Inference for Censored Survival Models
Authors:
Yasuyuki Hamura,
Takahiro Onizuka,
Shintaro Hashimoto,
Shonosuke Sugasawa
Abstract:
This paper proposes a robust Bayesian accelerated failure time model for censored survival data. We develop a new family of life-time distributions using a scale mixture of the generalized gamma distributions, where we propose a novel super heavy-tailed distribution as a mixing density. We theoretically show that, under some conditions, the proposed method satisfies the full posterior robustness,…
▽ More
This paper proposes a robust Bayesian accelerated failure time model for censored survival data. We develop a new family of life-time distributions using a scale mixture of the generalized gamma distributions, where we propose a novel super heavy-tailed distribution as a mixing density. We theoretically show that, under some conditions, the proposed method satisfies the full posterior robustness, which guarantees robustness of point estimation as well as uncertainty quantification. For posterior computation, we employ an integral expression of the proposed heavy-tailed distribution to develop an efficient posterior computation algorithm based on the Markov chain Monte Carlo. The performance of the proposed method is illustrated through numerical experiments and real data example.
△ Less
Submitted 15 April, 2025;
originally announced April 2025.
-
Geometric Ergodicity of Gibbs Algorithms for a Normal Model With a Global-Local Shrinkage Prior
Authors:
Yasuyuki Hamura
Abstract:
In this paper, we consider Gibbs samplers for a normal linear regression model with a global-local shrinkage prior. We show that they produce geometrically ergodic Markov chains under some assumptions. In the first half of the paper, we prove geometric ergodicity under the horseshoe local prior and a three-parameter beta global prior which does not have a finite $(p / 5)$-th negative moment, where…
▽ More
In this paper, we consider Gibbs samplers for a normal linear regression model with a global-local shrinkage prior. We show that they produce geometrically ergodic Markov chains under some assumptions. In the first half of the paper, we prove geometric ergodicity under the horseshoe local prior and a three-parameter beta global prior which does not have a finite $(p / 5)$-th negative moment, where $p$ is the number of regression coefficients. This is in contrast to the case of a known general result which is applicable if the global parameter has a finite approximately $(p / 2)$-th negative moment. In the second half of the paper, we consider a more general class of global-local shrinkage priors. Geometric ergodicity is proved for two-stage and three-stage Gibbs samplers based on rejection sampling without assuming the negative moment condition.
△ Less
Submitted 9 May, 2025; v1 submitted 1 March, 2025;
originally announced March 2025.
-
A Short Note on the Efficiency of Markov Chains for Bayesian Linear Regression Models with Heavy-Tailed Errors
Authors:
Yasuyuki Hamura
Abstract:
In this short note, we consider posterior simulation for a linear regression model when the error distribution is given by a scale mixture of multivariate normals. We first show that the sampler of Backlund and Hobert (2020) for the case of the conditionally conjugate normal-inverse Wishart prior continues to be geometrically ergodic even when the error density is heavier-tailed. Moreover, we prov…
▽ More
In this short note, we consider posterior simulation for a linear regression model when the error distribution is given by a scale mixture of multivariate normals. We first show that the sampler of Backlund and Hobert (2020) for the case of the conditionally conjugate normal-inverse Wishart prior continues to be geometrically ergodic even when the error density is heavier-tailed. Moreover, we prove that the ergodicity is uniform by verifying the minorization condition. In the second half of this note, we treat an improper case and show that the sampler of Section 4 of Roy and Hobert (2010) is geometrically ergodic under significantly milder conditions.
△ Less
Submitted 28 October, 2024; v1 submitted 22 October, 2024;
originally announced October 2024.
-
An Approximate Identity Link Function for Bayesian Generalized Linear Models
Authors:
Yasuyuki Hamura
Abstract:
In this note, we consider using a link function that has heavier tails than the usual exponential link function. We construct efficient Gibbs algorithms for Poisson and Multinomial models based on this link function by introducing gamma and inverse Gaussian latent variables and show that the algorithms generate geometrically ergodic Markov chains in simple settings. Our algorithms can be used for…
▽ More
In this note, we consider using a link function that has heavier tails than the usual exponential link function. We construct efficient Gibbs algorithms for Poisson and Multinomial models based on this link function by introducing gamma and inverse Gaussian latent variables and show that the algorithms generate geometrically ergodic Markov chains in simple settings. Our algorithms can be used for more complicated models with many parameters. We fit our simple Poisson model to a real dataset and confirm that the posterior distribution has similar implications to those under the usual Poisson regression model based on the exponential link function. Although less interpretable, our models are potentially more tractable or flexible from a computational point of view in some cases.
△ Less
Submitted 14 October, 2024;
originally announced October 2024.
-
State-Space Modeling of Shape-constrained Functional Time Series
Authors:
Daichi Hiraki,
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Functional time series data frequently appears in econometric analyses, where the functions of interest are subject to some shape constraints, including monotonicity and convexity, as typical of the estimation of the Lorenz curve. This paper proposes a state-space model for time-varying functions to extract trends and serial dependence from functional time series while imposing the shape constrain…
▽ More
Functional time series data frequently appears in econometric analyses, where the functions of interest are subject to some shape constraints, including monotonicity and convexity, as typical of the estimation of the Lorenz curve. This paper proposes a state-space model for time-varying functions to extract trends and serial dependence from functional time series while imposing the shape constraints on the estimated functions. The function of interest is modeled by a convex combination of selected basis functions to satisfy the shape constraints, where the time-varying convex weights on simplex follow the dynamic multi-logit models. To enable posterior computation by an efficient Markov chain Monte Carlo method, a novel data augmentation technique is devised for the complicated likelihood of this model. The proposed method is applied to the estimation of time-varying Lorenz curves, and its utility is illustrated through numerical experiments and analysis of panel data of household incomes in Japan.
△ Less
Submitted 1 December, 2024; v1 submitted 11 April, 2024;
originally announced April 2024.
-
Posterior Robustness with Milder Conditions: Contamination Models Revisited
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Robust Bayesian linear regression is a classical but essential statistical tool. Although novel robustness properties of posterior distributions have been proved recently under a certain class of error distributions, their sufficient conditions are restrictive and exclude several important situations. In this work, we revisit a classical two-component mixture model for response variables, also kno…
▽ More
Robust Bayesian linear regression is a classical but essential statistical tool. Although novel robustness properties of posterior distributions have been proved recently under a certain class of error distributions, their sufficient conditions are restrictive and exclude several important situations. In this work, we revisit a classical two-component mixture model for response variables, also known as contamination model, where one component is a light-tailed regression model and the other component is heavy-tailed. The latter component is independent of the regression parameters, which is crucial in proving the posterior robustness. We obtain new sufficient conditions for posterior (non-)robustness and reveal non-trivial robustness results by using those conditions. In particular, we find that even the Student-$t$ error distribution can achieve the posterior robustness in our framework. A numerical study is performed to check the Kullback-Leibler divergence between the posterior distribution based on full data and that based on data obtained by removing outliers.
△ Less
Submitted 3 April, 2024; v1 submitted 1 March, 2023;
originally announced March 2023.
-
Gibbs Sampler for Matrix Generalized Inverse Gaussian Distributions
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Sampling from matrix generalized inverse Gaussian (MGIG) distributions is required in Markov Chain Monte Carlo (MCMC) algorithms for a variety of statistical models. However, an efficient sampling scheme for the MGIG distributions has not been fully developed. We here propose a novel blocked Gibbs sampler for the MGIG distributions, based on the Choleski decomposition. We show that the full condit…
▽ More
Sampling from matrix generalized inverse Gaussian (MGIG) distributions is required in Markov Chain Monte Carlo (MCMC) algorithms for a variety of statistical models. However, an efficient sampling scheme for the MGIG distributions has not been fully developed. We here propose a novel blocked Gibbs sampler for the MGIG distributions, based on the Choleski decomposition. We show that the full conditionals of the diagonal and unit lower-triangular entries are univariate generalized inverse Gaussian and multivariate normal distributions, respectively. Several variants of the Metropolis-Hastings algorithm can also be considered for this problem, but we mathematically prove that the average acceptance rates become extremely low in particular scenarios. We demonstrate the computational efficiency of the proposed Gibbs sampler through simulation studies and data analysis.
△ Less
Submitted 19 February, 2023;
originally announced February 2023.
-
Locally Adaptive Bayesian Isotonic Regression using Half Shrinkage Priors
Authors:
Ryo Okano,
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Isotonic regression or monotone function estimation is a problem of estimating function values under monotonicity constraints, which appears naturally in many scientific fields. This paper proposes a new Bayesian method with global-local shrinkage priors for estimating monotone function values. Specifically, we introduce half shrinkage priors for positive valued random variables and assign them fo…
▽ More
Isotonic regression or monotone function estimation is a problem of estimating function values under monotonicity constraints, which appears naturally in many scientific fields. This paper proposes a new Bayesian method with global-local shrinkage priors for estimating monotone function values. Specifically, we introduce half shrinkage priors for positive valued random variables and assign them for the first-order differences of function values. We also develop fast and simple Gibbs sampling algorithms for full posterior analysis. By incorporating advanced shrinkage priors, the proposed method is adaptive to local abrupt changes or jumps in target functions. We show this adaptive property theoretically by proving that the posterior mean estimators are robust to large differences and that asymptotic risk for unchanged points can be improved. Finally, we demonstrate the proposed methods through simulations and applications to a real data set.
△ Less
Submitted 6 February, 2024; v1 submitted 9 August, 2022;
originally announced August 2022.
-
Sparse Bayesian inference on gamma-distributed observations using shape-scale inverse-gamma mixtures
Authors:
Yasuyuki Hamura,
Takahiro Onizuka,
Shintaro Hashimoto,
Shonosuke Sugasawa
Abstract:
In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma…
▽ More
In various applications, we deal with high-dimensional positive-valued data that often exhibits sparsity. This paper develops a new class of continuous global-local shrinkage priors tailored to analyzing gamma-distributed observations where most of the underlying means are concentrated around a certain value. Unlike existing shrinkage priors, our new prior is a shape-scale mixture of inverse-gamma distributions, which has a desirable interpretation of the form of posterior mean and admits flexible shrinkage. We show that the proposed prior has two desirable theoretical properties; Kullback-Leibler super-efficiency under sparsity and robust shrinkage rules for large observations. We propose an efficient sampling algorithm for posterior inference. The performance of the proposed method is illustrated through simulation and two real data examples, the average length of hospital stay for COVID-19 in South Korea and adaptive variance estimation of gene expression data.
△ Less
Submitted 30 November, 2022; v1 submitted 16 March, 2022;
originally announced March 2022.
-
On Data Augmentation for Models Involving Reciprocal Gamma Functions
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
In this paper, we introduce a new and efficient data augmentation approach to the posterior inference of the models with shape parameters when the reciprocal gamma function appears in full conditional densities. Our approach is to approximate full conditional densities of shape parameters by using Gauss's multiplication formula and Stirling's formula for the gamma function, where the approximation…
▽ More
In this paper, we introduce a new and efficient data augmentation approach to the posterior inference of the models with shape parameters when the reciprocal gamma function appears in full conditional densities. Our approach is to approximate full conditional densities of shape parameters by using Gauss's multiplication formula and Stirling's formula for the gamma function, where the approximation error can be made arbitrarily small. We use the techniques to construct efficient Gibbs and Metropolis-Hastings algorithms for a variety of models that involve the gamma distribution, Student's $t$-distribution, the Dirichlet distribution, the negative binomial distribution, and the Wishart distribution. The proposed sampling method is numerically demonstrated through simulation studies.
△ Less
Submitted 26 August, 2022; v1 submitted 3 March, 2022;
originally announced March 2022.
-
Robust Bayesian Modeling of Counts with Zero inflation and Outliers: Theoretical Robustness and Efficient Computation
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Count data with zero inflation and large outliers are ubiquitous in many scientific applications. However, posterior analysis under a standard statistical model, such as Poisson or negative binomial distribution, is sensitive to such contamination. This study introduces a novel framework for Bayesian modeling of counts that is robust to both zero inflation and large outliers. In doing so, we intro…
▽ More
Count data with zero inflation and large outliers are ubiquitous in many scientific applications. However, posterior analysis under a standard statistical model, such as Poisson or negative binomial distribution, is sensitive to such contamination. This study introduces a novel framework for Bayesian modeling of counts that is robust to both zero inflation and large outliers. In doing so, we introduce rescaled beta distribution and adopt it to absorb undesirable effects from zero and outlying counts. The proposed approach has two appealing features: the efficiency of the posterior computation via a custom Gibbs sampling algorithm and a theoretically guaranteed posterior robustness, where extreme outliers are automatically removed from the posterior distribution. We demonstrate the usefulness of the proposed method by applying it to trend filtering and spatial modeling using predictive Gaussian processes.
△ Less
Submitted 8 May, 2024; v1 submitted 19 June, 2021;
originally announced June 2021.
-
Bayesian Point Estimation and Predictive Density Estimation for the Binomial Distribution with a Restricted Probability Parameter
Authors:
Yasuyuki Hamura
Abstract:
In this paper, we consider Bayesian point estimation and predictive density estimation in the binomial case. After presenting preliminary results on these problems, we compare the risk functions of the Bayes estimators based on the truncated and untruncated beta priors and obtain dominance conditions when the probability parameter is less than or equal to a known constant. The case where there are…
▽ More
In this paper, we consider Bayesian point estimation and predictive density estimation in the binomial case. After presenting preliminary results on these problems, we compare the risk functions of the Bayes estimators based on the truncated and untruncated beta priors and obtain dominance conditions when the probability parameter is less than or equal to a known constant. The case where there are both a lower bound restriction and an upper bound restriction is also treated. Then our problems are shown to be related to similar problems in the Poisson case. Finally, numerical studies are presented.
△ Less
Submitted 20 March, 2021; v1 submitted 28 February, 2021;
originally announced March 2021.
-
Bayesian Shrinkage Approaches to Unbalanced Problems of Estimation and Prediction on the Basis of Negative Multinomial Samples
Authors:
Yasuyuki Hamura
Abstract:
In this paper, we treat estimation and prediction problems where negative multinomial variables are observed and in particular consider unbalanced settings. First, the problem of estimating multiple negative multinomial parameter vectors under the standardized squared error loss is treated and a new empirical Bayes estimator which dominates the UMVU estimator under suitable conditions is derived.…
▽ More
In this paper, we treat estimation and prediction problems where negative multinomial variables are observed and in particular consider unbalanced settings. First, the problem of estimating multiple negative multinomial parameter vectors under the standardized squared error loss is treated and a new empirical Bayes estimator which dominates the UMVU estimator under suitable conditions is derived. Second, we consider estimation of the joint predictive density of several multinomial tables under the Kullback-Leibler divergence and obtain a sufficient condition under which the Bayesian predictive density with respect to a hierarchical shrinkage prior dominates the Bayesian predictive density with respect to the Jeffreys prior. Third, our proposed Bayesian estimator and predictive density give risk improvements in simulations. Finally, the problem of estimating the joint predictive density of negative multinomial variables is discussed.
△ Less
Submitted 19 November, 2021; v1 submitted 6 October, 2020;
originally announced October 2020.
-
Bayesian Predictive Density Estimation for a Chi-squared Model Using Information from a Normal Observation with Unknown Mean and Variance
Authors:
Yasuyuki Hamura,
Tatsuya Kubokawa
Abstract:
In this paper, we consider the problem of estimating the density function of a Chi-squared variable on the basis of observations of another Chi-squared variable and a normal variable under the Kullback-Leibler divergence. We assume that these variables have a common unknown scale parameter and that the mean of the normal variable is also unknown. We compare the risk functions of two Bayesian predi…
▽ More
In this paper, we consider the problem of estimating the density function of a Chi-squared variable on the basis of observations of another Chi-squared variable and a normal variable under the Kullback-Leibler divergence. We assume that these variables have a common unknown scale parameter and that the mean of the normal variable is also unknown. We compare the risk functions of two Bayesian predictive densities: one with respect to a hierarchical shrinkage prior and the other based on a noninformative prior. The hierarchical Bayesian predictive density depends on the normal variable while the Bayesian predictive density based on the noninformative prior does not. Sufficient conditions for the former to dominate the latter are obtained. These predictive densities are compared by simulation.
△ Less
Submitted 19 July, 2020; v1 submitted 12 June, 2020;
originally announced June 2020.
-
Log-Regularly Varying Scale Mixture of Normals for Robust Regression
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Linear regression with the classical normality assumption for the error distribution may lead to an undesirable posterior inference of regression coefficients due to the potential outliers. This paper considers the finite mixture of two components with thin and heavy tails as the error distribution, which has been routinely employed in applied statistics. For the heavily-tailed component, we intro…
▽ More
Linear regression with the classical normality assumption for the error distribution may lead to an undesirable posterior inference of regression coefficients due to the potential outliers. This paper considers the finite mixture of two components with thin and heavy tails as the error distribution, which has been routinely employed in applied statistics. For the heavily-tailed component, we introduce the novel class of distributions; their densities are log-regularly varying and have heavier tails than those of Cauchy distribution, yet they are expressed as a scale mixture of normal distributions and enable the efficient posterior inference by Gibbs sampler. We prove the robustness to outliers of the posterior distributions under the proposed models with a minimal set of assumptions, which justifies the use of shrinkage priors with unbounded densities for the coefficient vector in the presence of outliers. The extensive comparison with the existing methods via simulation study shows the improved performance of our model in point and interval estimation, as well as its computational efficiency. Further, we confirm the posterior robustness of our method in the empirical study with the shrinkage priors for regression coefficients.
△ Less
Submitted 9 January, 2021; v1 submitted 6 May, 2020;
originally announced May 2020.
-
Bayesian Shrinkage Estimation of Negative Multinomial Parameter Vectors
Authors:
Yasuyuki Hamura,
Tatsuya Kubokawa
Abstract:
The negative multinomial distribution is a multivariate generalization of the negative binomial distribution. In this paper, we consider the problem of estimating an unknown matrix of probabilities on the basis of observations of negative multinomial variables under the standardized squared error loss. First, a general sufficient condition for a shrinkage estimator to dominate the UMVU estimator i…
▽ More
The negative multinomial distribution is a multivariate generalization of the negative binomial distribution. In this paper, we consider the problem of estimating an unknown matrix of probabilities on the basis of observations of negative multinomial variables under the standardized squared error loss. First, a general sufficient condition for a shrinkage estimator to dominate the UMVU estimator is derived and an empirical Bayes estimator satisfying the condition is constructed. Next, a hierarchical shrinkage prior is introduced, an associated Bayes estimator is shown to dominate the UMVU estimator under some conditions, and some remarks about posterior computation are presented. Finally, shrinkage estimators and the UMVU estimator are compared by simulation.
△ Less
Submitted 29 October, 2020; v1 submitted 27 January, 2020;
originally announced January 2020.
-
Shrinkage with Robustness: Log-Adjusted Priors for Sparse Signals
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
We introduce a new class of distributions named log-adjusted shrinkage priors for the analysis of sparse signals, which extends the three parameter beta priors by multiplying an additional log-term to their densities. The proposed prior has density tails that are heavier than even those of the Cauchy distribution and realizes the tail-robustness of the Bayes estimator, while keeping the strong shr…
▽ More
We introduce a new class of distributions named log-adjusted shrinkage priors for the analysis of sparse signals, which extends the three parameter beta priors by multiplying an additional log-term to their densities. The proposed prior has density tails that are heavier than even those of the Cauchy distribution and realizes the tail-robustness of the Bayes estimator, while keeping the strong shrinkage effect on noises. We verify this property via the improved posterior mean squared errors in the tail. An integral representation with latent variables for the new density is available and enables fast and simple Gibbs samplers for the full posterior analysis. Our log-adjusted prior is significantly different from existing shrinkage priors with logarithms for allowing its further generalization by multiple log-terms in the density. The performance of the proposed priors is investigated through simulation studies and data analysis.
△ Less
Submitted 26 January, 2020; v1 submitted 23 January, 2020;
originally announced January 2020.
-
On Global-local Shrinkage Priors for Count Data
Authors:
Yasuyuki Hamura,
Kaoru Irie,
Shonosuke Sugasawa
Abstract:
Global-local shrinkage prior has been recognized as useful class of priors which can strongly shrink small signals towards prior means while keeping large signals unshrunk. Although such priors have been extensively discussed under Gaussian responses, we intensively encounter count responses in practice in which the previous knowledge of global-local shrinkage priors cannot be directly imported. I…
▽ More
Global-local shrinkage prior has been recognized as useful class of priors which can strongly shrink small signals towards prior means while keeping large signals unshrunk. Although such priors have been extensively discussed under Gaussian responses, we intensively encounter count responses in practice in which the previous knowledge of global-local shrinkage priors cannot be directly imported. In this paper, we discuss global-local shrinkage priors for analyzing sequence of counts. We provide sufficient conditions under which the posterior mean keeps the observation as it is for very large signals, known as tail robustness property. Then, we propose tractable priors to meet the derived conditions approximately or exactly and develop an efficient posterior computation algorithm for Bayesian inference. The proposed methods are free from tuning parameters, that is, all the hyperparameters are automatically estimated based on the data. We demonstrate the proposed methods through simulation and an application to a real dataset.
△ Less
Submitted 16 August, 2020; v1 submitted 2 July, 2019;
originally announced July 2019.
