Title:Posterior Contraction and Credible Sets for Multivariate Regression Mode with Two-stage Improvements

Abstract:Locating the maximum of a function and its size in presence of noise is an important problem. The optimal rates for estimating them are respectively the same as those of estimating the function and all its first order partial derivatives, if one is allowed to sample in one shot only. It has been recently observed that substantial improvements are possible when one can obtain samples in two stages: a pilot estimate obtained in the first stage that guides to optimal sampling locations for the second stage sampling. If the second stage design points are chosen appropriately, the second stage rate can match the optimal sequential rate. In the Bayesian paradigm, one can naturally update uncertainty quantification based on past information and hence the two-stage method fits very naturally within the Bayesian framework. Nevertheless, Bayesian two-stage procedures for mode-hunting have not been studied in the literature. In this article, we provide posterior contraction rate and Bayesian credible sets with guaranteed frequentist coverage, which will allow us to quantify the uncertainty in the process. We consider anisotropic functions where function smoothness varies by direction. We use a random series prior based on tensor product B-splines with normal basis coefficients for the underlying function, and the error variance is either estimated using empirical Bayes or is further endowed with a conjugate inverse-gamma prior. The credible set obtained in the first stage is used to mark the sampling area for second stage sampling. We show that the second stage estimation achieves the optimal sequential rate and avoids the curse of dimensionality.