Showing 1–35 of 35 results for author: van Zanten, H

Optimal testing using combined test statistics across independent studies

Authors: Botond Szabó, Aad van der Vaart, Lasse Vuursteen, Harry van Zanten

Abstract: Combining test statistics from independent trials or experiments is a popular method of meta-analysis. However, there is very limited theoretical understanding of the power of the combined test, especially in high-dimensional models considering composite hypotheses tests. We derive a mathematical framework to study standard {meta-analysis} testing approaches in the context of the many normal means… ▽ More Combining test statistics from independent trials or experiments is a popular method of meta-analysis. However, there is very limited theoretical understanding of the power of the combined test, especially in high-dimensional models considering composite hypotheses tests. We derive a mathematical framework to study standard {meta-analysis} testing approaches in the context of the many normal means model, which serves as the platform to investigate more complex models. We introduce a natural and mild restriction on the meta-level combination functions of the local trials. This allows us to mathematically quantify the cost of compressing $m$ trials into real-valued test statistics and combining these. We then derive minimax lower and matching upper bounds for the separation rates of standard combination methods for e.g. p-values and e-values, quantifying the loss relative to using the full, pooled data. We observe an elbow effect, revealing that in certain cases combining the locally optimal tests in each trial results in a sub-optimal {meta-analysis} method and develop approaches to achieve the global optima. We also explore the possible gains of allowing limited coordination between the trial designs. Our results connect meta-analysis with bandwidth constraint distributed inference and build on recent information theoretic developments in the latter field. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Comments: NeuRIPS 2023. 27 pages, 3 figures

MSC Class: 62B10; 62C20; 62F30; 62F03

Semi-parametric Bernstein-von Mises in Linear Inverse Problems

Authors: Adel Magra, Aad van der Vaart, Harry van Zanten

Abstract: We consider a Bayesian approach for the recovery of scalar parameters arising in inverse problems. We consider a general signal-in white noise model where we have access to two independent noisy observations of a function, and of a linear transformation of the function. The linear operator is unknown up to a scalar parameter. We present a Bernstein-von Mises theorem for the marginal posterior of t… ▽ More We consider a Bayesian approach for the recovery of scalar parameters arising in inverse problems. We consider a general signal-in white noise model where we have access to two independent noisy observations of a function, and of a linear transformation of the function. The linear operator is unknown up to a scalar parameter. We present a Bernstein-von Mises theorem for the marginal posterior of the scalar under regularity assumptions of the operator. We further derive Bernstein-von Mises results for different priors and apply them to two concrete examples: the recovery of the thermal diffusivity in a heat equation problem, and the recovery of a location parameter in a semi-blind deconvolution problem. △ Less

Submitted 13 March, 2025; v1 submitted 4 October, 2023; originally announced October 2023.

MSC Class: 62F15; 62E20

Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression

Authors: Dennis Nieman, Botond Szabo, Harry van Zanten

Abstract: We investigate the frequentist guarantees of the variational sparse Gaussian process regression model. In the theoretical analysis, we focus on the variational approach with spectral features as inducing variables. We derive guarantees and limitations for the frequentist coverage of the resulting variational credible sets. We also derive sufficient and necessary lower bounds for the number of indu… ▽ More We investigate the frequentist guarantees of the variational sparse Gaussian process regression model. In the theoretical analysis, we focus on the variational approach with spectral features as inducing variables. We derive guarantees and limitations for the frequentist coverage of the resulting variational credible sets. We also derive sufficient and necessary lower bounds for the number of inducing variables required to achieve minimax posterior contraction rates. The implications of these results are demonstrated for different choices of priors. In a numerical analysis we consider a wider range of inducing variable methods and observe similar phenomena beyond the scope of our theoretical findings. △ Less

Submitted 28 September, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Optimal high-dimensional and nonparametric distributed testing under communication constraints

Authors: Botond Szabó, Lasse Vuursteen, Harry van Zanten

Abstract: We derive minimax testing errors in a distributed framework where the data is split over multiple machines and their communication to a central machine is limited to $b$ bits. We investigate both the $d$- and infinite-dimensional signal detection problem under Gaussian white noise. We also derive distributed testing algorithms reaching the theoretical lower bounds. Our results show that distribu… ▽ More We derive minimax testing errors in a distributed framework where the data is split over multiple machines and their communication to a central machine is limited to $b$ bits. We investigate both the $d$- and infinite-dimensional signal detection problem under Gaussian white noise. We also derive distributed testing algorithms reaching the theoretical lower bounds. Our results show that distributed testing is subject to fundamentally different phenomena that are not observed in distributed estimation. Among our findings, we show that testing protocols that have access to shared randomness can perform strictly better in some regimes than those that do not. We also observe that consistent nonparametric distributed testing is always possible, even with as little as $1$-bit of communication and the corresponding test outperforms the best local test using only the information available at a single local machine. Furthermore, we also derive adaptive nonparametric distributed testing strategies and the corresponding theoretical lower bounds. △ Less

Submitted 11 December, 2022; v1 submitted 2 February, 2022; originally announced February 2022.

Comments: 53 pages

MSC Class: 62G10; 62F30; 62F03

Contraction rates for sparse variational approximations in Gaussian process regression

Authors: Dennis Nieman, Botond Szabo, Harry van Zanten

Abstract: We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009a) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matérn, squared exponential, rand… ▽ More We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009a) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. As examples we show that for three particular covariance kernels (Matérn, squared exponential, random series prior) the VB approach can achieve optimal, minimax contraction rates for a sufficiently large number of appropriately chosen inducing variables. The theoretical findings are demonstrated by numerical experiments. △ Less

Submitted 30 March, 2022; v1 submitted 22 September, 2021; originally announced September 2021.

Comments: 26 pages, 6 figures, 1 table

Optimal distributed composite testing in high-dimensional Gaussian models with 1-bit communication

Authors: Botond Szabo, Lasse Vuursteen, Harry van Zanten

Abstract: In this paper we study the problem of signal detection in Gaussian noise in a distributed setting where the local machines in the star topology can communicate a single bit of information. We derive a lower bound on the Euclidian norm that the signal needs to have in order to be detectable. Moreover, we exhibit optimal distributed testing strategies that attain the lower bound. In this paper we study the problem of signal detection in Gaussian noise in a distributed setting where the local machines in the star topology can communicate a single bit of information. We derive a lower bound on the Euclidian norm that the signal needs to have in order to be detectable. Moreover, we exhibit optimal distributed testing strategies that attain the lower bound. △ Less

Submitted 24 February, 2022; v1 submitted 9 December, 2020; originally announced December 2020.

Comments: 33 pages, 2 figures. To appear in IEEE Transactions on Information Theory

MSC Class: Primary: 62F03; 62F30. Secondary: 94A13; 68W15

Distributed function estimation: adaptation using minimal communication

Authors: Botond Szabo, Harry van Zanten

Abstract: We investigate whether in a distributed setting, adaptive estimation of a smooth function at the optimal rate is possible under minimal communication. It turns out that the answer depends on the risk considered and on the number of servers over which the procedure is distributed. We show that for the $L_\infty$-risk, adaptively obtaining optimal rates under minimal communication is not possible. F… ▽ More We investigate whether in a distributed setting, adaptive estimation of a smooth function at the optimal rate is possible under minimal communication. It turns out that the answer depends on the risk considered and on the number of servers over which the procedure is distributed. We show that for the $L_\infty$-risk, adaptively obtaining optimal rates under minimal communication is not possible. For the $L_2$-risk, it is possible over a range of regularities that depends on the relation between the number of local servers and the total sample size. △ Less

Submitted 28 March, 2020; originally announced March 2020.

Comments: 40 pages

MSC Class: 62G20; 62G10; 62G05; 94A15

Adaptive distributed methods under communication constraints

Authors: Botond Szabo, Harry van Zanten

Abstract: We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting. We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting. △ Less

Submitted 4 February, 2019; v1 submitted 3 April, 2018; originally announced April 2018.

Comments: 46 pages

MSC Class: 62G05; 62G20

An asymptotic analysis of distributed nonparametric methods

Authors: Botond Szabo, Harry van Zanten

Abstract: We investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which serves as a benchmark model for studying performance in this setting. The results show how the design and tuning of a distributed method can have great impact… ▽ More We investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which serves as a benchmark model for studying performance in this setting. The results show how the design and tuning of a distributed method can have great impact on convergence rates and validity of uncertainty quantification. Moreover, we highlight the difficulty of designing nonparametric distributed procedures that automatically adapt to smoothness. △ Less

Submitted 8 November, 2017; originally announced November 2017.

Comments: 29 pages, 4 figures

MSC Class: 62G20; 62G15; 62G05

Minimax lower bounds for function estimation on graphs

Authors: Alisa Kirichenko, Harry van Zanten

Abstract: We study minimax lower bounds for function estimation problems on large graph when the target function is smoothly varying over the graph. We derive minimax rates in the context of regression and classification problems on graphs that satisfy an asymptotic shape assumption and with a smoothness condition on the target function, both formulated in terms of the graph Laplacian. We study minimax lower bounds for function estimation problems on large graph when the target function is smoothly varying over the graph. We derive minimax rates in the context of regression and classification problems on graphs that satisfy an asymptotic shape assumption and with a smoothness condition on the target function, both formulated in terms of the graph Laplacian. △ Less

Submitted 15 February, 2018; v1 submitted 19 September, 2017; originally announced September 2017.

Full adaptation to smoothness using randomly truncated series priors with Gaussian coefficients and inverse gamma scaling

Authors: Jan van Waaij, Harry van Zanten

Abstract: We study random series priors for estimating a functional parameter (f\in L^2[0,1]). We show that with a series prior with random truncation, Gaussian coefficients, and inverse gamma multiplicative scaling, it is possible to achieve posterior contraction at optimal rates and adaptation to arbitrary degrees of smoothness. We present general results that can be combined with existing rate of contrac… ▽ More We study random series priors for estimating a functional parameter (f\in L^2[0,1]). We show that with a series prior with random truncation, Gaussian coefficients, and inverse gamma multiplicative scaling, it is possible to achieve posterior contraction at optimal rates and adaptation to arbitrary degrees of smoothness. We present general results that can be combined with existing rate of contraction results for various nonparametric estimation problems. We give concrete examples for signal estimation in white noise and drift estimation for a one-dimensional SDE. △ Less

Submitted 5 December, 2016; v1 submitted 6 September, 2016; originally announced September 2016.

Estimating a smooth function on a large graph by Bayesian Laplacian regularisation

Authors: Alisa Kirichenko, Harry van Zanten

Abstract: We study a Bayesian approach to estimating a smooth function in the context of regression or classification problems on large graphs. We derive theoretical results that show how asymptotically optimal Bayesian regularization can be achieved under an asymptotic shape assumption on the underlying graph and a smoothness condition on the target function, both formulated in terms of the graph Laplacian… ▽ More We study a Bayesian approach to estimating a smooth function in the context of regression or classification problems on large graphs. We derive theoretical results that show how asymptotically optimal Bayesian regularization can be achieved under an asymptotic shape assumption on the underlying graph and a smoothness condition on the target function, both formulated in terms of the graph Laplacian. The priors we study are randomly scaled Gaussians with precision operators involving the Laplacian of the graph. △ Less

Submitted 5 March, 2017; v1 submitted 8 November, 2015; originally announced November 2015.

Rejoinder to discussions of "Frequentist coverage of adaptive nonparametric Bayesian credible sets"

Authors: Botond Szabó, A. W. van der Vaart, J. H. van Zanten

Abstract: Rejoinder of "Frequentist coverage of adaptive nonparametric Bayesian credible sets" by Szabó, van der Vaart and van Zanten [arXiv:1310.4489v5]. Rejoinder of "Frequentist coverage of adaptive nonparametric Bayesian credible sets" by Szabó, van der Vaart and van Zanten [arXiv:1310.4489v5]. △ Less

Submitted 7 September, 2015; originally announced September 2015.

Comments: Published at http://dx.doi.org/10.1214/15-AOS1270REJ in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1270REJ

Journal ref: Annals of Statistics 2015, Vol. 43, No. 4, 1463-1470

Gaussian process methods for one-dimensional diffusions: optimal rates and adaptation

Authors: Jan van Waaij, Harry van Zanten

Abstract: We study the performance of nonparametric Bayes procedures for one-dimensional diffusions with periodic drift. We improve existing convergence rate results for Gaussian process (GP) priors with fixed hyper parameters. Moreover, we exhibit several possibilities to achieve adaptation to smoothness. We achieve this by considering hierarchical procedures that involve either a prior on a multiplicative… ▽ More We study the performance of nonparametric Bayes procedures for one-dimensional diffusions with periodic drift. We improve existing convergence rate results for Gaussian process (GP) priors with fixed hyper parameters. Moreover, we exhibit several possibilities to achieve adaptation to smoothness. We achieve this by considering hierarchical procedures that involve either a prior on a multiplicative scaling parameter, or a prior on the regularity parameter of the GP. △ Less

Submitted 8 February, 2016; v1 submitted 1 June, 2015; originally announced June 2015.

Optimality of Poisson processes intensity learning with Gaussian processes

Authors: Alisa Kirichenko, Harry van Zanten

Abstract: In this paper we provide theoretical support for the so-called "Sigmoidal Gaussian Cox Process" approach to learning the intensity of an inhomogeneous Poisson process on a $d$-dimensional domain. This method was proposed by Adams, Murray and MacKay (ICML, 2009), who developed a tractable computational approach and showed in simulation and real data experiments that it can work quite satisfactorily… ▽ More In this paper we provide theoretical support for the so-called "Sigmoidal Gaussian Cox Process" approach to learning the intensity of an inhomogeneous Poisson process on a $d$-dimensional domain. This method was proposed by Adams, Murray and MacKay (ICML, 2009), who developed a tractable computational approach and showed in simulation and real data experiments that it can work quite satisfactorily. The results presented in the present paper provide theoretical underpinning of the method. In particular, we show how to tune the priors on the hyper parameters of the model in order for the procedure to automatically adapt to the degree of smoothness of the unknown intensity and to achieve optimal convergence rates. △ Less

Submitted 2 March, 2015; v1 submitted 17 September, 2014; originally announced September 2014.

Honest Bayesian confidence sets for the L2-norm

Authors: Botond Szabo, Aad van der Vaart, Harry van Zanten

Abstract: We investigate the problem of constructing Bayesian credible sets that are honest and adaptive for the L2-loss over a scale of Sobolev classes with regularity ranging between [D; 2D], for some given D in the context of the signal-in-white-noise model. We consider a scale of prior distributions indexed by a regularity hyper-parameter and choose the hyper-parameter both by marginal likelihood empiri… ▽ More We investigate the problem of constructing Bayesian credible sets that are honest and adaptive for the L2-loss over a scale of Sobolev classes with regularity ranging between [D; 2D], for some given D in the context of the signal-in-white-noise model. We consider a scale of prior distributions indexed by a regularity hyper-parameter and choose the hyper-parameter both by marginal likelihood empirical Bayes and by hierarchical Bayes method, respectively. Next we consider a ball centered around the corresponding posterior mean with prescribed posterior probability. We show by theory and examples that both the empirical Bayes and the hierarchical Bayes credible sets give misleading, overconfident uncertainty quantification for certain oddly behaving truth. Then we construct a new empirical Bayes method based on risk estimation, which provides the correct uncertainty quantification and optimal size. △ Less

Submitted 23 April, 2014; v1 submitted 29 November, 2013; originally announced November 2013.

Comments: 24 pages, 3 figures

MSC Class: 62G15; 62G05 (Primary); 62G20 (secondary )

Guided proposals for simulating multi-dimensional diffusion bridges

Authors: Moritz Schauer, Frank van der Meulen, Harry van Zanten

Abstract: A Monte Carlo method for simulating a multi-dimensional diffusion process conditioned on hitting a fixed point at a fixed future time is developed. Proposals for such diffusion bridges are obtained by superimposing an additional guiding term to the drift of the process under consideration. The guiding term is derived via approximation of the target process by a simpler diffusion processes with kno… ▽ More A Monte Carlo method for simulating a multi-dimensional diffusion process conditioned on hitting a fixed point at a fixed future time is developed. Proposals for such diffusion bridges are obtained by superimposing an additional guiding term to the drift of the process under consideration. The guiding term is derived via approximation of the target process by a simpler diffusion processes with known transition densities. Acceptance of a proposal can be determined by computing the likelihood ratio between the proposal and the target bridge, which is derived in closed form. We show under general conditions that the likelihood ratio is well defined and show that a class of proposals with guiding term obtained from linear approximations fall under these conditions. △ Less

Submitted 13 October, 2015; v1 submitted 14 November, 2013; originally announced November 2013.

MSC Class: 60J60 (Primary); 65C30 (Secondary); 65C05

Journal ref: Bernoulli Volume 23, Number 4A (November 2017), 2917-2950

Frequentist coverage of adaptive nonparametric Bayesian credible sets

Authors: Botond Szabó, A. W. van der Vaart, J. H. van Zanten

Abstract: We investigate the frequentist coverage of Bayesian credible sets in a nonparametric setting. We consider a scale of priors of varying regularity and choose the regularity by an empirical Bayes method. Next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity. We show that such an adaptive Bayes credible set gives correct uncertainty… ▽ More We investigate the frequentist coverage of Bayesian credible sets in a nonparametric setting. We consider a scale of priors of varying regularity and choose the regularity by an empirical Bayes method. Next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity. We show that such an adaptive Bayes credible set gives correct uncertainty quantification of "polished tail" parameters, in the sense of high probability of coverage of such parameters. On the negative side, we show by theory and example that adaptation of the prior necessarily leads to gross and haphazard uncertainty quantification for some true parameters that are still within the hyperrectangle regularity scale. △ Less

Submitted 4 September, 2015; v1 submitted 16 October, 2013; originally announced October 2013.

Comments: Published at http://dx.doi.org/10.1214/14-AOS1270 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1270

Journal ref: Annals of Statistics 2015, Vol. 43, No. 4, 1391-1428

Rate-optimal Bayesian intensity smoothing for inhomogeneous Poisson processes

Authors: Eduard Belitser, Paulo Serra, Harry van Zanten

Abstract: We apply nonparametric Bayesian methods to study the problem of estimating the intensity function of an inhomogeneous Poisson process. We exhibit a prior on intensities which both leads to a computationally feasible method and enjoys desirable theoretical optimality properties. The prior we use is based on B-spline expansions with free knots, adapted from well-established methods used in regressio… ▽ More We apply nonparametric Bayesian methods to study the problem of estimating the intensity function of an inhomogeneous Poisson process. We exhibit a prior on intensities which both leads to a computationally feasible method and enjoys desirable theoretical optimality properties. The prior we use is based on B-spline expansions with free knots, adapted from well-established methods used in regression, for instance. We illustrate its practical use in the Poisson process setting by analyzing count data coming from a call centre. Theoretically we derive a new general theorem on contraction rates for posteriors in the setting of intensity function estimation. Practical choices that have to be made in the construction of our concrete prior, such as choosing the priors on the number and the locations of the spline knots, are based on these theoretical findings. The results assert that when properly constructed, our approach yields a rate-optimal procedure that automatically adapts to the regularity of the unknown intensity function. △ Less

Submitted 27 November, 2013; v1 submitted 22 April, 2013; originally announced April 2013.

Optimal two-stage procedures for estimating location and size of the maximum of a multivariate regression function

Authors: Eduard Belitser, Subhashis Ghosal, Harry van Zanten

Abstract: We propose a two-stage procedure for estimating the location $\boldsμ$ and size M of the maximum of a smooth d-variate regression function f. In the first stage, a preliminary estimator of $\boldsμ$ obtained from a standard nonparametric smoothing method is used. At the second stage, we "zoom-in" near the vicinity of the preliminary estimator and make further observations at some design points in… ▽ More We propose a two-stage procedure for estimating the location $\boldsμ$ and size M of the maximum of a smooth d-variate regression function f. In the first stage, a preliminary estimator of $\boldsμ$ obtained from a standard nonparametric smoothing method is used. At the second stage, we "zoom-in" near the vicinity of the preliminary estimator and make further observations at some design points in that vicinity. We fit an appropriate polynomial regression model to estimate the location and size of the maximum. We establish that, under suitable smoothness conditions and appropriate choice of the zooming, the second stage estimators have better convergence rates than the corresponding first stage estimators of $\boldsμ$ and M. More specifically, for $α$-smooth regression functions, the optimal nonparametric rates $n^{-(α-1)/(2α+d)}$ and $n^{-α/(2α+d)}$ at the first stage can be improved to $n^{-(α-1)/(2α)}$ and $n^{-1/2}$, respectively, for $α>1+\sqrt{1+d/2}$. These rates are optimal in the class of all possible sequential estimators. Interestingly, the two-stage procedure resolves "the curse of the dimensionality" problem to some extent, as the dimension d does not control the second stage convergence rates, provided that the function class is sufficiently smooth. We consider a multi-stage generalization of our procedure that attains the optimal rate for any smoothness level $α>2$ starting with a preliminary estimator with any power-law rate at the first stage. △ Less

Submitted 19 February, 2013; originally announced February 2013.

Comments: Published in at http://dx.doi.org/10.1214/12-AOS1053 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1053

Journal ref: Annals of Statistics 2012, Vol. 40, No. 6, 2850-2876

Consistent nonparametric Bayesian inference for discretely observed scalar diffusions

Authors: Frank van der Meulen, Harry van Zanten

Abstract: We study Bayes procedures for the problem of nonparametric drift estimation for one-dimensional, ergodic diffusion models from discrete-time, low-frequency data. We give conditions for posterior consistency and verify these conditions for concrete priors, including priors based on wavelet expansions. We study Bayes procedures for the problem of nonparametric drift estimation for one-dimensional, ergodic diffusion models from discrete-time, low-frequency data. We give conditions for posterior consistency and verify these conditions for concrete priors, including priors based on wavelet expansions. △ Less

Submitted 31 January, 2013; originally announced January 2013.

Comments: Published in at http://dx.doi.org/10.3150/11-BEJ385 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

Report number: IMS-BEJ-BEJ385

Journal ref: Bernoulli 2013, Vol. 19, No. 1, 44-63

Adaptive nonparametric Bayesian inference using location-scale mixture priors

Authors: R. de Jonge, J. H. van Zanten

Abstract: We study location-scale mixture priors for nonparametric statistical problems, including multivariate regression, density estimation and classification. We show that a rate-adaptive procedure can be obtained if the prior is properly constructed. In particular, we show that adaptation is achieved if a kernel mixture prior on a regression function is constructed using a Gaussian kernel, an inverse g… ▽ More We study location-scale mixture priors for nonparametric statistical problems, including multivariate regression, density estimation and classification. We show that a rate-adaptive procedure can be obtained if the prior is properly constructed. In particular, we show that adaptation is achieved if a kernel mixture prior on a regression function is constructed using a Gaussian kernel, an inverse gamma bandwidth, and Gaussian mixing weights. △ Less

Submitted 9 November, 2012; originally announced November 2012.

Comments: Published in at http://dx.doi.org/10.1214/10-AOS811 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS811

Journal ref: Annals of Statistics 2010, Vol. 38, No. 6, 3300-3320

Bayes procedures for adaptive inference in inverse problems for the white noise model

Authors: B. T. Knapik, B. T. Szabó, A. W. van der Vaart, J. H. van Zanten

Abstract: We study empirical and hierarchical Bayes approaches to the problem of estimating an infinite-dimensional parameter in mildly ill-posed inverse problems. We consider a class of prior distributions indexed by a hyperparameter that quantifies regularity. We prove that both methods we consider succeed in automatically selecting this parameter optimally, resulting in optimal convergence rates for trut… ▽ More We study empirical and hierarchical Bayes approaches to the problem of estimating an infinite-dimensional parameter in mildly ill-posed inverse problems. We consider a class of prior distributions indexed by a hyperparameter that quantifies regularity. We prove that both methods we consider succeed in automatically selecting this parameter optimally, resulting in optimal convergence rates for truths with Sobolev or analytic "smoothness", without using knowledge about this regularity. Both methods are illustrated by simulation examples. △ Less

Submitted 29 May, 2013; v1 submitted 17 September, 2012; originally announced September 2012.

Comments: 41 pages, 2 figures

MSC Class: 62G05; 62C15 (Primary) 62G20 (Secondary)

Bayesian recovery of the initial condition for the heat equation

Authors: B. T. Knapik, A. W. van der Vaart, J. H. van Zanten

Abstract: We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying "smoothness" and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and… ▽ More We study a Bayesian approach to recovering the initial condition for the heat equation from noisy observations of the solution at a later time. We consider a class of prior distributions indexed by a parameter quantifying "smoothness" and show that the corresponding posterior distributions contract around the true parameter at a rate that depends on the smoothness of the true initial condition and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the optimal minimax rate. One type of priors leads to a rate-adaptive Bayesian procedure. The frequentist coverage of credible sets is shown to depend on the combination of the prior and true parameter as well, with smoother priors leading to zero coverage and rougher priors to (extremely) conservative results. In the latter case credible sets are much larger than frequentist confidence sets, in that the ratio of diameters diverges to infinity. The results are numerically illustrated by a simulated data example. △ Less

Submitted 1 March, 2013; v1 submitted 24 November, 2011; originally announced November 2011.

Comments: 17 pages, 4 figures. Published in Comm. Statist. Theory Methods. This version differs from the original in pagination and typographic detail. arXiv admin note: text overlap with arXiv:1103.2692

MSC Class: 62G05; 62G15; 62G20

Journal ref: Communications in Statistics - Theory and Methods 2013 Volume 42, Issue 7, 1294-1313

Bayesian inverse problems with Gaussian priors

Authors: B. T. Knapik, A. W. van der Vaart, J. H. van Zanten

Abstract: The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoothe… ▽ More The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive. △ Less

Submitted 23 February, 2012; v1 submitted 14 March, 2011; originally announced March 2011.

Comments: Published in at http://dx.doi.org/10.1214/11-AOS920 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS920

Journal ref: Annals of Statistics 2011, Vol. 39, No. 5, 2626-2657

Nonparametric methods for volatility density estimation

Authors: Bert van Es, Peter Spreij, Harry van Zanten

Abstract: Stochastic volatility modelling of financial processes has become increasingly popular. The proposed models usually contain a stationary volatility process. We will motivate and review several nonparametric methods for estimation of the density of the volatility process. Both models based on discretely sampled continuous time processes and discrete time models will be discussed. The key insigh… ▽ More Stochastic volatility modelling of financial processes has become increasingly popular. The proposed models usually contain a stationary volatility process. We will motivate and review several nonparametric methods for estimation of the density of the volatility process. Both models based on discretely sampled continuous time processes and discrete time models will be discussed. The key insight for the analysis is a transformation of the volatility density estimation problem to a deconvolution model for which standard methods exist. Three type of nonparametric density estimators are reviewed: the Fourier-type deconvolution kernel density estimator, a wavelet deconvolution density estimator and a penalized projection estimator. The performance of these estimators will be compared. Key words: stochastic volatility models, deconvolution, density estimation, kernel estimator, wavelets, minimum contrast estimation, mixing △ Less

Submitted 27 October, 2009; originally announced October 2009.

MSC Class: 62G07; 62G08; 62M07; 62P20; 91G70

Journal ref: Advanced Mathematical Methods for Finance, Chapter 11, 293-312, Giulia di Nunno, Bernt Øksendal Eds., Springer (2011)

Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth

Authors: A. W. van der Vaart, J. H. van Zanten

Abstract: We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimatio… ▽ More We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive. △ Less

Submitted 25 August, 2009; originally announced August 2009.

Comments: Published in at http://dx.doi.org/10.1214/08-AOS678 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS678 MSC Class: 62H30; 62-07 (Primary); 65U05; 68T05 (Secondary)

Journal ref: Annals of Statistics 2009, Vol. 37, No. 5B, 2655-2675

Rates of contraction of posterior distributions based on Gaussian process priors

Authors: A. W. van der Vaart, J. H. van Zanten

Abstract: We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of… ▽ More We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors. △ Less

Submitted 18 June, 2008; originally announced June 2008.

Comments: Published in at http://dx.doi.org/10.1214/009053607000000613 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS0301 MSC Class: 60G15; 62G05 (Primary)

Journal ref: Annals of Statistics 2008, Vol. 36, No. 3, 1435-1463

Reproducing kernel Hilbert spaces of Gaussian priors

Authors: A. W. van der Vaart, J. H. van Zanten

Abstract: We review definitions and properties of reproducing kernel Hilbert spaces attached to Gaussian variables and processes, with a view to applications in nonparametric Bayesian statistics using Gaussian priors. The rate of contraction of posterior distributions based on Gaussian priors can be described through a concentration function that is expressed in the reproducing Hilbert space. Absolute con… ▽ More We review definitions and properties of reproducing kernel Hilbert spaces attached to Gaussian variables and processes, with a view to applications in nonparametric Bayesian statistics using Gaussian priors. The rate of contraction of posterior distributions based on Gaussian priors can be described through a concentration function that is expressed in the reproducing Hilbert space. Absolute continuity of Gaussian measures and concentration inequalities play an important role in understanding and deriving this result. Series expansions of Gaussian variables and transformations of their reproducing kernel Hilbert spaces under linear maps are useful tools to compute the concentration function. △ Less

Submitted 21 May, 2008; originally announced May 2008.

Comments: Published in at http://dx.doi.org/10.1214/074921708000000156 the IMS Collections (http://www.imstat.org/publications/imscollections.htm) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-COLL3-IMSCOLL315 MSC Class: 60G15; 62G05 (Primary)

Journal ref: IMS Collections 2008, Vol. 3, 200-222

Small Deviations of Smooth Stationary Gaussian Processes

Authors: F. Aurzada, I. A. Ibragimov, M. A. Lifshits, J. H. van Zanten

Abstract: We investigate the small deviation probabilities of a class of very smooth stationary Gaussian processes playing an important role in Bayesian statistical inference. Our calculations are based on the appropriate modification of the entropy method due to Kuelbs, Li, and Linde as well as on classical results about the entropy of classes of analytic functions. They also involve Tsirelson's upper bo… ▽ More We investigate the small deviation probabilities of a class of very smooth stationary Gaussian processes playing an important role in Bayesian statistical inference. Our calculations are based on the appropriate modification of the entropy method due to Kuelbs, Li, and Linde as well as on classical results about the entropy of classes of analytic functions. They also involve Tsirelson's upper bound for small deviations and shed some light on the limits of sharpness for that estimate. △ Less

Submitted 29 March, 2008; originally announced March 2008.

MSC Class: 60G10; 60G15; 46E15; 62A15

Journal ref: Theor. Probab. Appl., 2008, 53, 4, 788--798 (in Russian), 697--707 (in English)

Bayesian inference with rescaled Gaussian process priors

Authors: Aad van der Vaart, Harry van Zanten

Abstract: We use rescaled Gaussian processes as prior models for functional parameters in nonparametric statistical models. We show how the rate of contraction of the posterior distributions depends on the scaling factor. In particular, we exhibit rescaled Gaussian process priors yielding posteriors that contract around the true parameter at optimal convergence rates. To derive our results we establish bo… ▽ More We use rescaled Gaussian processes as prior models for functional parameters in nonparametric statistical models. We show how the rate of contraction of the posterior distributions depends on the scaling factor. In particular, we exhibit rescaled Gaussian process priors yielding posteriors that contract around the true parameter at optimal convergence rates. To derive our results we establish bounds on small deviation probabilities for smooth stationary Gaussian processes. △ Less

Submitted 19 October, 2007; originally announced October 2007.

Comments: Published in at http://dx.doi.org/10.1214/07-EJS098 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-EJS-EJS_2007_98 MSC Class: 62G05; 62C10 (Primary) 60G15 (Secondary)

Journal ref: Electronic Journal of Statistics 2007, Vol. 1, 433-448

Donsker theorems for diffusions: Necessary and sufficient conditions

Authors: Aad van der Vaart, Harry van Zanten

Abstract: We consider the empirical process G_t of a one-dimensional diffusion with finite speed measure, indexed by a collection of functions F. By the central limit theorem for diffusions, the finite-dimensional distributions of G_t converge weakly to those of a zero-mean Gaussian random process G. We prove that the weak convergence G_t\Rightarrow G takes place in \ell^{\infty}(F) if and only if the lim… ▽ More We consider the empirical process G_t of a one-dimensional diffusion with finite speed measure, indexed by a collection of functions F. By the central limit theorem for diffusions, the finite-dimensional distributions of G_t converge weakly to those of a zero-mean Gaussian random process G. We prove that the weak convergence G_t\Rightarrow G takes place in \ell^{\infty}(F) if and only if the limit G exists as a tight, Borel measurable map. The proof relies on majorizing measure techniques for continuous martingales. Applications include the weak convergence of the local time density estimator and the empirical distribution function on the full state space. △ Less

Submitted 21 July, 2005; originally announced July 2005.

Comments: Published at http://dx.doi.org/10.1214/009117905000000152 in the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOP-AOP0059 MSC Class: 60J60; 60J55; 60F17; 62M05 (Primary)

Journal ref: Annals of Probability 2005, Vol. 33, No. 4, 1422-1451

Krein's spectral theory and the Paley-Wiener expansion for fractional Brownian motion

Authors: Kacha Dzhaparidze, Harry van Zanten

Abstract: In this paper we develop the spectral theory of the fractional Brownian motion (fBm) using the ideas of Krein's work on continuous analogous of orthogonal polynomials on the unit circle. We exhibit the functions which are orthogonal with respect to the spectral measure of the fBm and obtain an explicit reproducing kernel in the frequency domain. We use these results to derive an extension of the… ▽ More In this paper we develop the spectral theory of the fractional Brownian motion (fBm) using the ideas of Krein's work on continuous analogous of orthogonal polynomials on the unit circle. We exhibit the functions which are orthogonal with respect to the spectral measure of the fBm and obtain an explicit reproducing kernel in the frequency domain. We use these results to derive an extension of the classical Paley-Wiener expansion of the ordinary Brownian motion to the fractional case. △ Less

Submitted 29 March, 2005; originally announced March 2005.

Comments: Published at http://dx.doi.org/10.1214/009117904000000955 in the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOP-AOP039 MSC Class: 60G15; 60G51; 62M15 (Primary)

Journal ref: Annals of Probability 2005, Vol. 33, No. 2, 620-644

Nonparametric volatility density estimation for discrete time models

Authors: Bert van Es, Peter Spreij, Harry van Zanten

Abstract: We consider discrete time models for asset prices with a stationary volatility process. We aim at estimating the multivariate density of this process at a set of consecutive time instants. A Fourier type deconvolution kernel density estimator based on the logarithm of the squared process is proposed to estimate the volatility density. Expansions of the bias and bounds on the variance are derived… ▽ More We consider discrete time models for asset prices with a stationary volatility process. We aim at estimating the multivariate density of this process at a set of consecutive time instants. A Fourier type deconvolution kernel density estimator based on the logarithm of the squared process is proposed to estimate the volatility density. Expansions of the bias and bounds on the variance are derived. △ Less

Submitted 14 June, 2002; originally announced June 2002.

MSC Class: 62G07; 62M07; 62P20

Journal ref: Journal of Nonparametric Statistics 17 (2), 237-249 (2005)

Nonparametric Volatility Density Estimation

Authors: Bert van Es, Peter Spreij, Harry van Zanten

Abstract: We consider two kinds of stochastic volatility models. Both kinds of models contain a stationary volatility process, the density of which, at a fixed instant in time, we aim to estimate. We discuss discrete time models where for instance a log price process is modeled as the product of a volatility process and i.i.d. noise. We also consider samples of certain continuous time diffusion processe… ▽ More We consider two kinds of stochastic volatility models. Both kinds of models contain a stationary volatility process, the density of which, at a fixed instant in time, we aim to estimate. We discuss discrete time models where for instance a log price process is modeled as the product of a volatility process and i.i.d. noise. We also consider samples of certain continuous time diffusion processes. The sampled time instants will be be equidistant with vanishing distance. A Fourier type deconvolution kernel density estimator based on the logarithm of the squared processes is proposed to estimate the volatility density. Expansions of the bias and bounds on the variances are derived. △ Less

Submitted 16 June, 2002; v1 submitted 19 July, 2001; originally announced July 2001.

MSC Class: 62G07; 62M07; 62P20

Journal ref: Bernoulli 9 (3), 451-645 (2003)