Frequentist coverage and sup-norm convergence rate in Gaussian process regression

by   Yun Yang, et al.

Gaussian process (GP) regression is a powerful interpolation technique due to its flexibility in capturing non-linearity. In this paper, we provide a general framework for understanding the frequentist coverage of point-wise and simultaneous Bayesian credible sets in GP regression. As an intermediate result, we develop a Bernstein von-Mises type result under supremum norm in random design GP regression. Identifying both the mean and covariance function of the posterior distribution of the Gaussian process as regularized M-estimators, we show that the sampling distribution of the posterior mean function and the centered posterior distribution can be respectively approximated by two population level GPs. By developing a comparison inequality between two GPs, we provide exact characterization of frequentist coverage probabilities of Bayesian point-wise credible intervals and simultaneous credible bands of the regression function. Our results show that inference based on GP regression tends to be conservative; when the prior is under-smoothed, the resulting credible intervals and bands have minimax-optimal sizes, with their frequentist coverage converging to a non-degenerate value between their nominal level and one. As a byproduct of our theory, we show that the GP regression also yields minimax-optimal posterior contraction rate relative to the supremum norm, which provides a positive evidence to the long standing problem on optimal supremum norm contraction rate in GP regression.


page 19

page 20


Posterior contraction for deep Gaussian process priors

We study posterior contraction rates for a class of deep Gaussian proces...

Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision making

As a powerful Bayesian non-parameterized algorithm, the Gaussian process...

Hierarchical Gaussian Process Models for Regression Discontinuity/Kink under Sharp and Fuzzy Designs

We propose nonparametric Bayesian estimators for causal inference exploi...

Optimal plug-in Gaussian processes for modelling derivatives

Derivatives are a key nonparametric functional in wide-ranging applicati...

Coverage of Credible Intervals in Bayesian Multivariate Isotonic Regression

We consider the nonparametric multivariate isotonic regression problem, ...

Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria

Approximate Bayesian computation (ABC) can be used for model fitting whe...

Gaussian Processes with Errors in Variables: Theory and Computation

Covariate measurement error in nonparametric regression is a common prob...

Please sign up or login with your details

Forgot password? Click here to reset