The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions

by   Raj Agrawal, et al.

Discovering interaction effects on a response of interest is a fundamental problem faced in biology, medicine, economics, and many other scientific disciplines. In theory, Bayesian methods for discovering pairwise interactions enjoy many benefits such as coherent uncertainty quantification, the ability to incorporate background knowledge, and desirable shrinkage properties. In practice, however, Bayesian methods are often computationally intractable for even moderate-dimensional problems. Our key insight is that many hierarchical models of practical interest admit a particular Gaussian process (GP) representation; the GP allows us to capture the posterior with a vector of O(p) kernel hyper-parameters rather than O(p^2) interactions and main effects. With the implicit representation, we can run Markov chain Monte Carlo (MCMC) over model hyper-parameters in time and memory linear in p per iteration. We focus on sparsity-inducing models and show on datasets with a variety of covariate behaviors that our method: (1) reduces runtime by orders of magnitude over naive applications of MCMC, (2) provides lower Type I and Type II error relative to state-of-the-art LASSO-based approaches, and (3) offers improved computational scaling in high dimensions relative to existing Bayesian and LASSO-based approaches.


page 1

page 2

page 3

page 4


Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes

We study preferential Bayesian optimization (BO) where reliable feedback...

Evaluating the accuracy of Gaussian approximations in VSWIR imaging spectroscopy retrievals

The joint retrieval of surface reflectances and atmospheric parameters i...

Rethinking Sparse Gaussian Processes: Bayesian Approaches to Inducing-Variable Approximations

Variational inference techniques based on inducing variables provide an ...

Fast Markov chain Monte Carlo for high dimensional Bayesian regression models with shrinkage priors

In the past decade, many Bayesian shrinkage models have been developed f...

The SKIM-FA Kernel: High-Dimensional Variable Selection and Nonlinear Interaction Discovery in Linear Time

Many scientific problems require identifying a small set of covariates t...

Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization

Bayesian optimization (BO) primarily uses Gaussian processes (GP) as the...

Bayesian Analysis for miRNA and mRNA Interactions Using Expression Data

MicroRNAs (miRNAs) are small RNA molecules composed of 19-22 nt, which p...

Please sign up or login with your details

Forgot password? Click here to reset