Hamiltonian Monte Carlo with Energy Conserving Subsampling

by   Khue-Dung Dang, et al.

Hamiltonian Monte Carlo (HMC) has recently received considerable attention in the literature due to its ability to overcome the slow exploration of the parameter space inherent in random walk proposals. In tandem, data subsampling has been extensively used to overcome the computational bottlenecks in posterior sampling algorithms that require evaluating the likelihood over the whole data set, or its gradient. However, while data subsampling has been successful in traditional MCMC algorithms such as Metropolis-Hastings, it has been demonstrated to be unsuccessful in the context of HMC, both in terms of poor sampling efficiency and in producing highly biased inferences. We propose an efficient HMC-within-Gibbs algorithm that utilizes data subsampling to speed up computations and simulates from a slightly perturbed target, which is within O(m^-2) of the true target, where m is the size of the subsample. We also show how to modify the method to obtain exact inference on any function of the parameters. Contrary to previous unsuccessful approaches, we perform subsampling in a way that conserves energy but for a modified Hamiltonian. We can therefore maintain high acceptance rates even for distant proposals. We apply the method for simulating from the posterior distribution of a high-dimensional spline model for bankruptcy data and document speed ups of several orders of magnitude compare to standard HMC and, moreover, demonstrate a negligible bias.


page 1

page 2

page 3

page 4


Hamiltonian Monte Carlo Swindles

Hamiltonian Monte Carlo (HMC) is a powerful Markov chain Monte Carlo (MC...

Variational Langevin Hamiltonian Monte Carlo for Distant Multi-modal Sampling

The Hamiltonian Monte Carlo (HMC) sampling algorithm exploits Hamiltonia...

Constrained Bayesian Nonparametric Regression for Grain Boundary Energy Predictions

Grain boundary (GB) energy is a fundamental property that affects the fo...

Energy-preserving splitting integrators for sampling from Gaussian distributions with Hamiltonian Monte Carlo method

The diffusive behaviour of simple random-walk proposals of many Markov C...

Stochastic Gradient Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for d...

Reconstructing the Universe with Variational self-Boosted Sampling

Forward modeling approaches in cosmology have made it possible to recons...

Deep surrogate accelerated delayed-acceptance HMC for Bayesian inference of spatio-temporal heat fluxes in rotating disc systems

We study the Bayesian inverse problem of inferring the Biot number, a sp...

Please sign up or login with your details

Forgot password? Click here to reset