Rapid mixing in unimodal landscapes and efficient simulatedannealing for multimodal distributions
share
We consider nearest neighbor weighted random walks on the d-dimensional box [n]^d that are governed by some function g:[0,1] [0,), by which we mean that standing at x, a neighbor y of x is picked at random and the walk then moves there with probability (1/2)g(n^-1y)/(g(n^-1y)+g(n^-1x)). We do this for g of the form f^m_n for some function f which assumed to be analytically well-behaved and where m_n as n. This class of walks covers an abundance of interesting special cases, e.g., the mean-field Potts model, posterior collapsed Gibbs sampling for Latent Dirichlet allocation and certain Bayesian posteriors for models in nuclear physics. The following are among the results of this paper: * If f is unimodal with negative definite Hessian at its global maximum, then the mixing time of the random walk is O(nlog n). * If f is multimodal, then the mixing time is exponential in n, but we show that there is a simulated annealing scheme governed by f^K for an increasing sequence of K that mixes in time O(n^2). Using a varying step size that decreases with K, this can be taken down to O(nlog n). * If the process is studied on a general graph rather than the d-dimensional box, a simulated annealing scheme expressed in terms of conductances of the underlying network, works similarly. Several examples are given, including the ones mentioned above.
READ FULL TEXT