Asymptotic Consistency of α-Rényi-Approximate Posteriors
share
In this work, we study consistency properties of α-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the latter chosen to minimize the α-Rényi divergence from the true posterior. Unique to our work is that we consider settings with α > 1, resulting in approximations that upperbound the log-likelihood, and result in approximations with a wider spread than traditional variational approaches that minimize the Kullback-Liebler divergence from the posterior. We provide sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family. We discuss examples where this holds and show how the existence of such a good sequence implies posterior consistency in the limit of an infinite number of observations.
READ FULL TEXT