Tight Dimension Dependence of the Laplace Approximation
share
In Bayesian inference, a widespread technique to approximately sample from and compute statistics of a high-dimensional posterior is to use the Laplace approximation, a Gaussian proxy to the posterior. The Laplace approximation accuracy improves as sample size grows, but the question of how fast dimension d can grow with sample size n has not been fully resolved. Prior works have shown that d^3≪ n is a sufficient condition for accuracy of the approximation. But by deriving the leading order contribution to the TV error, we show that d^2≪ n is sufficient. We show for a logistic regression posterior that this growth condition is necessary.
READ FULL TEXT