On the Tightness of the Laplace Approximation for Statistical Inference
share
Laplace's method is used to approximate intractable integrals in a wide range of statistical problems, including Bayesian inference and frequentist marginal likelihood models. It is classically known that the relative error rate of the approximation is not worse than O_p(n^-1) under standard regularity conditions, where n is the sample size. It is unknown whether the error rate can be better than O_p(n^-1) in common applications. We provide the first statistical lower bounds showing that the n^-1 rate is tight. We prove stochastic lower bounds for two simple models: Bayesian inference on fair coin flips, and frequentist marginal likelihood estimation for an over-dispersed Poisson model. We conclude that any set of assumptions under which a faster rate can be derived must be so restrictive as to exclude these simple models, and hence the n^-1 rate is, for practical purposes, the best that can be obtained.
READ FULL TEXT