Abstract
Laplace's method approximates a target density with a Gaussian distributionat its mode. It is computationally efficient and asymptotically exact forBayesian inference due to the Bernstein-von Mises theorem, but for complextargets and finite-data posteriors it is often too crude an approximation. Arecent generalization of the Laplace Approximation transforms the Gaussianapproximation according to a chosen Riemannian geometry providing a richerapproximation family, while still retaining computational efficiency. However,as shown here, its properties depend heavily on the chosen metric, indeed themetric adopted in previous work results in approximations that are overlynarrow as well as being biased even at the limit of infinite data. We correctthis shortcoming by developing the approximation family further, deriving twoalternative variants that are exact at the limit of infinite data, extendingthe theoretical analysis of the method, and demonstrating practicalimprovements in a range of experiments.