Abstract
A key challenge for modern Bayesian statistics is how to perform scalableinference of posterior distributions. To address this challenge, variationalBayes (VB) methods have emerged as a popular alternative to the classicalMarkov chain Monte Carlo (MCMC) methods. VB methods tend to be faster whileachieving comparable predictive performance. However, there are few theoreticalresults around VB. In this paper, we establish frequentist consistency andasymptotic normality of VB methods. Specifically, we connect VB methods topoint estimates based on variational approximations, called frequentistvariational approximations, and we use the connection to prove a variationalBernstein-von Mises theorem. The theorem leverages the theoreticalcharacterizations of frequentist variational approximations to understandasymptotic properties of VB. In summary, we prove that (1) the VB posteriorconverges to the Kullback-Leibler (KL) minimizer of a normal distribution,centered at the truth and (2) the corresponding variational expectation of theparameter is consistent and asymptotically normal. As applications of thetheorem, we derive asymptotic properties of VB posteriors in Bayesian mixturemodels, Bayesian generalized linear mixed models, and Bayesian stochastic blockmodels. We conduct a simulation study to illustrate these theoretical results.