Abstract
Many probabilistic models of interest in scientific computing and machinelearning have expensive, black-box likelihoods that prevent the application ofstandard techniques for Bayesian inference, such as MCMC, which would requireaccess to the gradient or a large number of likelihood evaluations. Weintroduce here a novel sample-efficient inference framework, VariationalBayesian Monte Carlo (VBMC). VBMC combines variational inference withGaussian-process based, active-sampling Bayesian quadrature, using the latterto efficiently approximate the intractable integral in the variationalobjective. Our method produces both a nonparametric approximation of theposterior distribution and an approximate lower bound of the model evidence,useful for model selection. We demonstrate VBMC both on several syntheticlikelihoods and on a neuronal model with data from real neurons. Across alltested problems and dimensions (up to $D = 10$), VBMC performs consistentlywell in reconstructing the posterior and the model evidence with a limitedbudget of likelihood evaluations, unlike other methods that work only in verylow dimensions. Our framework shows great promise as a novel tool for posteriorand model inference with expensive, black-box likelihoods.