Abstract
Expectation propagation (EP) is a family of algorithms for performingapproximate inference in probabilistic models. The updates of EP involve theevaluation of moments -- expectations of certain functions -- which can beestimated from Monte Carlo (MC) samples. However, the updates are not robust toMC noise when performed naively, and various prior works have attempted toaddress this issue in different ways. In this work, we provide a novelperspective on the moment-matching updates of EP; namely, that they performnatural-gradient-based optimisation of a variational objective. We use thisinsight to motivate two new EP variants, with updates that are particularlywell-suited to MC estimation. They remain stable and are most sample-efficientwhen estimated with just a single sample. These new variants combine thebenefits of their predecessors and address key weaknesses. In particular, theyare easier to tune, offer an improved speed-accuracy trade-off, and do not relyon the use of debiasing estimators. We demonstrate their efficacy on a varietyof probabilistic inference tasks.