Improved Marginal Unbiased Score Expansion (MUSE) via Implicit Differentiation

  • 2022-09-21 18:20:20
  • Marius Millea
  • 0

Abstract

We apply the technique of implicit differentiation to boost performance,reduce numerical error, and remove required user-tuning in the MarginalUnbiased Score Expansion (MUSE) algorithm for hierarchical Bayesian inference.We demonstrate these improvements on three representative inference problems:1) an extended Neal's funnel 2) Bayesian neural networks, and 3) probabilisticprincipal component analysis. On our particular test cases, MUSE with implicitdifferentiation is faster than Hamiltonian Monte Carlo by factors of 155, 397,and 5, respectively, or factors of 65, 278, and 1 without implicitdifferentiation, and yields good approximate marginal posteriors. The Julia andPython MUSE packages have been updated to use implicit differentiation, and cansolve problems defined by hand or with any of a number of popular probabilisticprogramming languages and automatic differentiation backends.

 

Quick Read (beta)

loading the full paper ...