Abstract

By providing a simple and efficient way of computing low-variance gradientsof continuous random variables, the reparameterization trick has become thetechnique of choice for training a variety of latent variable models. However,it is not applicable to a number of important continuous distributions. Weintroduce an alternative approach to computing reparameterization gradientsbased on implicit differentiation and demonstrate its broader applicability byapplying it to Gamma, Beta, Dirichlet, and von Mises distributions, whichcannot be used with the classic reparameterization trick. Our experiments showthat the proposed approach is faster and more accurate than the existinggradient estimators for these distributions.

 

Quick Read (beta)