Abstract
Variational inference (VI) can be cast as an optimization problem in whichthe variational parameters are tuned to closely align a variationaldistribution with the true posterior. The optimization task can be approachedthrough vanilla gradient descent in black-box VI or natural-gradient descent innatural-gradient VI. In this work, we reframe VI as the optimization of anobjective that concerns probability distributions defined over a\textit{variational parameter space}. Subsequently, we propose Wassersteingradient descent for tackling this optimization problem. Notably, theoptimization techniques, namely black-box VI and natural-gradient VI, can bereinterpreted as specific instances of the proposed Wasserstein gradientdescent. To enhance the efficiency of optimization, we develop practicalmethods for numerically solving the discrete gradient flows. We validate theeffectiveness of the proposed methods through empirical experiments on asynthetic dataset, supplemented by theoretical analyses.