Abstract
Seeking informative projecting directions has been an important task inutilizing sliced Wasserstein distance in applications. However, finding thesedirections usually requires an iterative optimization procedure over the spaceof projecting directions, which is computationally expensive. Moreover, thecomputational issue is even more severe in deep learning applications, wherecomputing the distance between two mini-batch probability measures is repeatedseveral times. This nested loop has been one of the main challenges thatprevent the usage of sliced Wasserstein distances based on good projections inpractice. To address this challenge, we propose to utilize thelearning-to-optimize technique or amortized optimization to predict theinformative direction of any given two mini-batch probability measures. To thebest of our knowledge, this is the first work that bridges amortizedoptimization and sliced Wasserstein generative models. In particular, we derivelinear amortized models, generalized linear amortized models, and non-linearamortized models which are corresponding to three types of novel mini-batchlosses, named amortized sliced Wasserstein. We demonstrate the favorableperformance of the proposed sliced losses in deep generative modeling onstandard benchmark datasets.