Abstract
We study in this paper the rate of convergence for learning distributionswith the Generative Adversarial Networks (GAN) framework, which subsumesWasserstein, Sobolev and MMD GANs as special cases. We study a wide range ofparametric and nonparametric target distributions, under a collection ofobjective evaluation metrics. On the nonparametric end, we investigate theminimax optimal rates and fundamental difficulty of the density estimationunder the adversarial framework. On the parametric end, we establish theory forneural network classes, that characterizes the interplay between the choice ofgenerator and discriminator. We investigate how to improve the GAN frameworkwith better theoretical guarantee through the lens of regularization. Wediscover and isolate a new notion of regularization, called the\textit{generator/discriminator pair regularization}, that sheds light on theadvantage of GAN compared to classic parametric and nonparametric approachesfor density estimation.