Abstract
Deep learning's success has led to larger and larger models to handle moreand more complex tasks; trained models can contain millions of parameters.These large models are compute- and memory-intensive, which makes it achallenge to deploy them with minimized latency, throughput, and storagerequirements. Some model compression methods have been successfully applied toimage classification and detection or language models, but there has been verylittle work compressing generative adversarial networks (GANs) performingcomplex tasks. In this paper, we show that a standard model compressiontechnique, weight pruning, cannot be applied to GANs using existing methods. Wethen develop a self-supervised compression technique which uses the traineddiscriminator to supervise the training of a compressed generator. We show thatthis framework has a compelling performance to high degrees of sparsity, can beeasily applied to new tasks and models, and enables meaningful comparisonsbetween different pruning granularities.