Abstract

Generative Adversarial Networks (GANs) have recently achieved impressiveresults for many real-world applications, and many GAN variants have emergedwith improvements in sample quality and training stability. However,visualization and understanding of GANs is largely missing. How does a GANrepresent our visual world internally? What causes the artifacts in GANresults? How do architectural choices affect GAN learning? Answering suchquestions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understandGANs at the unit-, object-, and scene-level. We first identify a group ofinterpretable units that are closely related to object concepts with asegmentation-based network dissection method. Then, we quantify the causaleffect of interpretable units by measuring the ability of interventions tocontrol objects in the output. Finally, we examine the contextual relationshipbetween these units and their surrounding by inserting the discovered objectconcepts into new images. We show several practical applications enabled by ourframework, from comparing internal representations across different layers,models, and datasets, to improving GANs by locating and removingartifact-causing units, to interactively manipulating objects in the scene. Weprovide open source interpretation tools to help peer researchers andpractitioners better understand their GAN models.

 

Introduction (beta)

None

 

Conclusion (beta)

None