Abstract
A large body of recent work has identified transformations in the latentspaces of generative adversarial networks (GANs) that consistently andinterpretably transform generated images. But existing techniques foridentifying these transformations rely on either a fixed vocabulary ofpre-specified visual concepts, or on unsupervised disentanglement techniqueswhose alignment with human judgments about perceptual salience is unknown. Thispaper introduces a new method for building open-ended vocabularies of primitivevisual concepts represented in a GAN's latent space. Our approach is built fromthree components: (1) automatic identification of perceptually salientdirections based on their layer selectivity; (2) human annotation of thesedirections with free-form, compositional natural language descriptions; and (3)decomposition of these annotations into a visual concept vocabulary, consistingof distilled directions labeled with single words. Experiments show thatconcepts learned with our approach are reliable and composable -- generalizingacross classes, contexts, and observers, and enabling fine-grained manipulationof image style and content.