The latent spaces of typical GAN models often have semantically meaningfuldirections. Moving in these directions corresponds to human-interpretable imagetransformations, such as zooming or recoloring, enabling a more controllablegeneration process. However, the discovery of such directions is currentlyperformed in a supervised manner, requiring human labels, pretrained models, orsome form of self-supervision. These requirements can severely limit a range ofdirections existing approaches can discover. In this paper, we introduce anunsupervised method to identify interpretable directions in the latent space ofa pretrained GAN model. By a simple model-agnostic procedure, we finddirections corresponding to sensible semantic manipulations without any form of(self-)supervision. Furthermore, we reveal several non-trivial findings, whichwould be difficult to obtain by existing methods, e.g., a directioncorresponding to background removal. As an immediate practical benefit of ourwork, we show how to exploit this finding to achieve a new state-of-the-art forthe problem of saliency detection.