We propose a novel method for solving regression tasks using few-shot or weaksupervision. At the core of our method is the fundamental observation that GANsare incredibly successful at encoding semantic information within their latentspace, even in a completely unsupervised setting. For modern generativeframeworks, this semantic encoding manifests as smooth, linear directions whichaffect image attributes in a disentangled manner. These directions have beenwidely used in GAN-based image editing. We show that such directions are notonly linear, but that the magnitude of change induced on the respectiveattribute is approximately linear with respect to the distance traveled alongthem. By leveraging this observation, our method turns a pre-trained GAN into aregression model, using as few as two labeled samples. This enables solvingregression tasks on datasets and attributes which are difficult to producequality supervision for. Additionally, we show that the same latent-distancescan be used to sort collections of images by the strength of given attributes,even in the absence of explicit supervision. Extensive experimental evaluationsdemonstrate that our method can be applied across a wide range of domains,leverage multiple latent direction discovery frameworks, and achievestate-of-the-art results in few-shot and low-supervision settings, even whencompared to methods designed to tackle a single task.