Abstract
Recently, the power of unconditional image synthesis has significantlyadvanced through the use of Generative Adversarial Networks (GANs). The task ofinverting an image into its corresponding latent code of the trained GAN is ofutmost importance as it allows for the manipulation of real images, leveragingthe rich semantics learned by the network. Recognizing the limitations ofcurrent inversion approaches, in this work we present a novel inversion schemethat extends current encoder-based inversion methods by introducing aniterative refinement mechanism. Instead of directly predicting the latent codeof a given real image using a single pass, the encoder is tasked withpredicting a residual with respect to the current estimate of the invertedlatent code in a self-correcting manner. Our residual-based encoder, namedReStyle, attains improved accuracy compared to current state-of-the-artencoder-based methods with a negligible increase in inference time. We analyzethe behavior of ReStyle to gain valuable insights into its iterative nature. Wethen evaluate the performance of our residual encoder and analyze itsrobustness compared to optimization-based inversion and state-of-the-artencoders.