Abstract
We introduce a new local sparse attention layer that preservestwo-dimensional geometry and locality. We show that by just replacing the denseattention layer of SAGAN with our construction, we obtain very significant FID,Inception score and pure visual improvements. FID score is improved from$18.65$ to $15.94$ on ImageNet, keeping all other parameters the same. Thesparse attention patterns that we propose for our new layer are designed usinga novel information theoretic criterion that uses information flow graphs. Wealso present a novel way to invert Generative Adversarial Networks withattention. Our method extracts from the attention layer of the discriminator asaliency map, which we use to construct a new loss function for the inversion.This allows us to visualize the newly introduced attention heads and show thatthey indeed capture interesting aspects of two-dimensional geometry of realimages.