Abstract
We present Face Swapping GAN (FSGAN) for face swapping and reenactment.Unlike previous work, FSGAN is subject agnostic and can be applied to pairs offaces without requiring training on those faces. To this end, we describe anumber of technical contributions. We derive a novel recurrent neural network(RNN)-based approach for face reenactment which adjusts for both pose andexpression variations and can be applied to a single image or a video sequence.For video sequences, we introduce continuous interpolation of the face viewsbased on reenactment, Delaunay Triangulation, and barycentric coordinates.Occluded face regions are handled by a face completion network. Finally, we usea face blending network for seamless blending of the two faces while preservingtarget skin color and lighting conditions. This network uses a novel Poissonblending loss which combines Poisson optimization with perceptual loss. Wecompare our approach to existing state-of-the-art systems and show our resultsto be both qualitatively and quantitatively superior.