Facial caricature is an art form of drawing faces in an exaggerated way toconvey humor or sarcasm. In this paper, we propose the first GenerativeAdversarial Network (GAN) for unpaired photo-to-caricature translation, whichwe call "CariGANs". It explicitly models geometric exaggeration and appearancestylization using two components: CariGeoGAN, which only models thegeometry-to-geometry transformation from face photos to caricatures, andCariStyGAN, which transfers the style appearance from caricatures to facephotos without any geometry deformation. In this way, a difficult cross-domaintranslation problem is decoupled into two easier tasks. The perceptual studyshows that caricatures generated by our CariGANs are closer to the hand-drawnones, and at the same time better persevere the identity, compared tostate-of-the-art methods. Moreover, our CariGANs allow users to control theshape exaggeration degree and change the color/texture style by tuning theparameters or giving an example caricature.