Abstract
In this work, we propose a novel Cycle In Cycle Generative AdversarialNetwork (C$^2$GAN) for the task of keypoint-guided image generation. Theproposed C$^2$GAN is a cross-modal framework exploring a joint exploitation ofthe keypoint and the image data in an interactive manner. C$^2$GAN contains twodifferent types of generators, i.e., keypoint-oriented generator andimage-oriented generator. Both of them are mutually connected in an end-to-endlearnable fashion and explicitly form three cycled sub-networks, i.e., oneimage generation cycle and two keypoint generation cycles. Each cycle not onlyaims at reconstructing the input domain, and also produces useful outputinvolving in the generation of another cycle. By so doing, the cycles constraineach other implicitly, which provides complementary information from the twodifferent modalities and brings extra supervision across cycles, thusfacilitating more robust optimization of the whole network. Extensiveexperimental results on two publicly available datasets, i.e., Radboud Facesand Market-1501, demonstrate that our approach is effective to generate morephoto-realistic images compared with state-of-the-art models.