Age progression and regression refers to aesthetically rendering a given faceimage to present effects of face aging and rejuvenation, respectively. Althoughnumerous studies have been conducted in this topic, there are still two majorproblems: 1) multiple models are usually trained to simulate different agemappings, and 2) the photo-realism of generated face images is heavilyinfluenced by the variation of training images in terms of pose, illumination,and background. To address these issues, in this paper, we propose a frameworkbased on conditional Generative Adversarial Networks (cGANs) to achieve ageprogression and regression simultaneously. Particularly, since face aging andrejuvenation are largely different in terms of image translation patterns, wemodel these two processes using two separate generators, each dedicated to oneage changing process. In addition, we exploit the spatial attention mechanismto limit image modifications to regions closely related to age changes, so thatimages with high visual fidelity could be synthesized for in-the-wild cases.Experiments on multiple datasets demonstrate the ability of our model insynthesizing lifelike face images at desired ages with personalized featureswell preserved.