This paper presents a novel approach for synthesizing automaticallyage-progressed facial images in video sequences using Deep ReinforcementLearning. The proposed method models facial structures and the longitudinalface-aging process of given subjects coherently across video frames. Theapproach is optimized using a long-term reward, Reinforcement Learning functionwith deep feature extraction from Deep Convolutional Neural Network. Unlikeprevious age-progression methods that are only able to synthesize an agedlikeness of a face from a single input image, the proposed approach is capableof age-progressing facial likenesses in videos with consistently synthesizedfacial features across frames. In addition, the deep reinforcement learningmethod guarantees preservation of the visual identity of input faces afterage-progression. Results on videos of our new collected aging face AGFW-v2database demonstrate the advantages of the proposed solution in terms of bothquality of age-progressed faces, temporal smoothness, and cross-age faceverification.