Abstract
The ability to predict, anticipate and reason about future outcomes is a keycomponent of intelligent decision-making systems. In light of the success ofdeep learning in computer vision, deep-learning-based video prediction emergedas a promising research direction. Defined as a self-supervised learning task,video prediction represents a suitable framework for representation learning,as it demonstrated potential capabilities for extracting meaningfulrepresentations of the underlying patterns in natural videos. Motivated by theincreasing interest in this task, we provide a review on the deep learningmethods for prediction in video sequences. We firstly define the videoprediction fundamentals, as well as mandatory background concepts and the mostused datasets. Next, we carefully analyze existing video prediction modelsorganized according to a proposed taxonomy, highlighting their contributionsand their significance in the field. The summary of the datasets and methods isaccompanied with experimental results that facilitate the assessment of thestate of the art on a quantitative basis. The paper is summarized by drawingsome general conclusions, identifying open research challenges and by pointingout future research directions.