Reinforcement Learning and recently Deep Reinforcement Learning are popularmethods for solving sequential decision making problems modeled as MarkovDecision Processes. RL modeling of a problem and selecting algorithms andhyper-parameters require careful considerations as different configurations mayentail completely different performances. These considerations are mainly thetask of RL experts; however, RL is progressively becoming popular in otherfields where the researchers and system designers are not RL experts. Besides,many modeling decisions, such as defining state and action space, size ofbatches and frequency of batch updating, and number of timesteps are typicallymade manually. For these reasons, automating different components of RLframework is of great importance and it has attracted much attention in recentyears. Automated RL provides a framework in which different components of RLincluding MDP modeling, algorithm selection and hyper-parameter optimizationare modeled and defined automatically. In this article, we explore theliterature and present recent work that can be used in automated RL. Moreover,we discuss the challenges, open questions and research directions in AutoRL.