Abstract
The combination of Reinforcement Learning (RL) with deep learning has led toa series of impressive feats, with many believing (deep) RL provides a pathtowards generally capable agents. However, the success of RL agents is oftenhighly sensitive to design choices in the training process, which may requiretedious and error-prone manual tuning. This makes it challenging to use RL fornew problems, while also limits its full potential. In many other areas ofmachine learning, AutoML has shown it is possible to automate such designchoices and has also yielded promising initial results when applied to RL.However, Automated Reinforcement Learning (AutoRL) involves not only standardapplications of AutoML but also includes additional challenges unique to RL,that naturally produce a different set of methods. As such, AutoRL has beenemerging as an important area of research in RL, providing promise in a varietyof applications from RNA design to playing games such as Go. Given thediversity of methods and environments considered in RL, much of the researchhas been conducted in distinct subfields, ranging from meta-learning toevolution. In this survey we seek to unify the field of AutoRL, we provide acommon taxonomy, discuss each area in detail and pose open problems which wouldbe of interest to researchers going forward.