Deep Reinforcement Learning (DRL) has become increasingly powerful in recentyears, with notable achievements such as Deepmind's AlphaGo. It has beensuccessfully deployed in commercial vehicles like Mobileye's path planningsystem. However, a vast majority of work on DRL is focused on toy examples incontrolled synthetic car simulator environments such as TORCS and CARLA. Ingeneral, DRL is still at its infancy in terms of usability in real-worldapplications. Our goal in this paper is to encourage real-world deployment ofDRL in various autonomous driving (AD) applications. We first provide anoverview of the tasks in autonomous driving systems, reinforcement learningalgorithms and applications of DRL to AD systems. We then discuss thechallenges which must be addressed to enable further progress towardsreal-world deployment.