Abstract
Reinforcement learning has been demonstrated to outperform even the besthumans in complex domains like video games. However, running reinforcementlearning experiments on the required scale for autonomous driving is extremelydifficult. Building a large scale reinforcement learning system anddistributing it across many GPUs is challenging. Gathering experience duringtraining on real world vehicles is prohibitive from a safety and scalabilityperspective. Therefore, an efficient and realistic driving simulator isrequired that uses a large amount of data from real-world driving. We bringthese capabilities together and conduct large-scale reinforcement learningexperiments for autonomous driving. We demonstrate that our policy performanceimproves with increasing scale. Our best performing policy reduces the failurerate by 64% while improving the rate of driving progress by 25% compared to thepolicies produced by state-of-the-art machine learning for autonomous driving.