Abstract
Our goal is to train a policy for autonomous driving via imitation learningthat is robust enough to drive a real vehicle. We find that standard behaviorcloning is insufficient for handling complex driving scenarios, even when weleverage a perception system for preprocessing the input and a controller forexecuting the output on the car: 30 million examples are still not enough. Wepropose exposing the learner to synthesized data in the form of perturbationsto the expert's driving, which creates interesting situations such ascollisions and/or going off the road. Rather than purely imitating all data, weaugment the imitation loss with additional losses that penalize undesirableevents and encourage progress -- the perturbations then provide an importantsignal for these losses and lead to robustness of the learned model. We showthat the ChauffeurNet model can handle complex situations in simulation, andpresent ablation experiments that emphasize the importance of each of ourproposed changes and show that the model is responding to the appropriatecausal factors. Finally, we demonstrate the model driving a car in the realworld.