Abstract
The common pipeline in autonomous driving systems is highly modular andincludes a perception component which extracts lists of surrounding objects andpasses these lists to a high-level decision component. In this case, leveragingthe benefits of deep reinforcement learning for high-level decision makingrequires special architectures to deal with multiple variable-length sequencesof different object types, such as vehicles, lanes or traffic signs. At thesame time, the architecture has to be able to cover interactions betweentraffic participants in order to find the optimal action to be taken. In thiswork, we propose the novel Deep Scenes architecture, that can learn complexinteraction-aware scene representations based on extensions of either 1) DeepSets or 2) Graph Convolutional Networks. We present the Graph-Q and DeepScene-Qoff-policy reinforcement learning algorithms, both outperformingstate-of-the-art methods in evaluations with the publicly available trafficsimulator SUMO.