Abstract
Out of the many deep reinforcement learning approaches for autonomousdriving, only few make use of the options (or skills) framework. That issurprising, as this framework is naturally suited for hierarchical controlapplications in general, and autonomous driving tasks in specific. Therefore,in this work the options framework is applied and tailored to autonomousdriving tasks on highways. More specifically, we define dedicated options forlongitudinal and lateral manoeuvres with embedded safety and comfortconstraints. This way, prior domain knowledge can be incorporated into thelearning process and the learned driving behaviour can be constrained moreeasily. We propose several setups for hierarchical control with options andderive practical algorithms following state-of-the-art reinforcement learningtechniques. By separately selecting actions for longitudinal and lateralcontrol, the introduced policies over combined and hybrid options obtain thesame expressiveness and flexibility that human drivers have, while being easierto interpret than classical policies over continuous actions. Of all theinvestigated approaches, these flexible policies over hybrid options performthe best under varying traffic conditions, outperforming the baseline policiesover actions.