SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning

  • 2018-11-30 23:01:23
  • Daoming Lyu, Fangkai Yang, Bo Liu, Steven Gustafson
  • 0

Abstract

Deep reinforcement learning (DRL) has gained great success by learningdirectly from high-dimensional sensory inputs, yet is notorious for the lack ofinterpretability. Interpretability of the subtasks is critical in hierarchicaldecision-making as it increases the transparency of black-box-style DRLapproach and helps the RL practitioners to understand the high-level behaviorof the system better. In this paper, we introduce symbolic planning into DRLand propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that canhandle both high-dimensional sensory inputs and symbolic planning. Thetask-level interpretability is enabled by relating symbolic actions tooptions.This framework features a planner -- controller -- meta-controllerarchitecture, which takes charge of subtask scheduling, data-driven subtasklearning, and subtask evaluation, respectively. The three componentscross-fertilize each other and eventually converge to an optimal symbolic planalong with the learned subtasks, bringing together the advantages of long-termplanning capability with symbolic knowledge and end-to-end reinforcementlearning directly from a high-dimensional sensory input. Experimental resultsvalidate the interpretability of subtasks, along with improved data efficiencycompared with state-of-the-art approaches.

 

Quick Read (beta)

loading the full paper ...