Abstract
Deep reinforcement learning has been successful in a variety of tasks, suchas game playing and robotic manipulation. However, attempting to learn\textit{tabula rasa} disregards the logical structure of many domains as wellas the wealth of readily available knowledge from domain experts that couldhelp "warm start" the learning process. We present a novel reinforcementlearning technique that allows for intelligent initialization of a neuralnetwork weights and architecture. Our approach permits the encoding domainknowledge directly into a neural decision tree, and improves upon thatknowledge with policy gradient updates. We empirically validate our approach ontwo OpenAI Gym tasks and two modified StarCraft 2 tasks, showing that our novelarchitecture outperforms multilayer-perceptron and recurrent architectures. Ourknowledge-based framework finds superior policies compared to imitationlearning-based and prior knowledge-based approaches. Importantly, wedemonstrate that our approach can be used by untrained humans to initiallyprovide >80% increase in expected reward relative to baselines prior totraining (p < 0.001), which results in a >60% increase in expected reward afterpolicy optimization (p = 0.011).