Abstract
Deep Reinforcement Learning has enabled the control of increasingly complexand high-dimensional problems. However, the need of vast amounts of data beforereasonable performance is attained prevents its widespread application. Weemploy binary corrective feedback as a general and intuitive manner toincorporate human intuition and domain knowledge in model-free machinelearning. The uncertainty in the policy and the corrective feedback is combineddirectly in the action space as probabilistic conditional exploration. As aresult, the greatest part of the otherwise ignorant learning process can beavoided. We demonstrate the proposed method, Predictive Probabilistic Mergingof Policies (PPMP), in combination with DDPG. In experiments on continuouscontrol problems of the OpenAI Gym, we achieve drastic improvements in sampleefficiency, final performance, and robustness to erroneous feedback, both forhuman and synthetic feedback. Additionally, we show solutions beyond thedemonstrated knowledge.