Abstract
Reinforcement Learning (RL) can be extremely effective in solving complex,real-world problems. However, injecting human knowledge into an RL agent mayrequire extensive effort and expertise on the human designer's part. To date,human factors are generally not considered in the development and evaluation ofpossible RL approaches. In this article, we set out to investigate howdifferent methods for injecting human knowledge are applied, in practice, byhuman designers of varying levels of knowledge and skill. We perform the firstempirical evaluation of several methods, including a newly proposed methodnamed SASS which is based on the notion of similarities in the agent'sstate-action space. Through this human study, consisting of 51 humanparticipants, we shed new light on the human factors that play a key role inRL. We find that the classical reward shaping technique seems to be the mostnatural method for most designers, both expert and non-expert, to speed up RL.However, we further find that our proposed method SASS can be effectively andefficiently combined with reward shaping, and provides a beneficial alternativeto using only a single speedup method with minimal human designer effortoverhead.