Abstract
Training deep reinforcement learning agents complex behaviors in 3D virtualenvironments requires significant computational resources. This is especiallytrue in environments with high degrees of aliasing, where many states sharenearly identical visual features. Minecraft is an exemplar of such anenvironment. We hypothesize that interactive machine learning IML, whereinhuman teachers play a direct role in training through demonstrations, critique,or action advice, may alleviate agent susceptibility to aliasing. However,interactive machine learning is only practical when the number of humaninteractions is limited, requiring a balance between human teacher effort andagent performance. We conduct experiments with two reinforcement learningalgorithms which enable human teachers to give action advice, FeedbackArbitration and Newtonian Action Advice, under visual aliasing conditions. Toassess potential cognitive load per advice type, we vary the accuracy andfrequency of various human action advice techniques. Training efficiency,robustness against infrequent and inaccurate advisor input, and sensitivity toaliasing are examined.