Abstract
This paper addresses the problem of inverse reinforcement learning (IRL) --inferring the reward function of an agent from observing its behavior. IRL canprovide a generalizable and compact representation for apprenticeship learning,and enable accurately inferring the preferences of a human in order to assistthem. %and provide for more accurate prediction. However, effective IRL ischallenging, because many reward functions can be compatible with an observedbehavior. We focus on how prior reinforcement learning (RL) experience can beleveraged to make learning these preferences faster and more efficient. Wepropose the IRL algorithm BASIS (Behavior Acquisition through Successor-featureIntention inference from Samples), which leverages multi-task RL pre-trainingand successor features to allow an agent to build a strong basis for intentionsthat spans the space of possible goals in a given domain. When exposed to justa few expert demonstrations optimizing a novel goal, the agent uses its basisto quickly and effectively infer the reward function. Our experiments revealthat our method is highly effective at inferring and optimizing demonstratedreward functions, accurately inferring reward functions from less than 100trajectories.