Abstract
Imitation learning is an effective alternative approach to learn a policywhen the reward function is sparse. In this paper, we consider a challengingsetting where an agent and an expert use different actions from each other. Weassume that the agent has access to a sparse reward function and state-onlyexpert observations. We propose a method which gradually balances between theimitation learning cost and the reinforcement learning objective. In addition,this method adapts the agent's policy based on either mimicking expert behavioror maximizing sparse reward. We show, through navigation scenarios, that (i) anagent is able to efficiently leverage sparse rewards to outperform standardstate-only imitation learning, (ii) it can learn a policy even when its actionsare different from the expert, and (iii) the performance of the agent is notbounded by that of the expert, due to the optimized usage of sparse rewards.