Abstract
Reinforcement learning provides a powerful and general framework for decisionmaking and control, but its application in practice is often hindered by theneed for extensive feature and reward engineering. Deep reinforcement learningmethods can remove the need for explicit engineering of policy or valuefeatures, but still require a manually specified reward function. Inversereinforcement learning holds the promise of automatic reward acquisition, buthas proven exceptionally difficult to apply to large, high-dimensional problemswith unknown dynamics. In this work, we propose adverserial inversereinforcement learning (AIRL), a practical and scalable inverse reinforcementlearning algorithm based on an adversarial reward learning formulation. Wedemonstrate that AIRL is able to recover reward functions that are robust tochanges in dynamics, enabling us to learn policies even under significantvariation in the environment seen during training. Our experiments show thatAIRL greatly outperforms prior methods in these transfer settings.