Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints

  • 2019-10-29 15:10:09
  • Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Devidze, Adish Singla
  • 0

Abstract

Inverse reinforcement learning (IRL) enables an agent to learn complexbehavior by observing demonstrations from a (near-)optimal policy. The typicalassumption is that the learner's goal is to match the teacher's demonstratedbehavior. In this paper, we consider the setting where the learner has its ownpreferences that it additionally takes into consideration. These preferencescan for example capture behavioral biases, mismatched worldviews, or physicalconstraints. We study two teaching approaches: learner-agnostic teaching, wherethe teacher provides demonstrations from an optimal policy ignoring thelearner's preferences, and learner-aware teaching, where the teacher accountsfor the learner's preferences. We design learner-aware teaching algorithms andshow that significant performance improvements can be achieved overlearner-agnostic teaching.

 

Quick Read (beta)

loading the full paper ...