Abstract
Deep reinforcement learning agents may learn complex tasks more efficientlywhen they coordinate with one another. We consider a teacher-studentcoordination scheme wherein an agent may ask another agent for demonstrations.Despite the benefits of sharing demonstrations, however, potential adversariesmay obtain sensitive information belonging to the teacher by observing thedemonstrations. In particular, deep reinforcement learning algorithms are knownto be vulnerable to membership attacks, which make accurate inferences aboutthe membership of the entries of training datasets. Therefore, there is a needto safeguard the teacher against such privacy threats. We fix the teacher'spolicy as the context of the demonstrations, which allows for differentinternal models across the student and the teacher, and contrasts the existingmethods. We make the following two contributions. (i) We develop adifferentially private mechanism that protects the privacy of the teacher'straining dataset. (ii) We propose a proximal policy-optimization objective thatenables the student to benefit from the demonstrations despite theperturbations of the privacy mechanism. We empirically show that the algorithmimproves the student's learning upon convergence rate and utility.Specifically, compared with an agent who learns the same task on its own, weobserve that the student's policy converges faster, and the converging policyaccumulates higher rewards more robustly.