Policy Poisoning in Batch Reinforcement Learning and Control

  • 2019-10-31 01:58:12
  • Yuzhe Ma, Xuezhou Zhang, Wen Sun, Xiaojin Zhu
  • 0

Abstract

We study a security threat to batch reinforcement learning and control wherethe attacker aims to poison the learned policy. The victim is a reinforcementlearner / controller which first estimates the dynamics and the rewards from abatch data set, and then solves for the optimal policy with respect to theestimates. The attacker can modify the data set slightly before learninghappens, and wants to force the learner into learning a target policy chosen bythe attacker. We present a unified framework for solving batch policy poisoningattacks, and instantiate the attack on two standard victims: tabular certaintyequivalence learner in reinforcement learning and linear quadratic regulator incontrol. We show that both instantiation result in a convex optimizationproblem on which global optimality is guaranteed, and provide analysis onattack feasibility and attack cost. Experiments show the effectiveness ofpolicy poisoning attacks.

 

Quick Read (beta)

loading the full paper ...