Abstract
Robustness of Deep Reinforcement Learning (DRL) algorithms towardsadversarial attacks in real world applications such as those deployed incyber-physical systems (CPS) are of increasing concern. Numerous studies haveinvestigated the mechanisms of attacks on the RL agent's state space.Nonetheless, attacks on the RL agent's action space (AS) (corresponding toactuators in engineering systems) are equally perverse; such attacks arerelatively less studied in the ML literature. In this work, we first frame theproblem as an optimization problem of minimizing the cumulative reward of an RLagent with decoupled constraints as the budget of attack. We propose awhite-box Myopic Action Space (MAS) attack algorithm that distributes theattacks across the action space dimensions. Next, we reformulate theoptimization problem above with the same objective function, but with atemporally coupled constraint on the attack budget to take into account theapproximated dynamics of the agent. This leads to the white-box Look-aheadAction Space (LAS) attack algorithm that distributes the attacks across theaction and temporal dimensions. Our results shows that using the same amount ofresources, the LAS attack deteriorates the agent's performance significantlymore than the MAS attack. This reveals the possibility that with limitedresource, an adversary can utilize the agent's dynamics to malevolently craftattacks that causes the agent to fail. Additionally, we leverage these attackstrategies as a possible tool to gain insights on the potential vulnerabilitiesof DRL agents.