Abstract
Adoption of machine learning (ML)-enabled cyber-physical systems (CPS) arebecoming prevalent in various sectors of modern society such as transportation,industrial, and power grids. Recent studies in deep reinforcement learning(DRL) have demonstrated its benefits in a large variety of data-drivendecisions and control applications. As reliance on ML-enabled systems grows, itis imperative to study the performance of these systems under malicious stateand actuator attacks. Traditional control systems employresilient/fault-tolerant controllers that counter these attacks by correctingthe system via error observations. However, in some applications, a resilientcontroller may not be sufficient to avoid a catastrophic failure. Ideally, arobust approach is more useful in these scenarios where a system is inherentlyrobust (by design) to adversarial attacks. While robust control has a longhistory of development, robust ML is an emerging research area that has alreadydemonstrated its relevance and urgency. However, the majority of robust MLresearch has focused on perception tasks and not on decision and control tasks,although the ML (specifically RL) models used for control applications areequally vulnerable to adversarial attacks. In this paper, we show that awell-performing DRL agent that is initially susceptible to action spaceperturbations (e.g. actuator attacks) can be robustified against similarperturbations through adversarial training.