Abstract
We address the problem of detecting adversarial attacks against cooperativemulti-agent reinforcement learning with continuous action space. We propose adecentralized detector that relies solely on the local observations of theagents and makes use of a statistical characterization of the normal behaviorof observable agents. The proposed detector utilizes deep neural networks toapproximate the normal behavior of agents as parametric multivariate Gaussiandistributions. Based on the predicted density functions, we define a normalityscore and provide a characterization of its mean and variance. Thischaracterization allows us to employ a two-sided CUSUM procedure for detectingdeviations of the normality score from its mean, serving as a detector ofanomalous behavior in real-time. We evaluate our scheme on various multi-agentPettingZoo benchmarks against different state-of-the-art attack methods, andour results demonstrate the effectiveness of our method in detecting impactfuladversarial attacks. Particularly, it outperforms the discrete counterpart byachieving AUC-ROC scores of over 0.95 against the most impactful attacks in allevaluated environments.