Abstract
Recent advancements in multi-agent reinforcement learning (MARL) have openedup vast application prospects, such as swarm control of drones, collaborativemanipulation by robotic arms, and multi-target encirclement. However, potentialsecurity threats during the MARL deployment need more attention and thoroughinvestigation. Recent research reveals that attackers can rapidly exploit thevictim's vulnerabilities, generating adversarial policies that result in thefailure of specific tasks. For instance, reducing the winning rate of asuperhuman-level Go AI to around 20%. Existing studies predominantly focus ontwo-player competitive environments, assuming attackers possess complete globalstate observation. In this study, we unveil, for the first time, the capability of attackers togenerate adversarial policies even when restricted to partial observations ofthe victims in multi-agent competitive environments. Specifically, we propose anovel black-box attack (SUB-PLAY) that incorporates the concept of constructingmultiple subgames to mitigate the impact of partial observability and suggestssharing transitions among subpolicies to improve attackers' exploitativeability. Extensive evaluations demonstrate the effectiveness of SUB-PLAY underthree typical partial observability limitations. Visualization results indicatethat adversarial policies induce significantly different activations of thevictims' policy networks. Furthermore, we evaluate three potential defensesaimed at exploring ways to mitigate security threats posed by adversarialpolicies, providing constructive recommendations for deploying MARL incompetitive environments.