Abstract
Deep reinforcement learning (RL) is emerging as a viable strategy forautomated cyber defense (ACD). The traditional RL approach represents networksas a list of computers in various states of safety or threat. Unfortunately,these models are forced to overfit to specific network topologies, renderingthem ineffective when faced with even small environmental perturbations. Inthis work, we frame ACD as a two-player context-based partially observableMarkov decision problem with observations represented as attributed graphs.This approach allows our agents to reason through the lens of relationalinductive bias. Agents learn how to reason about hosts interacting with othersystem entities in a more general manner, and their actions are understood asedits to the graph representing the environment. By introducing this bias, wewill show that our agents can better reason about the states of networks andzero-shot adapt to new ones. We show that this approach outperforms thestate-of-the-art by a wide margin, and makes our agents capable of defendingnever-before-seen networks against a wide range of adversaries in a variety ofcomplex, and multi-agent environments.