Abstract
Reinforcement learning (RL) has been applied to attack graphs for penetrationtesting, however, trained agents do not reflect reality because the attackgraphs lack operational nuances typically captured within the intelligencepreparation of the battlefield (IPB) that include notions of (cyber) terrain.In particular, current practice constructs attack graphs exclusively using theCommon Vulnerability Scoring System (CVSS) and its components. We presentmethods for constructing attack graphs using notions from IPB on cyber terrainanalysis of obstacles, avenues of approach, key terrain, observation and fieldsof fire, and cover and concealment. We demonstrate our methods on an examplewhere firewalls are treated as obstacles and represented in (1) the rewardspace and (2) the state dynamics. We show that terrain analysis can be used tobring realism to attack graphs for RL.