Abstract
In video games, non-player characters (NPCs) are used to enhance the players'experience in a variety of ways, e.g., as enemies, allies, or innocentbystanders. A crucial component of NPCs is navigation, which allows them tomove from one point to another on the map. The most popular approach for NPCnavigation in the video game industry is to use a navigation mesh (NavMesh),which is a graph representation of the map, with nodes and edges indicatingtraversable areas. Unfortunately, complex navigation abilities that extend thecharacter's capacity for movement, e.g., grappling hooks, jetpacks,teleportation, or double-jumps, increases the complexity of the NavMesh, makingit intractable in many practical scenarios. Game designers are thus constrainedto only add abilities that can be handled by a NavMesh if they want to have NPCnavigation. As an alternative, we propose to use Deep Reinforcement Learning(Deep RL) to learn how to navigate 3D maps using any navigation ability. Wetest our approach on complex 3D environments in the Unity game engine that arenotably an order of magnitude larger than maps typically used in the Deep RLliterature. One of these maps is directly modeled after a Ubisoft AAA game. Wefind that our approach performs surprisingly well, achieving at least $90\%$success rate on all tested scenarios. A video of our results is available athttps://youtu.be/WFIf9Wwlq8M.