Abstract
Recent breakthroughs in AI for multi-agent games like Go, Poker, and Dota,have seen great strides in recent years. Yet none of these games address thereal-life challenge of cooperation in the presence of unknown and uncertainteammates. This challenge is a key game mechanism in hidden role games. Here wedevelop the DeepRole algorithm, a multi-agent reinforcement learning agent thatwe test on The Resistance: Avalon, the most popular hidden role game. DeepRolecombines counterfactual regret minimization (CFR) with deep value networkstrained through self-play. Our algorithm integrates deductive reasoning intovector-form CFR to reason about joint beliefs and deduce partially observableactions. We augment deep value networks with constraints that yieldinterpretable representations of win probabilities. These innovations enableDeepRole to scale to the full Avalon game. Empirical game-theoretic methodsshow that DeepRole outperforms other hand-crafted and learned agents infive-player Avalon. DeepRole played with and against human players on the webin hybrid human-agent teams. We find that DeepRole outperforms human players asboth a cooperator and a competitor.