Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

  • 2020-07-27 15:21:22
  • Noam Brown, Anton Bakhtin, Adam Lerer, Qucheng Gong
  • 3


The combination of deep reinforcement learning and search at both trainingand test time is a powerful paradigm that has led to a number of a successes insingle-agent settings and perfect-information games, best exemplified by thesuccess of AlphaZero. However, algorithms of this form have been unable to copewith imperfect-information games. This paper presents ReBeL, a generalframework for self-play reinforcement learning and search forimperfect-information games. In the simpler setting of perfect-informationgames, ReBeL reduces to an algorithm similar to AlphaZero. Results show ReBeLleads to low exploitability in benchmark imperfect-information games andachieves superhuman performance in heads-up no-limit Texas hold'em poker, whileusing far less domain knowledge than any prior poker AI. We also prove thatReBeL converges to a Nash equilibrium in two-player zero-sum games in tabularsettings.


Quick Read (beta)

loading the full paper ...