AlphaSeq: Sequence Discovery with Deep Reinforcement Learning

  • 2019-01-15 21:21:05
  • Yulin Shao, Soung Chang Liew, Taotao Wang
  • 0

Abstract

Sequences play an important role in many applications and systems.Discovering sequences with desired properties has long been an interestingintellectual pursuit. This paper puts forth a new paradigm, AlphaSeq, todiscover desired sequences algorithmically using deep reinforcement learning(DRL) techniques. AlphaSeq treats the sequence discovery problem as an episodicsymbol-filling game, in which a player fills symbols in the vacant positions ofa sequence set sequentially during an episode of the game. Each episode endswith a completely-filled sequence set, upon which a reward is given based onthe desirability of the sequence set. AlphaSeq models the game as a MarkovDecision Process (MDP), and adapts the DRL framework of AlphaGo to solve theMDP. Sequences discovered improve progressively as AlphaSeq, starting as anovice, learns to become an expert game player through many episodes of gameplaying. Compared with traditional sequence construction by mathematical tools,AlphaSeq is particularly suitable for problems with complex objectivesintractable to mathematical analysis. We demonstrate the searching capabilitiesof AlphaSeq in two applications: 1) AlphaSeq successfully rediscovers a set ofideal complementary codes that can zero-force all potential interferences inmulti-carrier CDMA systems. 2) AlphaSeq discovers new sequences that triple thesignal-to-interference ratio -- benchmarked against the well-known Legendresequence -- of a mismatched filter estimator in pulse compression radarsystems.

 

Quick Read (beta)

loading the full paper ...