Simultaneously Evolving Deep Reinforcement Learning Models using Multifactorial Optimization

  • 2020-03-23 10:47:41
  • Aritz D. Martinez, Eneko Osaba, Javier Del Ser, Francisco Herrera
  • 0

Abstract

In recent years, Multifactorial Optimization (MFO) has gained a notablemomentum in the research community. MFO is known for its inherent capability toefficiently address multiple optimization tasks at the same time, whiletransferring information among such tasks to improve their convergence speed.On the other hand, the quantum leap made by Deep Q Learning (DQL) in theMachine Learning field has allowed facing Reinforcement Learning (RL) problemsof unprecedented complexity. Unfortunately, complex DQL models usually find itdifficult to converge to optimal policies due to the lack of exploration orsparse rewards. In order to overcome these drawbacks, pre-trained models arewidely harnessed via Transfer Learning, extrapolating knowledge acquired in asource task to the target task. Besides, meta-heuristic optimization has beenshown to reduce the lack of exploration of DQL models. This work proposes a MFOframework capable of simultaneously evolving several DQL models towards solvinginterrelated RL tasks. Specifically, our proposed framework blends together thebenefits of meta-heuristic optimization, Transfer Learning and DQL to automatethe process of knowledge transfer and policy learning of distributed RL agents.A thorough experimentation is presented and discussed so as to assess theperformance of the framework, its comparison to the traditional methodology forTransfer Learning in terms of convergence, speed and policy quality , and theintertask relationships found and exploited over the search process.

 

Quick Read (beta)

loading the full paper ...