Dynamic Sparse Training for Deep Reinforcement Learning

  • 2022-05-05 19:37:44
  • Ghada Sokar, Elena Mocanu, Decebal Constantin Mocanu, Mykola Pechenizkiy, Peter Stone
  • 0

Abstract

Deep reinforcement learning (DRL) agents are trained through trial-and-errorinteractions with the environment. This leads to a long training time for denseneural networks to achieve good performance. Hence, prohibitive computation andmemory resources are consumed. Recently, learning efficient DRL agents hasreceived increasing attention. Yet, current methods focus on acceleratinginference time. In this paper, we introduce for the first time a dynamic sparsetraining approach for deep reinforcement learning to accelerate the trainingprocess. The proposed approach trains a sparse neural network from scratch anddynamically adapts its topology to the changing data distribution duringtraining. Experiments on continuous control tasks show that our dynamic sparseagents achieve higher performance than the equivalent dense methods, reduce theparameter count and floating-point operations (FLOPs) by 50%, and have a fasterlearning speed that enables reaching the performance of dense agents with40-50% reduction in the training steps.

 

Quick Read (beta)

loading the full paper ...