Abstract
Deep reinforcement learning has achieved significant success in manydecision-making tasks in various fields. However, it requires a large trainingtime of dense neural networks to obtain a good performance. This hinders itsapplicability on low-resource devices where memory and computation are strictlyconstrained. In a step towards enabling deep reinforcement learning agents tobe applied to low-resource devices, in this work, we propose for the first timeto dynamically train deep reinforcement learning agents with sparse neuralnetworks from scratch. We adopt the evolution principles of dynamic sparsetraining in the reinforcement learning paradigm and introduce a trainingalgorithm that optimizes the sparse topology and the weight values jointly todynamically fit the incoming data. Our approach is easy to be integrated intoexisting deep reinforcement learning algorithms and has many favorableadvantages. First, it allows for significant compression of the network sizewhich reduces the memory and computation costs substantially. This wouldaccelerate not only the agent inference but also its training process. Second,it speeds up the agent learning process and allows for reducing the number ofrequired training steps. Third, it can achieve higher performance than trainingthe dense counterpart network. We evaluate our approach on OpenAI gymcontinuous control tasks. The experimental results show the effectiveness ofour approach in achieving higher performance than one of the state-of-artbaselines with a 50\% reduction in the network size and floating-pointoperations (FLOPs). Moreover, our proposed approach can reach the sameperformance achieved by the dense network with a 40-50\% reduction in thenumber of training steps.