Abstract
Dynamic sparse training (DST) literature demonstrates that a highly sparseneural network can match the performance of its corresponding dense network insupervised and unsupervised learning when it is trained from scratch whilesubstantially reducing the computational and memory costs. In this paper, weshow for the first time that deep reinforcement learning can also benefit fromdynamic sparse training. We demonstrate that DST can be leveraged to decreasethe long training time required by deep reinforcement learning agents withoutsacrificing performance. To achieve this, we propose a DST algorithm thatadapts to the online nature and instability of the deep reinforcement learningparadigm. We integrate our proposed algorithm with state-of-the-art deepreinforcement learning methods. Experimental results demonstrate that ourdynamic sparse compact agents can effectively learn and achieve higherperformance than the original dense methods while reducing the parameter countand floating-point operations (FLOPs) by 50%. More impressively, our dynamicsparse agents have a faster learning speed. They can reach the finalperformance achieved by dense agents after 40-50% of the steps required by thelatter. We evaluate our approach on OpenAI gym continuous control tasks.