Abstract
The loss of plasticity in learning agents, analogous to the solidification ofneural pathways in biological brains, significantly impedes learning andadaptation in reinforcement learning due to its non-stationary nature. Toaddress this fundamental challenge, we propose a novel approach, {\itNeuroplastic Expansion} (NE), inspired by cortical expansion in cognitivescience. NE maintains learnability and adaptability throughout the entiretraining process by dynamically growing the network from a smaller initial sizeto its full dimension. Our method is designed with three key components:(\textit{1}) elastic topology generation based on potential gradients,(\textit{2}) dormant neuron pruning to optimize network expressivity, and(\textit{3}) neuron consolidation via experience review to strike a balance inthe plasticity-stability dilemma. Extensive experiments demonstrate that NEeffectively mitigates plasticity loss and outperforms state-of-the-art methodsacross various tasks in MuJoCo and DeepMind Control Suite environments. NEenables more adaptive learning in complex, dynamic environments, whichrepresents a crucial step towards transitioning deep reinforcement learningfrom static, one-time training paradigms to more flexible, continually adaptingmodels.