Abstract
Akin to neuroplasticity in human brains, the plasticity of deep neuralnetworks enables their quick adaption to new data. This makes plasticityparticularly crucial for deep Reinforcement Learning (RL) agents: Onceplasticity is lost, an agent's performance will inevitably plateau because itcannot improve its policy to account for changes in the data distribution,which are a necessary consequence of its learning process. Thus, developingwell-performing and sample-efficient agents hinges on their ability to remainplastic during training. Furthermore, the loss of plasticity can be connectedto many other issues plaguing deep RL, such as training instabilities, scalingfailures, overestimation bias, and insufficient exploration. With this survey,we aim to provide an overview of the emerging research on plasticity loss foracademics and practitioners of deep reinforcement learning. First, we propose aunified definition of plasticity loss based on recent works, relate it todefinitions from the literature, and discuss metrics for measuring plasticityloss. Then, we categorize and discuss numerous possible causes of plasticityloss before reviewing currently employed mitigation strategies. Our taxonomy isthe first systematic overview of the current state of the field. Lastly, wediscuss prevalent issues within the literature, such as a necessity for broaderevaluation, and provide recommendations for future research, like gaining abetter understanding of an agent's neural activity and behavior.