Abstract
Critical periods are phases in the early development of humans and animalsduring which experience can affect the structure of neuronal networksirreversibly. In this work, we study the effects of visual stimulus deficits onthe training of artificial neural networks (ANNs). Introducingwell-characterized visual deficits, such as cataract-like blurring, in theearly training phase of a standard deep neural network causes irreversibleperformance loss that closely mimics that reported in humans and animal models.Deficits that do not affect low-level image statistics, such as verticalflipping of the images, have no lasting effect on the ANN's performance and canbe rapidly overcome with additional training, as observed in humans. Inaddition, deeper networks show a more prominent critical period. To betterunderstand this phenomenon, we use techniques from information theory to studythe strength of the network connections during training. Our analysis suggeststhat the first few epochs are critical for the allocation of resources acrossdifferent layers, determined by the initial input data distribution. Once suchinformation organization is established, the network resources do notre-distribute through additional training. These findings suggest that theinitial rapid learning phase of training of ANNs, under-scrutinized compared toits asymptotic behavior, plays a key role in defining the final performance ofnetworks.