Abstract
Humans appear to have a critical period (CP) for language acquisition: Secondlanguage (L2) acquisition becomes harder after early childhood, and ceasingexposure to a first language (L1) after this period (but not before) typicallydoes not lead to substantial loss of L1 proficiency. It is unknown whetherthese CP effects result from innately determined brain maturation or as astabilization of neural connections naturally induced by experience. In thisstudy, we use language models (LMs) to test the extent to which these phenomenaare peculiar to humans, or shared by a broader class of language learners. Wevary the age of exposure by training LMs on language pairs in variousexperimental conditions, and find that LMs, which lack any direct analog toinnate maturational stages, do not show CP effects when the age of exposure ofL2 is delayed. Our results contradict the claim that CP effects are aninevitable result of statistical learning, and they are consistent with aninnate mechanism for CP effects. We show that we can reverse-engineer the CP byintroducing a regularizer partway through training to simulate a maturationaldecrease in plasticity. All in all, our results suggest that L1 learning on itsown may not be enough to induce a CP, and additional engineering is necessaryto make language models more cognitively plausible.