Abstract
Humans appear to have a critical period (CP) for language acquisition: Secondlanguage (L2) acquisition becomes harder after early childhood, and ceasingexposure to a first language (L1) after this period (but not before) typicallydoes not lead to substantial loss of L1 proficiency. It is unknown whetherthese CP effects result from innately determined brain maturation or as astabilization of neural connections naturally induced by experience. In thisstudy, we use language models (LMs) to test the extent to which these phenomenaare peculiar to humans, or shared by a broader class of language learners. Wevary the age of exposure by training LMs on language pairs in variousexperimental conditions, and find that LMs, which lack any direct analog toinnate maturational stages, do not show CP effects when trained sequentially onL1 and L2. Our results contradict the claim that CP effects are an inevitableresult of learning in statistical learners, and they are consistent with aninnate mechanism for CP effects. We show that we can reverse-engineer the CP byintroducing a regularizer partway through training to simulate a maturationaldecrease in plasticity. All in all, our results suggest that L1 learning on itsown may not be enough to induce a CP, and additional engineering is necessaryto make language models more cognitively plausible.