Abstract
A central goal of linguistic theory is to find a precise characterization ofthe notion "possible human language", in the form of a computational devicethat is capable of describing all and only the languages that can be acquiredby a typically developing human child. The success of recent large languagemodels (LLMs) in NLP applications arguably raises the possibility that LLMsmight be computational devices that meet this goal. This would only be the caseif, in addition to succeeding in learning human languages, LLMs struggle tolearn "impossible" human languages. Kallini et al. (2024; "Mission: ImpossibleLanguage Models", Proc. ACL) conducted experiments aiming to test this bytraining GPT-2 on a variety of synthetic languages, and found that it learnssome more successfully than others. They present these asymmetries as supportfor the idea that LLMs' inductive biases align with what is regarded as"possible" for human languages, but the most significant comparison has aconfound that makes this conclusion unwarranted. In this paper I explain theconfound and suggest some ways forward towards constructing a comparison thatappropriately tests the underlying issue.