Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models

  • 2023-11-15 18:39:56
  • James A. Michaelov, Catherine Arnett, Tyler A. Chang, Benjamin K. Bergen
  • 0


Abstract grammatical knowledge - of parts of speech and grammatical patterns- is key to the capacity for linguistic generalization in humans. But howabstract is grammatical knowledge in large language models? In the humanliterature, compelling evidence for grammatical abstraction comes fromstructural priming. A sentence that shares the same grammatical structure as apreceding sentence is processed and produced more readily. Because confoundsexist when using stimuli in a single language, evidence of abstraction is evenmore compelling from crosslingual structural priming, where use of a syntacticstructure in one language primes an analogous structure in another language. Wemeasure crosslingual structural priming in large language models, comparingmodel behavior to human experimental results from eight crosslingualexperiments covering six languages, and four monolingual structural primingexperiments in three non-English languages. We find evidence for abstractmonolingual and crosslingual grammatical representations in the models thatfunction similarly to those found in humans. These results demonstrate thatgrammatical representations in multilingual language models are not onlysimilar across languages, but they can causally influence text produced indifferent languages.


Quick Read (beta)

loading the full paper ...