Abstract

For general modeling methods applied to diverse languages, a natural questionis: how well should we expect our models to work on languages with differingtypological profiles? In this work, we develop an evaluation framework for faircross-linguistic comparison of language models, using translated text so thatall models are asked to predict approximately the same information. We thenconduct a study on 21 languages, demonstrating that in some languages, thetextual expression of the information is harder to predict with both $n$-gramand LSTM language models. We show complex inflectional morphology to be a causeof performance differences among languages.

 

Quick Read (beta)