Abstract
Identifying factors that make certain languages harder to model than othersis essential to reach language equality in future Natural Language Processingtechnologies. Free-order case-marking languages, such as Russian, Latin orTamil, have proved more challenging than fixed-order languages for the tasks ofsyntactic parsing and subject-verb agreement prediction. In this work, weinvestigate whether this class of languages is also more difficult to translateby state-of-the-art Neural Machine Translation models (NMT). Using a variety ofsynthetic languages and a newly introduced translation challenge set, we findthat word order flexibility in the source language only leads to a very smallloss of NMT quality, even though the core verb arguments become impossible todisambiguate in sentences without semantic cues. The latter issue is indeedsolved by the addition of case marking. However, in medium- and low-resourcesettings, the overall NMT quality of fixed-order languages remains unmatched.