Abstract
Many natural languages assign grammatical gender also to inanimate nouns inthe language. In such languages, words that relate to the gender-marked nounsare inflected to agree with the noun's gender. We show that this affects theword representations of inanimate nouns, resulting in nouns with the samegender being closer to each other than nouns with different gender. While"embedding debiasing" methods fail to remove the effect, we demonstrate that acareful application of methods that neutralize grammatical gender signals fromthe words' context when training word embeddings is effective in removing it.Fixing the grammatical gender bias yields a positive effect on the quality ofthe resulting word embeddings, both in monolingual and cross-lingual settings.We note that successfully removing gender signals, while achievable, is nottrivial to do and that a language-specific morphological analyzer, togetherwith careful usage of it, are essential for achieving good results.