Importance-based Neuron Allocation for Multilingual Neural Machine Translation

  • 2021-07-14 09:15:05
  • Wanying Xie, Yang Feng, Shuhao Gu, Dong Yu
  • 2


Multilingual neural machine translation with a single model has drawn muchattention due to its capability to deal with multiple languages. However, thecurrent multilingual translation paradigm often makes the model tend topreserve the general knowledge, but ignore the language-specific knowledge.Some previous works try to solve this problem by adding various kinds oflanguage-specific modules to the model, but they suffer from the parameterexplosion problem and require specialized manual design. To solve theseproblems, we propose to divide the model neurons into general andlanguage-specific parts based on their importance across languages. The generalpart is responsible for preserving the general knowledge and participating inthe translation of all the languages, while the language-specific part isresponsible for preserving the language-specific knowledge and participating inthe translation of some specific languages. Experimental results on severallanguage pairs, covering IWSLT and Europarl corpus datasets, demonstrate theeffectiveness and universality of the proposed method.


Quick Read (beta)

loading the full paper ...