Language-Independent Representor for Neural Machine Translation

  • 2018-11-01 06:47:23
  • Long Zhou, Yuchen Liu, Jiajun Zhang, Chengqing Zong, Guoping Huang
  • 10


Current Neural Machine Translation (NMT) employs a language-specific encoderto represent the source sentence and adopts a language-specific decoder togenerate target translation. This language-dependent design leads tolarge-scale network parameters and makes the duality of the parallel dataunderutilized. To address the problem, we propose in this paper alanguage-independent representor to replace the encoder and decoder by usingweight sharing. This shared representor can not only reduce large portion ofnetwork parameters, but also facilitate us to fully explore the languageduality by jointly training source-to-target, target-to-source, left-to-rightand right-to-left translations within a multi-task learning framework.Experiments show that our proposed framework can obtain significantimprovements over conventional NMT models on resource-rich and low-resourcetranslation tasks with only a quarter of parameters.


Introduction (beta)



Conclusion (beta)