Multilingual NMT with a language-independent attention bridge

  • 2018-11-01 17:06:09
  • Raúl Vázquez, Alessandro Raganato, Jörg Tiedemann, Mathias Creutz
  • 8

Abstract

In this paper, we propose a multilingual encoder-decoder architecture capableof obtaining multilingual sentence representations by means of incorporating anintermediate {\em attention bridge} that is shared across all languages. Thatis, we train the model with language-specific encoders and decoders that areconnected via self-attention with a shared layer that we call attention bridge.This layer exploits the semantics from each language for performing translationand develops into a language-independent meaning representation that canefficiently be used for transfer learning. We present a new framework for theefficient development of multilingual NMT using this model and scheduledtraining. We have tested the approach in a systematic way with a multi-paralleldata set. We show that the model achieves substantial improvements over strongbilingual models and that it also works well for zero-shot translation, whichdemonstrates its ability of abstraction and transfer learning.

 

Introduction (beta)

None

 

Conclusion (beta)

None