Abstract
Universal language representation is the holy grail in machine translation(MT). Thanks to the new neural MT approach, it seems that there are goodperspectives towards this goal. In this paper, we propose a new architecturebased on combining variational autoencoders with encoder-decoders andintroducing an interlingual loss as an additional training objective. By addingand forcing this interlingual loss, we are able to train multiple encoders anddecoders for each language, sharing a common universal representation. Sincethe final objective of this universal representation is producing close resultsfor similar input sentences (in any language), we propose to evaluate it byencoding the same sentence in two different languages, decoding both latentrepresentations into the same language and comparing both outputs. Preliminaryresults on the WMT 2017 Turkish/English task shows that the proposedarchitecture is capable of learning a universal language representation andsimultaneously training both translation directions with state-of-the-artresults.