An Empirical Study of Generation Order for Machine Translation

  • 2019-10-29 17:54:36
  • William Chan, Mitchell Stern, Jamie Kiros, Jakob Uszkoreit
  • 34

Abstract

In this work, we present an empirical study of generation order for machinetranslation. Building on recent advances in insertion-based modeling, we firstintroduce a soft order-reward framework that enables us to train models tofollow arbitrary oracle generation policies. We then make use of this frameworkto explore a large variety of generation orders, including uninformed orders,location-based orders, frequency-based orders, content-based orders, andmodel-based orders. Curiously, we find that for the WMT'14 English $\to$ Germantranslation task, order does not have a substantial impact on output quality,with unintuitive orderings such as alphabetical and shortest-first matching theperformance of a standard Transformer. This demonstrates that traditionalleft-to-right generation is not strictly necessary to achieve high performance.On the other hand, results on the WMT'18 English $\to$ Chinese task tend tovary more widely, suggesting that translation for less well-aligned languagepairs may be more sensitive to generation order.

 

Quick Read (beta)

loading the full paper ...