Abstract
Language coverage bias, which indicates the content-dependent differencesbetween sentence pairs originating from the source and target languages, isimportant for neural machine translation (NMT) because the target-originaltraining data is not well exploited in current practice. By carefully designingexperiments, we provide comprehensive analyses of the language coverage bias inthe training data, and find that using only the source-original data achievescomparable performance with using full training data. Based on theseobservations, we further propose two simple and effective approaches toalleviate the language coverage bias problem through explicitly distinguishingbetween the source- and target-original training data, which consistentlyimprove the performance over strong baselines on six WMT20 translation tasks.Complementary to the translationese effect, language coverage bias providesanother explanation for the performance drop caused by back-translation. Wealso apply our approach to both back- and forward-translation and find thatmitigating the language coverage bias can improve the performance of both thetwo representative data augmentation methods and their tagged variants.