Abstract
Existing end-to-end neural network models for extractive ReadingComprehension (RC) have enjoyed the benefit of a large amount of hand-annotatedtraining data. However, such a dataset is usually available only in English,which limits one from building an extractive RC model for a language ofinterest. In this paper, we introduce the first extractive RC systems fornon-English languages without using language-specific RC training data, butinstead by using an English RC model and an attention-based Neural MachineTranslation (NMT) model. To train the NMT model for specific languagedirections, we take advantage of constantly growing web resources toautomatically construct parallel corpora, rather than assuming the availabilityof high quality parallel corpora of the target domain. Our method firsttranslates a paragraph-question pair into English so that the Englishextractive RC model can output its answer. The attention mechanism in the NMTmodel is further used to directly align the answer in the target text ofinterest. Experimental results in two non-English languages, namely Japaneseand French, show that our method significantly outperforms a back-translationbaseline of a state-of-the-art product-level machine translation system.Moreover, our ablation studies suggest that adding a small number of manuallytranslated questions, besides an automatically created corpus, could furtherimprove the performance of the extractive RC systems for non-English languages.