Abstract
Span-extraction reading comprehension models have made tremendous advancesenabled by the availability of large-scale, high-quality training datasets.Despite such rapid progress and widespread application, extractive readingcomprehension datasets in languages other than English remain scarce, andcreating such a sufficient amount of training data for each language is costlyand even impossible. An alternative to creating large-scale high-qualitymonolingual span-extraction training datasets is to develop multilingualmodeling approaches and systems which can transfer to the target languagewithout requiring training data in that language. In this paper, in order tosolve the scarce availability of extractive reading comprehension training datain the target language, we propose a multilingual extractive readingcomprehension approach called XLRC by simultaneously modeling the existingextractive reading comprehension training data in a multilingual environmentusing self-adaptive attention and multilingual attention. Specifically, wefirstly construct multilingual parallel corpora by translating the existingextractive reading comprehension datasets (i.e., CMRC 2018) from the targetlanguage (i.e., Chinese) into different language families (i.e., English).Secondly, to enhance the final target representation, we adopt self-adaptiveattention (SAA) to combine self-attention and inter-attention to extract thesemantic relations from each pair of the target and source languages.Furthermore, we propose multilingual attention (MLA) to learn the richknowledge from various language families. Experimental results show that ourmodel outperforms the state-of-the-art baseline (i.e., RoBERTa_Large) on theCMRC 2018 task, which demonstrate the effectiveness of our proposedmulti-lingual modeling approach and show the potentials in multilingual NLPtasks.