Abstract
Multi-choice Machine Reading Comprehension (MRC) requires model to decide thecorrect answer from a set of answer options when given a passage and aquestion. Thus in addition to a powerful pre-trained Language Model as encoder,multi-choice MRC especially relies on a matching network design which issupposed to effectively capture the relationship among the triplet of passage,question and answers. While the latest pre-trained Language Models have shownpowerful enough even without the support from a matching network, and thelatest matching network has been complicated enough, we thus propose a novelgoing-back-to-the-basic solution which straightforwardly models the MRCrelationship as attention mechanism inside network. The proposed DUalMulti-head Co-Attention (DUMA) has been shown simple but effective and iscapable of generally promoting pre-trained Language Models. Our proposed methodis evaluated on two benchmark multi-choice MRC tasks, DREAM and RACE, showingthat in terms of strong Language Models, DUMA may still boost the model toreach new state-of-the-art performance.