Abstract
This study tackles unsupervised domain adaptation of reading comprehension(UDARC). Reading comprehension (RC) is a task to learn the capability forquestion answering with textual sources. State-of-the-art models on RC still donot have general linguistic intelligence; i.e., their accuracy worsens forout-domain datasets that are not used in the training. We hypothesize that thisdiscrepancy is caused by a lack of the language modeling (LM) capability forthe out-domain. The UDARC task allows models to use supervised RC training datain the source domain and only unlabeled passages in the target domain. To solvethe UDARC problem, we provide two domain adaptation models. The first onelearns the out-domain LM and in-domain RC task sequentially. The second one isthe proposed model that uses a multi-task learning approach of LM and RC. Themodels can retain both the RC capability acquired from the supervised data inthe source domain and the LM capability from the unlabeled data in the targetdomain. We evaluated the models on UDARC with five datasets in differentdomains. The models outperformed the model without domain adaptation. Inparticular, the proposed model yielded an improvement of 4.3/4.2 points inEM/F1 in an unseen biomedical domain.
Quick Read (beta)
Unsupervised Domain Adaptation of Language Models
for Reading Comprehension
Abstract
Input: \algrenewcommand\algorithmicensureOutput: