Abstract
Reading comprehension (RC) has been studied in a variety of datasets with theboosted performance brought by deep neural networks. However, thegeneralization capability of these models across different domains remainsunclear. To alleviate this issue, we are going to investigate unsuperviseddomain adaptation on RC, wherein a model is trained on labeled source domainand to be applied to the target domain with only unlabeled samples. We firstshow that even with the powerful BERT contextual representation, theperformance is still unsatisfactory when the model trained on one dataset isdirectly applied to another target dataset. To solve this, we provide a novelconditional adversarial self-training method (CASe). Specifically, our approachleverages a BERT model fine-tuned on the source dataset along with theconfidence filtering to generate reliable pseudo-labeled samples in the targetdomain for self-training. On the other hand, it further reduces domaindistribution discrepancy through conditional adversarial learning acrossdomains. Extensive experiments show our approach achieves comparable accuracyto supervised models on multiple large-scale benchmark datasets.