Abstract
Speculation is a naturally occurring phenomena in textual data, forming anintegral component of many systems, especially in the biomedical informationretrieval domain. Previous work addressing cue detection and scope resolution(the two subtasks of speculation detection) have ranged from rule-based systemsto deep learning-based approaches. In this paper, we apply three populartransformer-based architectures, BERT, XLNet and RoBERTa to this task, on twopublicly available datasets, BioScope Corpus and SFU Review Corpus, reportingsubstantial improvements over previously reported results (by at least 0.29 F1points on cue detection and 4.27 F1 points on scope resolution). We alsoexperiment with joint training of the model on multiple datasets, whichoutperforms the single dataset training approach by a good margin. We observethat XLNet consistently outperforms BERT and RoBERTa, contrary to results onother benchmark datasets. To confirm this observation, we apply XLNet andRoBERTa to negation detection and scope resolution, reporting state-of-the-artresults on negation scope resolution for the BioScope Corpus (increase of 3.16F1 points on the BioScope Full Papers, 0.06 F1 points on the BioScopeAbstracts) and the SFU Review Corpus (increase of 0.3 F1 points).