Abstract
The exponential rise of social media and digital news in the past decade hashad the unfortunate consequence of escalating what the United Nations hascalled a global topic of concern: the growing prevalence of disinformation.Given the complexity and time-consuming nature of combating disinformationthrough human assessment, one is motivated to explore harnessing AI solutionsto automatically assess news articles for the presence of disinformation. Avaluable first step towards automatic identification of disinformation isstance detection, where given a claim and a news article, the aim is to predictif the article agrees, disagrees, takes no position, or is unrelated to theclaim. Existing approaches in literature have largely relied on hand-engineeredfeatures or shallow learned representations (e.g., word embeddings) to encodethe claim-article pairs, which can limit the level of representationalexpressiveness needed to tackle the high complexity of disinformationidentification. In this work, we explore the notion of harnessing large-scaledeep bidirectional transformer language models for encoding claim-article pairsin an effort to construct state-of-the-art stance detection geared foridentifying disinformation. Taking advantage of bidirectional cross-attentionbetween claim-article pairs via pair encoding with self-attention, we constructa large-scale language model for stance detection by performing transferlearning on a RoBERTa deep bidirectional transformer language model, and wereable to achieve state-of-the-art performance (weighted accuracy of 90.01%) onthe Fake News Challenge Stage 1 (FNC-I) benchmark. These promising resultsserve as motivation for harnessing such large-scale language models as powerfulbuilding blocks for creating effective AI solutions to combat disinformation.