ExBERT: An External Knowledge Enhanced BERT for Natural Language Inference

  • 2021-08-03 15:56:49
  • Amit Gajbhiye, Noura Al Moubayed, Steven Bradley
  • 1

Abstract

Neural language representation models such as BERT, pre-trained onlarge-scale unstructured corpora lack explicit grounding to real-worldcommonsense knowledge and are often unable to remember facts required forreasoning and inference. Natural Language Inference (NLI) is a challengingreasoning task that relies on common human understanding of language andreal-world commonsense knowledge. We introduce a new model for NLI calledExternal Knowledge Enhanced BERT (ExBERT), to enrich the contextualrepresentation with real-world commonsense knowledge from external knowledgesources and enhance BERT's language understanding and reasoning capabilities.ExBERT takes full advantage of contextual word representations obtained fromBERT and employs them to retrieve relevant external knowledge from knowledgegraphs and to encode the retrieved external knowledge. Our model adaptivelyincorporates the external knowledge context required for reasoning over theinputs. Extensive experiments on the challenging SciTail and SNLI benchmarksdemonstrate the effectiveness of ExBERT: in comparison to the previousstate-of-the-art, we obtain an accuracy of 95.9% on SciTail and 91.5% on SNLI.

 

Quick Read (beta)

loading the full paper ...