There is No Big Brother or Small Brother: Knowledge Infusion in Language Models for Link Prediction and Question Answering

  • 2023-01-10 14:59:33
  • Ankush Agarwal, Sakharam Gawade, Sachin Channabasavarajendra, Pushpak Bhattacharyya
  • 0

Abstract

The integration of knowledge graphs with deep learning is thriving inimproving the performance of various natural language processing (NLP) tasks.In this paper, we focus on knowledge-infused link prediction and questionanswering using language models, T5, and BLOOM across three domains: Aviation,Movie, and Web. In this context, we infuse knowledge in large and smalllanguage models and study their performance, and find the performance to besimilar. For the link prediction task on the Aviation Knowledge Graph, weobtain a 0.2 [email protected] score using T5-small, T5-base, T5-large, and BLOOM. Usingtemplate-based scripts, we create a set of 1 million synthetic factoid QA pairsin the aviation domain from National Transportation Safety Board (NTSB)reports. On our curated QA pairs, the three models of T5 achieve a 0.7 [email protected] We validate out findings with the paired student t-test and Cohen'skappa scores. For link prediction on Aviation Knowledge Graph using T5-smalland T5-large, we obtain a Cohen's kappa score of 0.76, showing substantialagreement between the models. Thus, we infer that small language models performsimilar to large language models with the infusion of knowledge.

 

Quick Read (beta)

loading the full paper ...