Curriculum-Meta Learning for Order-Robust Continual Relation Extraction

  • 2021-01-06 08:52:34
  • Tongtong Wu, Xuekai Li, Yuan-Fang Li, Reza Haffari, Guilin Qi, Yujin Zhu, Guoqiang Xu
  • 22

Abstract

Continual relation extraction is an important task that focuses on extractingnew facts incrementally from unstructured text. Given the sequential arrivalorder of the relations, this task is prone to two serious challenges, namelycatastrophic forgetting and order-sensitivity. We propose a novelcurriculum-meta learning method to tackle the above two challenges in continualrelation extraction. We combine meta learning and curriculum learning toquickly adapt model parameters to a new task and to reduce interference ofpreviously seen tasks on the current task. We design a novel relationrepresentation learning method through the distribution of domain and rangetypes of relations. Such representations are utilized to quantify thedifficulty of tasks for the construction of curricula. Moreover, we alsopresent novel difficulty-based metrics to quantitatively measure the extent oforder-sensitivity of a given model, suggesting new ways to evaluate modelrobustness. Our comprehensive experiments on three benchmark datasets show thatour proposed method outperforms the state-of-the-art techniques. The code isavailable at the anonymous GitHub repository:https://github.com/wutong8023/AAAI_CML.

 

Quick Read (beta)

loading the full paper ...