Abstract

Many problems in NLP require aggregating information from multiple mentionsof the same entity which may be far apart in the text. Existing RecurrentNeural Network (RNN) layers are biased towards short-term dependencies andhence not suited to such tasks. We present a recurrent layer which is insteadbiased towards coreferent dependencies. The layer uses coreference annotationsextracted from an external system to connect entity mentions belonging to thesame cluster. Incorporating this layer into a state-of-the-art readingcomprehension model improves performance on three datasets -- Wikihop, LAMBADAand the bAbi AI tasks -- with large gains when training data is scarce.

 

Quick Read (beta)