Positional Attention-based Frame Identification with BERT: A Deep Learning Approach to Target Disambiguation and Semantic Frame Selection

  • 2019-10-31 15:51:04
  • Sang-Sang Tan, Jin-Cheon Na
  • 0

Abstract

Semantic parsing is the task of transforming sentences from natural languageinto formal representations of predicate-argument structures. Under thisresearch area, frame-semantic parsing has attracted much interest. This parsingapproach leverages the lexical information defined in FrameNet to associatemarked predicates or targets with semantic frames, thereby assigning semanticroles to sentence components based on pre-specified frame elements in FrameNet.In this paper, a deep neural network architecture known as PositionalAttention-based Frame Identification with BERT (PAFIBERT) is presented as asolution to the frame identification subtask in frame-semantic parsing.Although the importance of this subtask is well-established, prior research hasyet to find a robust solution that works satisfactorily for both in-domain andout-of-domain data. This study thus set out to improve frame identification inlight of recent advancements of language modeling and transfer learning innatural language processing. The proposed method is partially empowered byBERT, a pre-trained language model that excels at capturing contextualinformation in texts. By combining the language representation power of BERTwith a position-based attention mechanism, PAFIBERT is able to attend totarget-specific contexts in sentences for disambiguating targets andassociating them with the most suitable semantic frames. Under variousexperimental settings, PAFIBERT outperformed existing solutions by asignificant margin, achieving new state-of-the-art results for both in-domainand out-of-domain benchmark test sets.

 

Quick Read (beta)

loading the full paper ...