Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

  • 2020-10-15 17:14:09
  • Ekta Sood, Simon Tannert, Philipp Mueller, Andreas Bulling
  • 7


A lack of corpora has so far limited advances in integrating human gaze dataas a supervisory signal in neural attention mechanisms for natural languageprocessing(NLP). We propose a novel hybrid text saliency model (TSM) that, forthe first time, combines a cognitive model of reading with explicit human gazesupervision in a single machine learning framework. On four different corporawe demonstrate that our hybrid TSM duration predictions are highly correlatedwith human gaze ground truth. We further propose a novel joint modellingapproach to integrate TSM predictions into the attention layer of a networkdesigned for a specific upstream NLP task without the need for anytask-specific human gaze data. We demonstrate that our joint model outperformsthe state of the art in paraphrase generation on the Quora Question Pairscorpus by more than 10% in BLEU-4 and achieves state-of-the-art performance forsentence compression on the challenging Google Sentence Compression corpus. Assuch, our work introduces a practical approach for bridging between data-drivenand cognitive models and demonstrates a new way to integrate human gaze-guidedneural attention into NLP tasks.


Quick Read (beta)

loading the full paper ...