Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)

  • 2019-06-06 14:52:22
  • Mariya Toneva, Leila Wehbe
  • 0

Abstract

Neural network models for NLP are typically implemented without the explicitencoding of language rules and yet they are able to break one performancerecord after another. Despite much work, it is still unclear what therepresentations learned by these networks correspond to. We propose here anovel approach for interpreting neural networks that relies on the onlyprocessing system we have that does understand language: the human brain. Weuse brain imaging recordings of subjects reading complex natural text tointerpret word and sequence embeddings from 4 recent NLP models - ELMo, USE,BERT and Transformer-XL. We study how their representations differ across layerdepth, context length, and attention type. Our results reveal differences inthe context-related representations across these models. Further, in thetransformer models, we find an interaction between layer depth and contextlength, and between layer depth and attention type. We finally use the insightsfrom the attention experiments to alter BERT: we remove the learned attentionat shallow layers, and show that this manipulation improves performance on awide range of syntactic tasks. Cognitive neuroscientists have already begunusing NLP networks to study the brain, and this work closes the loop to allowthe interaction between NLP and cognitive neuroscience to be a truecross-pollination.

 

Quick Read (beta)

loading the full paper ...