Choosing the Right Word: Using Bidirectional LSTM Tagger for Writing Support Systems

  • 2019-01-08 19:59:33
  • Victor Makarenkov, Lior Rokach, Bracha Shapira
  • 12

Abstract

Scientific writing is difficult. It is even harder for those for whom Englishis a second language (ESL learners). Scholars around the world spend asignificant amount of time and resources proofreading their work beforesubmitting it for review or publication. In this paper we present a novel machine learning based application forproper word choice task. Proper word choice is a generalization the lexicalsubstitution (LS) and grammatical error correction (GEC) tasks. We demonstrateand evaluate the usefulness of applying bidirectional Long Short Term Memory(LSTM) tagger, for this task. While state-of-the-art grammatical errorcorrection uses error-specific classifiers and machine translation methods, wedemonstrate an unsupervised method that is based solely on a high quality textcorpus and does not require manually annotated data. We use a bidirectionalRecurrent Neural Network (RNN) with LSTM for learning the proper word choicebased on a word's sentential context. We demonstrate and evaluate ourapplication on both a domain-specific (scientific), writing task and ageneral-purpose writing task. We show that our domain-specific andgeneral-purpose models outperform state-of-the-art general context learning. Asan additional contribution of this research, we also share our code,pre-trained models, and a new ESL learner test set with the research community.

 

Quick Read (beta)

loading the full paper ...