Multi-cell LSTM Based Neural Language Model

  • 2018-11-15 17:09:53
  • Thomas Cherian, Akshay Badola, Vineet Padmanabhan
  • 7

Abstract

Language models, being at the heart of many NLP problems, are always of greatinterest to researchers. Neural language models come with the advantage ofdistributed representations and long range contexts. With its particulardynamics that allow the cycling of information within the network, `Recurrentneural network' (RNN) becomes an ideal paradigm for neural language modeling.Long Short-Term Memory (LSTM) architecture solves the inadequacies of thestandard RNN in modeling long-range contexts. In spite of a plethora of RNNvariants, possibility to add multiple memory cells in LSTM nodes was seldomexplored. Here we propose a multi-cell node architecture for LSTMs and studyits applicability for neural language modeling. The proposed multi-cell LSTMlanguage models outperform the state-of-the-art results on well-known PennTreebank (PTB) setup.

 

Quick Read (beta)

loading the full paper ...