Abstract
LSTM based language models are an important part of modern LVCSR systems asthey significantly improve performance over traditional backoff languagemodels. Incorporating them efficiently into decoding has been notoriouslydifficult. In this paper we present an approach based on a combination ofone-pass decoding and lattice rescoring. We perform decoding with the LSTM-LMin the first pass but recombine hypothesis that share the last two words,afterwards we rescore the resulting lattice. We run our systems on GPGPUequipped machines and are able to produce competitive results on the Hub5'00and Librispeech evaluation corpora with a runtime better than real-time. Inaddition we shortly investigate the possibility to carry out the full sum overall state-sequences belonging to a given word-hypothesis during decodingwithout recombination.