Abstract
Neural network-based language models are commonly used in rescoringapproaches to improve the quality of modern automatic speech recognition (ASR)systems. Most of the existing methods are computationally expensive since theyuse autoregressive language models. We propose a novel rescoring approach,which processes the entire lattice in a single call to the model. The keyfeature of our rescoring policy is a novel non-autoregressive LatticeTransformer Language Model (LT-LM). This model takes the whole lattice as aninput and predicts a new language score for each arc. Additionally, we proposethe artificial lattices generation approach to incorporate a large amount oftext data in the LT-LM training process. Our single-shot rescoring performsorders of magnitude faster than other rescoring methods in our experiments. Itis more than 300 times faster than pruned RNNLM lattice rescoring and N-bestrescoring while slightly inferior in terms of WER.