Abstract

Masked language models have revolutionized natural language processingsystems in the past few years. A recently introduced generalization of maskedlanguage models called warped language models are trained to be more robust tothe types of errors that appear in automatic or manual transcriptions of spokenlanguage by exposing the language model to the same types of errors duringtraining. In this work we propose a novel approach that takes advantage of therobustness of warped language models to transcription noise for correctingtranscriptions of spoken language. We show that our proposed approach is ableto achieve up to 10% reduction in word error rates of both automatic and manualtranscriptions of spoken language.

 

Quick Read (beta)