Speech Model Pre-training for End-to-End Spoken Language Understanding

  • 2019-04-07 15:24:32
  • Loren Lugosch, Mirco Ravanelli, Patrick Ignoto, Vikrant Singh Tomar, Yoshua Bengio
  • 63


Whereas conventional spoken language understanding (SLU) systems map speechto text, and then text to intent, end-to-end SLU systems map speech directly tointent through a single trainable model. Achieving high accuracy with theseend-to-end models without a large amount of training data is difficult. Wepropose a method to reduce the data requirements of end-to-end SLU in which themodel is first pre-trained to predict words and phonemes, thus learning goodfeatures for SLU. We introduce a new SLU dataset, Fluent Speech Commands, andshow that our method improves performance both when the full dataset is usedfor training and when only a small subset is used. We also describe preliminaryexperiments to gauge the model's ability to generalize to new phrases not heardduring training.


Introduction (beta)



Conclusion (beta)