Abstract
This work explores better adaptation methods to low-resource languages usingan external language model (LM) under the framework of transfer learning. Wefirst build a language-independent ASR system in a unified sequence-to-sequence(S2S) architecture with a shared vocabulary among all languages. Duringadaptation, we perform LM fusion transfer, where an external LM is integratedinto the decoder network of the attention-based S2S model in the wholeadaptation stage, to effectively incorporate linguistic context of the targetlanguage. We also investigate various seed models for transfer learning.Experimental evaluations using the IARPA BABEL data set show that LM fusiontransfer improves performances on all target five languages compared withsimple transfer learning when the external text data is available. Our finalsystem drastically reduces the performance gap from the hybrid systems.