Abstract
Neural network pruning can be effectively applied to compress automaticspeech recognition (ASR) models. However, in multilingual ASR, performinglanguage-agnostic pruning may lead to severe performance degradation on somelanguages because language-agnostic pruning masks may not fit all languages anddiscard important language-specific parameters. In this work, we present ASRpathways, a sparse multilingual ASR model that activates language-specificsub-networks ("pathways"), such that the parameters for each language arelearned explicitly. With the overlapping sub-networks, the shared parameterscan also enable knowledge transfer for lower resource languages via jointmultilingual training. We propose a novel algorithm to learn ASR pathways, andevaluate the proposed method on 4 languages with a streaming RNN-T model. Ourproposed ASR pathways outperform both dense models (-5.0% average WER) and alanguage-agnostically pruned model (-21.4% average WER), and provide betterperformance on low-resource languages compared to the monolingual sparsemodels.