Language Scaling for Universal Suggested Replies Model

  • 2021-06-04 03:15:52
  • Qianlan Ying, Payal Bajaj, Budhaditya Deb, Yu Yang, Wei Wang, Bojia Lin, Milad Shokouhi, Xia Song, Yang Yang, Daxin Jiang
  • 0

Abstract

We consider the problem of scaling automated suggested replies for Outlookemail system to multiple languages. Faced with increased compute requirementsand low resources for language expansion, we build a single universal model forimproving the quality and reducing run-time costs of our production system.However, restricted data movement across regional centers prevents jointtraining across languages. To this end, we propose a multi-task continuallearning framework, with auxiliary tasks and language adapters to learnuniversal language representation across regions. The experimental results showpositive cross-lingual transfer across languages while reducing catastrophicforgetting across regions. Our online results on real user traffic showsignificant gains in CTR and characters saved, as well as 65% training costreduction compared with per-language models. As a consequence, we have scaledthe feature in multiple languages including low-resource markets.

 

Quick Read (beta)

loading the full paper ...