Abstract

This work focuses on building language models (LMs) for code-switched text.We propose two techniques that significantly improve these LMs: 1) A novelrecurrent neural network unit with dual components that focus on each languagein the code-switched text separately 2) Pretraining the LM using synthetic textfrom a generative model estimated using the training data. We demonstrate theeffectiveness of our proposed techniques by reporting perplexities on aMandarin-English task and derive significant reductions in perplexity.

 

Quick Read (beta)