Abstract
Although spoken language understanding (SLU) has achieved great success inhigh-resource languages, such as English, it remains challenging inlow-resource languages mainly due to the lack of high quality training data.The recent multilingual code-switching approach samples some words in an inpututterance and replaces them by expressions in some other languages of the samemeaning. The multilingual code-switching approach achieves better alignments ofrepresentations across languages in zero-shot cross-lingual SLU. Surprisingly,all existing multilingual code-switching methods disregard the inherentsemantic structure in SLU, i.e., most utterances contain one or more slots, andeach slot consists of one or more words. In this paper, we propose to exploitthe "utterance-slot-word" structure of SLU and systematically model thisstructure by a multi-level contrastive learning framework at the utterance,slot, and word levels. We develop novel code-switching schemes to generate hardnegative examples for contrastive learning at all levels. Furthermore, wedevelop a label-aware joint model to leverage label semantics for cross-lingualknowledge transfer. Our experimental results show that our proposed methodssignificantly improve the performance compared with the strong baselines on twozero-shot cross-lingual SLU benchmark datasets.