Language Embeddings for Typology and Cross-lingual Transfer Learning

  • 2021-06-03 19:00:02
  • Dian Yu, Taiqi He, Kenji Sagae
  • 2

Abstract

Cross-lingual language tasks typically require a substantial amount ofannotated data or parallel translation data. We explore whether languagerepresentations that capture relationships among languages can be learned andsubsequently leveraged in cross-lingual tasks without the use of parallel data.We generate dense embeddings for 29 languages using a denoising autoencoder,and evaluate the embeddings using the World Atlas of Language Structures (WALS)and two extrinsic tasks in a zero-shot setting: cross-lingual dependencyparsing and cross-lingual natural language inference.

 

Quick Read (beta)

loading the full paper ...