IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding

  • 2020-09-11 12:21:41
  • Bryan Wilie, Karissa Vincentio, Genta Indra Winata, Samuel Cahyawijaya, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, Ayu Purwarianti
  • 2

Abstract

Although Indonesian is known to be the fourth most frequently used languageover the internet, the research progress on this language in the naturallanguage processing (NLP) is slow-moving due to a lack of available resources.In response, we introduce the first-ever vast resource for the training,evaluating, and benchmarking on Indonesian natural language understanding(IndoNLU) tasks. IndoNLU includes twelve tasks, ranging from single sentenceclassification to pair-sentences sequence labeling with different levels ofcomplexity. The datasets for the tasks lie in different domains and styles toensure task diversity. We also provide a set of Indonesian pre-trained models(IndoBERT) trained from a large and clean Indonesian dataset Indo4B collectedfrom publicly available sources such as social media texts, blogs, news, andwebsites. We release baseline models for all twelve tasks, as well as theframework for benchmark evaluation, and thus it enables everyone to benchmarktheir system performances.

 

Quick Read (beta)

loading the full paper ...