Abstract
Low-resource languages such as Filipino suffer from data scarcity which makesit challenging to develop NLP applications for Filipino language. The use ofTransfer Learning (TL) techniques alleviates this problem in low-resourcesetting. In recent years, transformer-based models are proven to be effectivein low-resource tasks but faces challenges in accessibility due to its highcompute and memory requirements. For this reason, there's a need for a cheaperbut effective alternative. This paper has three contributions. First, release apre-trained AWD-LSTM language model for Filipino language. Second, benchmarkAWD-LSTM in the Hate Speech classification task and show that it performs onpar with transformer-based models. Third, analyze the the performance ofAWD-LSTM in low-resource setting using degradation test and compare it withtransformer-based models. ----- Ang mga low-resource languages tulad ng Filipino ay gipit sa accessible nadatos kaya't mahirap gumawa ng mga applications sa wikang ito. Ang mga TransferLearning (TL) techniques ay malaking tulong para sa low-resource setting o mgapagkakataong gipit sa datos. Sa mga nagdaang taon, nanaig ang mgatransformer-based TL techniques pagdating sa low-resource tasks ngunit ito aymataas na compute and memory requirements kaya nangangailangan ng mas mura peroepektibong alternatibo. Ang papel na ito ay may tatlong kontribusyon. Una,maglabas ng pre-trained AWD-LSTM language model sa wikang Filipino upang magingtuntungan sa pagbuo ng mga NLP applications sa wikang Filipino. Pangalawa, magbenchmark ng AWD-LSTM sa Hate Speech classification task at ipakita na kayangnitong makipagsabayan sa mga transformer-based models. Pangatlo, suriin angperformance ng AWD-LSTM sa low-resource setting gamit ang degradation test atikumpara ito sa mga transformer-based models.