Using Contextual Information for Sentence-level Morpheme Segmentation

  • 2024-05-14 18:22:05
  • Prabin Bhandari, Abhishek Paudel
  • 0

Abstract

Recent advancements in morpheme segmentation primarily emphasize word-levelsegmentation, often neglecting the contextual relevance within the sentence. Inthis study, we redefine the morpheme segmentation task as asequence-to-sequence problem, treating the entire sentence as input rather thanisolating individual words. Our findings reveal that the multilingual modelconsistently exhibits superior performance compared to monolingualcounterparts. While our model did not surpass the performance of the currentstate-of-the-art, it demonstrated comparable efficacy with high-resourcelanguages while revealing limitations in low-resource language scenarios.

 

Quick Read (beta)

loading the full paper ...