Cross-lingual Transfer of Abstractive Summarizer to Less-resource Language

  • 2021-09-02 13:54:06
  • Aleš Žagar, Marko Robnik-Šikonja
  • 0

Abstract

Automatic text summarization extracts important information from texts andpresents the information in the form of a summary. Abstractive summarizationapproaches progressed significantly by switching to deep neural networks, butresults are not yet satisfactory, especially for languages where large trainingsets do not exist. In several natural language processing tasks, across-lingual model transfer is successfully applied in less-resourcelanguages. For summarization, the cross-lingual model transfer was notattempted due to a non-reusable decoder side of neural models that cannotcorrect target language generation. In our work, we use a pre-trained Englishsummarization model based on deep neural networks and sequence-to-sequencearchitecture to summarize Slovene news articles. We address the problem ofinadequate decoder by using an additional language model for the evaluation ofthe generated text in target language. We test several cross-lingualsummarization models with different amounts of target data for fine-tuning. Weassess the models with automatic evaluation measures and conduct a small-scalehuman evaluation. Automatic evaluation shows that the summaries of our bestcross-lingual model are useful and of quality similar to the model trained onlyin the target language. Human evaluation shows that our best model generatessummaries with high accuracy and acceptable readability. However, similar toother abstractive models, our models are not perfect and may occasionallyproduce misleading or absurd content.

 

Quick Read (beta)

loading the full paper ...