Transfer Learning from LDA to BiLSTM-CNN for Offensive Language Detection in Twitter

  • 2018-11-07 14:40:10
  • Gregor Wiedemann, Eugen Ruppert, Raghav Jindal, Chris Biemann
  • 1


We investigate different strategies for automatic offensive languageclassification on German Twitter data. For this, we employ a sequentiallycombined BiLSTM-CNN neural network. Based on this model, three transferlearning tasks to improve the classification performance with backgroundknowledge are tested. We compare 1. Supervised category transfer: social mediadata annotated with near-offensive language categories, 2. Weakly-supervisedcategory transfer: tweets annotated with emojis they contain, 3. Unsupervisedcategory transfer: tweets annotated with topic clusters obtained by LatentDirichlet Allocation (LDA). Further, we investigate the effect of threedifferent strategies to mitigate negative effects of 'catastrophic forgetting'during transfer learning. Our results indicate that transfer learning ingeneral improves offensive language detection. Best results are achieved frompre-training our model on the unsupervised topic clustering of tweets incombination with thematic user cluster information.


Introduction (beta)



Conclusion (beta)