Abstract

Inductive transfer learning has greatly impacted computer vision, butexisting approaches in NLP still require task-specific modifications andtraining from scratch. We propose Universal Language Model Fine-tuning(ULMFiT), an effective transfer learning method that can be applied to any taskin NLP, and introduce techniques that are key for fine-tuning a language model.Our method significantly outperforms the state-of-the-art on six textclassification tasks, reducing the error by 18-24% on the majority of datasets.Furthermore, with only 100 labeled examples, it matches the performance oftraining from scratch on 100x more data. We open-source our pretrained modelsand code.

 

Quick Read (beta)