Nowadays, offensive content in social media has become a serious problem, andautomatically detecting offensive language is an essential task. In this paper,we build an offensive language detection system, which combines multi-tasklearning with BERT-based models. Using a pre-trained language model such asBERT, we can effectively learn the representations for noisy text in socialmedia. Besides, to boost the performance of offensive language detection, weleverage the supervision signals from other related tasks. In theOffensEval-2020 competition, our model achieves 91.51% F1 score in EnglishSub-task A, which is comparable to the first place (92.23%F1). An empiricalanalysis is provided to explain the effectiveness of our approaches.