Offensive language detection is an important and challenging task in naturallanguage processing. We present our submissions to the OffensEval 2020 sharedtask, which includes three English sub-tasks: identifying the presence ofoffensive language (Sub-task A), identifying the presence of target inoffensive language (Sub-task B), and identifying the categories of the target(Sub-task C). Our experiments explore using a domain-tuned contextualizedlanguage model (namely, BERT) for this task. We also experiment with differentcomponents and configurations (e.g., a multi-view SVM) stacked upon BERT modelsfor specific sub-tasks. Our submissions achieve F1 scores of 91.7% in Sub-taskA, 66.5% in Sub-task B, and 63.2% in Sub-task C. We perform an ablation studywhich reveals that domain tuning considerably improves the classificationperformance. Furthermore, error analysis shows common misclassification errorsmade by our model and outlines research directions for future.