Abstract
Aspect-Target Sentiment Classification (ATSC) is a subtask of Aspect-BasedSentiment Analysis (ABSA), which has many applications e.g. in e-commerce,where data and insights from reviews can be leveraged to create value forbusinesses and customers. Recently, deep transfer-learning methods have beenapplied successfully to a myriad of Natural Language Processing (NLP) tasks,including ATSC. Building on top of the prominent BERT language model, weapproach ATSC using a two-step procedure: self-supervised domain-specific BERTlanguage model finetuning, followed by supervised task-specific finetuning. Ourfindings on how to best exploit domain-specific language model finetuningenable us to produce new state-of-the-art performance on the SemEval 2014 Task4 restaurants dataset. In addition, to explore the real-world robustness of ourmodels, we perform cross-domain evaluation. We show that a cross-domain adaptedBERT language model performs significantly better than strong baseline modelslike vanilla BERT-base and XLNet-base. Finally, we conduct a case study tointerpret model prediction errors.