Abstract
Sentence completion (SC) questions present a sentence with one or more blanksthat need to be filled in, three to five possible words or phrases as options.SC questions are widely used for students learning English as a Second Language(ESL) and building computational approaches to automatically solve suchquestions is beneficial to language learners. In this work, we propose a neuralframework to solve SC questions in English examinations by utilizingpre-trained language models. We conduct extensive experiments on a real-worldK-12 ESL SC question dataset and the results demonstrate the superiority of ourmodel in terms of prediction accuracy. Furthermore, we run precision-recalltrade-off analysis to discuss the practical issues when deploying it inreal-life scenarios. To encourage reproducible results, we make our codepublicly available at \url{https://github.com/AIED2021/ESL-SentenceCompletion}.