Conditional story generation and contextual text continuation have becomeincreasingly popular topics in NLP community. Existing models are often proneto output paragraphs of texts that gradually diverge from the given prompt.Although the generated text may have a reasonable perplexity and diversity, itcould easily be identified by human as gibberish. The goal of our project is toimprove the coherence and consistency across sentences in a language-generationmodel. We aim to solve this issue by first training a sentence pair coherenceclassifier with GPT-2 pretrained model, and then co-train the GPT-2 languagemodel with this new coherence objective using a method analogous to theREINFORCE algorithm. This fine-tuned language model is able to generate lengthyparagraph conditioned on a given topic without diverging too much. Thesimplicity of this model allows it to be applicable to a variety of underlyinglanguage model architecture since it only modifies the final layer of thepre-trained model.