Poetry Generation involves teaching systems to automatically generate textthat resembles poetic work. A deep learning system can learn to generate poetryon its own by training on a corpus of poems and modeling the particular styleof language. In this paper, we propose taking an approach that fine-tunesGPT-2, a pre-trained language model, to our downstream task of poetrygeneration. We extend prior work on poetry generation by introducing creativeelements. Specifically, we generate poems that express emotion and elicit thesame in readers, and poems that use the language of dreams---called dreampoetry. We are able to produce poems that correctly elicit the emotions ofsadness and joy 87.5 and 85 percent, respectively, of the time. We producedreamlike poetry by training on a corpus of texts that describe dreams. Poemsfrom this model are shown to capture elements of dream poetry with scores of noless than 3.2 on the Likert scale. We perform crowdsourced human-evaluation forall our poems. We also make use of the Coh-Metrix tool, outlining metrics weuse to gauge the quality of text generated.