Abstract
Supervised training of abstractive language generation models results inlearning conditional probabilities over language sequences based on thesupervised training signal. When the training signal contains a variety ofwriting styles, such models may end up learning an 'average' style that isdirectly influenced by the training data make-up and cannot be controlled bythe needs of an application. We describe a family of model architecturescapable of capturing both generic language characteristics via shared modelparameters, as well as particular style characteristics via private modelparameters. Such models are able to generate language according to a specificlearned style, while still taking advantage of their power to model genericlanguage phenomena. Furthermore, we describe an extension that uses a mixtureof output distributions from all learned styles to perform on-the fly styleadaptation based on the textual input alone. Experimentally, we find that theproposed models consistently outperform models that encapsulate single-style oraverage-style language generation capabilities.