Abstract
The dominant approaches for controlling language models achieve prominence incontrolling high-level attributes (e.g. topic and sentiment). However, thesemethods often require condition-specific data or are computationally expensive.We propose a new simple guided decoding method, Gamma Sampling, which does notrequire any training data to achieve fine-grained controllable text generationwhile maintaining a fast generation speed. Gamma Sampling introducesattribute-related information (provided by humans or language modelsthemselves) into the sampling process to guide language models to generatetexts with desired attributes. Since no training is involved, Gamma Samplingcan be easily applied to any language model for controllable text generation.Through experiments, we show that Gamma Sampling-steered GPT2-small (117M)outperforms baselines such as PPLM (345M) and CTRL (1.6B) in diversity,attribute relevance, and overall quality of generated samples.