Text style transfer rephrases a text from a source style (e.g., informal) toa target style (e.g., formal) while keeping its original meaning. Despite thesuccess existing works have achieved using a parallel corpus for the twostyles, transferring text style has proven significantly more challenging whenthere is no parallel training corpus. In this paper, we address this challengeby using a reinforcement-learning-based generator-evaluator architecture. Ourgenerator employs an attention-based encoder-decoder to transfer a sentencefrom the source style to the target style. Our evaluator is an adversariallytrained style discriminator with semantic and syntactic constraints that scorethe generated sentence for style, meaning preservation, and fluency.Experimental results on two different style transfer tasks (sentiment transferand formality transfer) show that our model outperforms state-of-the-artapproaches. Furthermore, we perform a manual evaluation that demonstrates theeffectiveness of the proposed method using subjective metrics of generated textquality.