While reinforcement learning (RL) has been successful in natural languageprocessing (NLP) domains such as dialogue generation and text-based games, ittypically faces the problem of sparse rewards that leads to slow or noconvergence. Traditional methods that use text descriptions to extract only astate representation ignore the feedback inherently present in them. Intext-based games, for example, descriptions like "Good Job! You ate the food}"indicate progress, and descriptions like "You entered a new room" indicateexploration. Positive and negative cues like these can be converted to rewardsthrough sentiment analysis. This technique converts the sparse reward probleminto a dense one, which is easier to solve. Furthermore, this can enablereinforcement learning without rewards, in which the agent learns entirely fromthese intrinsic sentiment rewards. This framework is similar to intrinsicmotivation, where the environment does not necessarily provide the rewards, butthe agent analyzes and realizes them by itself. We find that providing denserewards in text-based games using sentiment analysis improves performance undersome conditions.