Abstract
Recent advances in reinforcement learning have shown its potential to tacklecomplex real-life tasks. However, as the dimensionality of the task increases,reinforcement learning methods tend to struggle. To overcome this, we exploremethods for representing the semantic information embedded in the state. Whileprevious methods focused on information in its raw form (e.g., raw visualinput), we propose to represent the state using natural language. Language canrepresent complex scenarios and concepts, making it a favorable candidate forrepresentation. Empirical evidence, within the domain of ViZDoom, suggests thatnatural language based agents are more robust, converge faster and performbetter than vision based agents, showing the benefit of using natural languagerepresentations for reinforcement learning.