Abstract
To solve difficult tasks, humans ask questions to acquire knowledge fromexternal sources. In contrast, classical reinforcement learning agents lacksuch an ability and often resort to exploratory behavior. This is exacerbatedas few present-day environments support querying for knowledge. In order tostudy how agents can be taught to query external knowledge via language, wefirst introduce two new environments: the grid-world-based Q-BabyAI and thetext-based Q-TextWorld. In addition to physical interactions, an agent canquery an external knowledge source specialized for these environments to gatherinformation. Second, we propose the "Asking for Knowledge" (AFK) agent, whichlearns to generate language commands to query for meaningful knowledge thathelps solve the tasks. AFK leverages a non-parametric memory, a pointermechanism and an episodic exploration bonus to tackle (1) a large querylanguage space, (2) irrelevant information, (3) delayed reward for makingmeaningful queries. Extensive experiments demonstrate that the AFK agentoutperforms recent baselines on the challenging Q-BabyAI and Q-TextWorldenvironments.