Abstract

Natural language understanding for robotics can require substantial domain-and platform-specific engineering. For example, for mobile robots topick-and-place objects in an environment to satisfy human commands, we canspecify the language humans use to issue such commands, and connect conceptwords like red can to physical object properties. One way to alleviate thisengineering for a new domain is to enable robots in human environments to adaptdynamically---continually learning new language constructions and perceptualconcepts. In this work, we present an end-to-end pipeline for translatingnatural language commands to discrete robot actions, and use clarificationdialogs to jointly improve language parsing and concept grounding. We train andevaluate this agent in a virtual setting on Amazon Mechanical Turk, and wetransfer the learned agent to a physical robot platform to demonstrate it inthe real world.

 

Introduction (beta)

None

 

Conclusion (beta)

None