Human-to-human conversation is not just talking and listening. It is anincremental process where participants continually establish a commonunderstanding to rule out misunderstandings. Current language understandingmethods for intelligent robots do not consider this. There exist numerousapproaches considering non-understandings, but they ignore the incrementalprocess of resolving misunderstandings. In this article, we present a firstformalization and experimental validation of incremental action-repair forrobotic instruction-following based on reinforcement learning. To evaluate ourapproach, we propose a collection of benchmark environments for actioncorrection in language-conditioned reinforcement learning, utilizing asynthetic instructor to generate language goals and their correspondingcorrections. We show that a reinforcement learning agent can successfully learnto understand incremental corrections of misunderstood instructions.