Abstract
Rearranging objects on a tabletop surface by means of nonprehensilemanipulation is a task which requires skillful interaction with the physicalworld. Usually, this is achieved by precisely modeling physical properties ofthe objects, robot, and the environment for explicit planning. In contrast, asexplicitly modeling the physical environment is not always feasible andinvolves various uncertainties, we learn a nonprehensile rearrangement strategywith deep reinforcement learning based on only visual feedback. For this, wemodel the task with rewards and train a deep Q-network. Our potentialfield-based heuristic exploration strategy reduces the amount of collisionswhich lead to suboptimal outcomes and we actively balance the training set toavoid bias towards poor examples. Our training process leads to quickerlearning and better performance on the task as compared to uniform explorationand standard experience replay. We demonstrate empirical evidence fromsimulation that our method leads to a success rate of 85%, show that our systemcan cope with sudden changes of the environment, and compare our performancewith human level performance.