Abstract
Manipulation tasks such as preparing a meal or assembling furniture remainhighly challenging for robotics and vision. The supervised approach ofimitation learning can handle short tasks but suffers from compounding errorsand the need of many demonstrations for longer and more complex tasks.Reinforcement learning (RL) can find solutions beyond demonstrations butrequires tedious and task-specific reward engineering for multi-step problems.In this work we address the difficulties of both methods and explore theircombination. To this end, we propose a RL policies operating on pre-trainedskills, that can learn composite manipulations using no intermediate rewardsand no demonstrations of full tasks. We also propose an efficient training ofbasic skills from few synthetic demonstrated trajectories by exploring recentCNN architectures and data augmentation. We show successful learning ofpolicies for composite manipulation tasks such as making a simple breakfast.Notably, our method achieves high success rates on a real robot, while usingsynthetic training data only.