Abstract
Reinforcement learning algorithms have shown great success in solvingdifferent problems ranging from playing video games to robotics. However, theystruggle to solve delicate robotic problems, especially those involving contactinteractions. Though in principle a policy directly outputting joint torquesshould be able to learn to perform these tasks, in practice we see that it hasdifficulty to robustly solve the problem without any given structure in theaction space. In this paper, we investigate how the choice of action space cangive robust performance in presence of contact uncertainties. We proposelearning a policy giving as output impedance and desired position in jointspace and compare the performance of that approach to torque and positioncontrol under different contact uncertainties. Furthermore, we propose anadditional reward term designed to regularize these variable impedance controlpolicies, giving them interpretability and facilitating their transfer to realsystems. We present extensive experiments in simulation of both floating andfixed-base systems in tasks involving contact uncertainties, as well as resultsfor running the learned policies on a real system.