Abstract
Reinforcement learning (RL) enables robots to learn skills from interactionswith the real world. In practice, the unstructured step-based exploration usedin Deep RL -- often very successful in simulation -- leads to jerky motionpatterns on real robots. Consequences of the resulting shaky behavior are poorexploration, or even damage to the robot. We address these issues by adaptingstate-dependent exploration (SDE) to current Deep RL algorithms. To enable thisadaptation, we propose three extensions to the original SDE, which leads to anew exploration method generalized state-dependent exploration (gSDE). Weevaluate gSDE both in simulation, on PyBullet continuous control tasks, anddirectly on a tendon-driven elastic robot. gSDE yields competitive results insimulation but outperforms the unstructured exploration on the real robot. Thecode is available at https://github.com/DLR-RM/stable-baselines3/tree/sde.