Abstract
Robot navigation using deep reinforcement learning (DRL) has shown greatpotential in improving the performance of mobile robots. Nevertheless, mostexisting DRL-based navigation methods primarily focus on training a policy thatdirectly commands the robot with low-level controls, like linear and angularvelocities, which leads to unstable speeds and unsmooth trajectories of therobot during the long-term execution. An alternative method is to train a DRLpolicy that outputs the navigation path directly. However, two roadblocks arisefor training a DRL policy that outputs paths: (1) The action space forpotential paths often involves higher dimensions comparing to low-levelcommands, which increases the difficulties of training; (2) It takes multipletime steps to track a path instead of a single time step, which requires thepath to predicate the interactions of the robot w.r.t. the dynamic environmentin multiple time steps. This, in turn, amplifies the challenges associated withtraining. In response to these challenges, we propose PathRL, a novel DRLmethod that trains the policy to generate the navigation path for the robot.Specifically, we employ specific action space discretization techniques andtailored state space representation methods to address the associatedchallenges. In our experiments, PathRL achieves better success rates andreduces angular rotation variability compared to other DRL navigation methods,facilitating stable and smooth robot movement. We demonstrate the competitiveedge of PathRL in both real-world scenarios and multiple challenging simulationenvironments.