Abstract
Autonomous underwater vehicle (AUV) plays an increasingly important role inocean exploration. Existing AUVs are usually not fully autonomous and generallylimited to pre-planning or pre-programming tasks. Reinforcement learning (RL)and deep reinforcement learning have been introduced into the AUV design andresearch to improve its autonomy. However, these methods are still difficult toapply directly to the actual AUV system because of the sparse rewards and lowlearning efficiency. In this paper, we proposed a deep interactivereinforcement learning method for path following of AUV by combining theadvantages of deep reinforcement learning and interactive RL. In addition,since the human trainer cannot provide human rewards for AUV when it is runningin the ocean and AUV needs to adapt to a changing environment, we furtherpropose a deep reinforcement learning method that learns from both humanrewards and environmental rewards at the same time. We test our methods in twopath following tasks---straight line and sinusoids curve following of AUV bysimulating in the Gazebo platform. Our experimental results show that with ourproposed deep interactive RL method, AUV can converge faster than a DQN learnerfrom only environmental reward. Moreover, AUV learning with our deep RL fromboth human and environmental rewards can also achieve a similar or even betterperformance than that with the deep interactive RL method and can adapt to theactual environment by further learning from environmental rewards.