Abstract
Unmanned Surface Vehicles technology (USVs) is an exciting topic thatessentially deploys an algorithm to safely and efficiently performs a mission.Although reinforcement learning is a well-known approach to modeling such atask, instability and divergence may occur when combining off-policy andfunction approximation. In this work, we used deep reinforcement learningcombining Q-learning with a neural representation to avoid instability. Ourmethodology uses deep q-learning and combines it with a rolling wave planningapproach on agile methodology. Our method contains two critical parts in orderto perform missions in an unknown environment. The first is a path planner thatis responsible for generating a potential effective path to a destinationwithout considering the details of the root. The latter is a decision-makingmodule that is responsible for short-term decisions on avoiding obstaclesduring the near future steps of USV exploitation within the context of thevalue function. Simulations were performed using two algorithms: a basicvanilla vessel navigator (VVN) as a baseline and an improved one for the vesselnavigator with a planner and local view (VNPLV). Experimental results show thatthe proposed method enhanced the performance of VVN by 55.31 on average forlong-distance missions. Our model successfully demonstrated obstacle avoidanceby means of deep reinforcement learning using planning adaptive paths inunknown environments.