Abstract
This paper reviews exploration techniques in deep reinforcement learning.Exploration techniques are of primary importance when solving sparse rewardproblems. In sparse reward problems, the reward is rare, which means that theagent will not find the reward often by acting randomly. In such a scenario, itis challenging for reinforcement learning to learn rewards and actionsassociation. Thus more sophisticated exploration methods need to be devised.This review provides a comprehensive overview of existing explorationapproaches, which are categorized based on the key contributions as followsreward novel states, reward diverse behaviours, goal-based methods,probabilistic methods, imitation-based methods, safe exploration andrandom-based methods. Then, the unsolved challenges are discussed to providevaluable future research directions. Finally, the approaches of differentcategories are compared in terms of complexity, computational effort andoverall performance.