Abstract
Deep Reinforcement Learning (DRL) has achieved remarkable success in solvingcomplex decision-making problems by combining the representation capabilitiesof deep learning with the decision-making power of reinforcement learning.However, learning in sparse reward environments remains challenging due toinsufficient feedback to guide the optimization of agents, especially inreal-life environments with high-dimensional states. To tackle this issue,experience replay is commonly introduced to enhance learning efficiency throughpast experiences. Nonetheless, current methods of experience replay, whetherbased on uniform or prioritized sampling, frequently struggle with suboptimallearning efficiency and insufficient utilization of samples. This paperproposes a novel approach, diversity-based experience replay (DBER), whichleverages the deterministic point process to prioritize diverse samples instate realizations. We conducted extensive experiments on Robotic Manipulationtasks in MuJoCo, Atari games, and realistic in-door environments in Habitat.The results show that our method not only significantly improves learningefficiency but also demonstrates superior performance in sparse rewardenvironments with high-dimensional states, providing a simple yet effectivesolution for this field.