Abstract
Deep reinforcement learning (deep RL) has achieved tremendous success onvarious domains through a combination of algorithmic design and carefulselection of hyper-parameters. Algorithmic improvements are often the result ofiterative enhancements built upon prior approaches, while hyper-parameterchoices are typically inherited from previous methods or fine-tunedspecifically for the proposed technique. Despite their crucial impact onperformance, hyper-parameter choices are frequently overshadowed by algorithmicadvancements. This paper conducts an extensive empirical study focusing on thereliability of hyper-parameter selection for value-based deep reinforcementlearning agents, including the introduction of a new score to quantify theconsistency and reliability of various hyper-parameters. Our findings not onlyhelp establish which hyper-parameters are most critical to tune, but also helpclarify which tunings remain consistent across different training regimes.