Designing profitable and reliable trading strategies is challenging in thehighly volatile cryptocurrency market. Existing works applied deepreinforcement learning methods and optimistically reported increased profits inbacktesting, which may suffer from the false positive issue due to overfitting.In this paper, we propose a practical approach to address backtest overfittingfor cryptocurrency trading using deep reinforcement learning. First, weformulate the detection of backtest overfitting as a hypothesis test. Then, wetrain the DRL agents, estimate the probability of overfitting, and reject theoverfitted agents, increasing the chance of good trading performance. Finally,on 10 cryptocurrencies over a testing period from 05/01/2022 to 06/27/2022(during which the crypto market crashed two times), we show that the lessoverfitted deep reinforcement learning agents have a higher return than that ofmore overfitted agents, an equal weight strategy, and the S&P DBM Index (marketbenchmark), offering confidence in possible deployment to a real market.