State-of-the-art image denoisers exploit various types of deep neuralnetworks via deterministic training. Alternatively, very recent works utilizedeep reinforcement learning for restoring images with diverse or unknowncorruptions. Though deep reinforcement learning can generate effective policynetworks for operator selection or architecture search in image restoration,how it is connected to the classic deterministic training in solving inverseproblems remains unclear. In this work, we propose a novel image denoisingscheme via Residual Recovery using Reinforcement Learning, dubbed R3L. We showthat R3L is equivalent to a deep recurrent neural network that is trained usinga stochastic reward, in contrast to many popular denoisers using supervisedlearning with deterministic losses. To benchmark the effectiveness ofreinforcement learning in R3L, we train a recurrent neural network with thesame architecture for residual recovery using the deterministic loss, thus toanalyze how the two different training strategies affect the denoisingperformance. With such a unified benchmarking system, we demonstrate that theproposed R3L has better generalizability and robustness in image denoising whenthe estimated noise level varies, comparing to its counterparts usingdeterministic training, as well as various state-of-the-art image denoisingalgorithms.