In this paper, we propose a new control framework called the moving endpointcontrol to restore images corrupted by different degradation levels in onemodel. The proposed control problem contains a restoration dynamics which ismodeled by an RNN. The moving endpoint, which is essentially the terminal timeof the associated dynamics, is determined by a policy network. We call theproposed model the dynamically unfolding recurrent restorer (DURR). Numericalexperiments show that DURR is able to achieve state-of-the-art performances onblind image denoising and JPEG image deblocking. Furthermore, DURR can wellgeneralize to images with higher degradation levels that are not included inthe training stage.