Abstract
The present article studies the minimization of convex, L-smooth functionsdefined on a separable real Hilbert space. We analyze regularized stochasticgradient descent (reg-SGD), a variant of stochastic gradient descent that usesa Tikhonov regularization with time-dependent, vanishing regularizationparameter. We prove strong convergence of reg-SGD to the minimum-norm solutionof the original problem without additional boundedness assumptions. Moreover,we quantify the rate of convergence and optimize the interplay betweenstep-sizes and regularization decay. Our analysis reveals how vanishingTikhonov regularization controls the flow of SGD and yields stable learningdynamics, offering new insights into the design of iterative algorithms forconvex problems, including those that arise in ill-posed inverse problems. Wevalidate our theoretical findings through numerical experiments on imagereconstruction and ODE-based inverse problems.