Abstract
Stochastic optimisation algorithms are the de facto standard for machinelearning with large amounts of data. Handling only a subset of available datain each optimisation step dramatically reduces the per-iteration computationalcosts, while still ensuring significant progress towards the solution. Drivenby the need to solve large-scale optimisation problems as efficiently aspossible, the last decade has witnessed an explosion of research in this area.Leveraging the parallels between machine learning and inverse problems hasallowed harnessing the power of this research wave for solving inverseproblems. In this survey, we provide a comprehensive account of thestate-of-the-art in stochastic optimisation from the viewpoint of inverseproblems. We present algorithms with diverse modalities of problemrandomisation and discuss the roles of variance reduction, acceleration,higher-order methods, and other algorithmic modifications, and comparetheoretical results with practical behaviour. We focus on the potential and thechallenges for stochastic optimisation that are unique to inverse imagingproblems and are not commonly encountered in machine learning. We conclude thesurvey with illustrative examples from imaging problems to examine theadvantages and disadvantages that this new generation of algorithms bring tothe field of inverse problems.