Abstract
There is a growing cross-disciplinary effort in the broad domain ofoptimization and learning with streams of data, applied to settings wheretraditional batch optimization techniques cannot produce solutions at timescales that match the inter-arrival times of the data points due tocomputational and/or communication bottlenecks. Special types of onlinealgorithms can handle this situation, and this article focuses on suchtime-varying optimization algorithms, with emphasis on Machine Leaning andSignal Processing, as well as data-driven Control. Approaches for the design oftime-varying or online first-order optimization methods are discussed, withemphasis on algorithms that can handle errors in the gradient, as may arisewhen the gradient is estimated. Insights on performance metrics andaccompanying claims are provided, along with evidence of cases where algorithmsthat are provably convergent in batch optimization may perform poorly in anonline regime. The role of distributed computation is discussed. Illustrativenumerical examples for a number of applications of broad interest are providedto convey key ideas.