Abstract
In this paper we study consensus-based optimization (CBO), a versatile,flexible and customizable optimization method suitable for performing nonconvexand nonsmooth global optimizations in high dimensions. CBO is a multi-particlemetaheuristic, which is effective in various applications and at the same timeamenable to theoretical analysis thanks to its minimalistic design. Theunderlying dynamics, however, is flexible enough to incorporate differentmechanisms widely used in evolutionary computation and machine learning, as weshow by analyzing a variant of CBO which makes use of memory effects andgradient information. We rigorously prove that this dynamics converges to aglobal minimizer of the objective function in mean-field law for a vast classof functions under minimal assumptions on the initialization of the method. Theproof in particular reveals how to leverage further, in some applicationsadvantageous, forces in the dynamics without loosing provable globalconvergence. To demonstrate the benefit of the herein investigated memoryeffects and gradient information in certain applications, we present numericalevidence for the superiority of this CBO variant in applications such asmachine learning and compressed sensing, which en passant widen the scope ofapplications of CBO.