Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds

  • 2024-06-04 14:17:24
  • Daniel Dodd, Louis Sharrock, Christopher Nemeth
  • 0


In recent years, interest in gradient-based optimization over Riemannianmanifolds has surged. However, a significant challenge lies in the reliance onhyperparameters, especially the learning rate, which requires meticulous tuningby practitioners to ensure convergence at a suitable rate. In this work, weintroduce innovative learning-rate-free algorithms for stochastic optimizationover Riemannian manifolds, eliminating the need for hand-tuning and providing amore robust and user-friendly approach. We establish high probabilityconvergence guarantees that are optimal, up to logarithmic factors, compared tothe best-known optimally tuned rate in the deterministic setting. Our approachis validated through numerical experiments, demonstrating competitiveperformance against learning-rate-dependent algorithms.


Quick Read (beta)

loading the full paper ...