Abstract
Federated learning (FL) has emerged as a powerful paradigm for collaborativemodel training across distributed clients while preserving data privacy.However, existing FL algorithms predominantly focus on unconstrainedoptimization problems with exact gradient information, limiting itsapplicability in scenarios where only noisy function evaluations are accessibleor where model parameters are constrained. To address these challenges, wepropose a novel zeroth-order projection-based algorithm on Riemannian manifoldsfor FL. By leveraging the projection operator, we introduce a computationallyefficient zeroth-order Riemannian gradient estimator. Unlike existingestimators, ours requires only a simple Euclidean random perturbation,eliminating the need to sample random vectors in the tangent space, thusreducing computational cost. Theoretically, we first prove the approximationproperties of the estimator and then establish the sublinear convergence of theproposed algorithm, matching the rate of its first-order counterpart.Numerically, we first assess the efficiency of our estimator using kernelprincipal component analysis. Furthermore, we apply the proposed algorithm totwo real-world scenarios: zeroth-order attacks on deep neural networks andlow-rank neural network training to validate the theoretical findings.