Abstract
Due to the curse of statistical heterogeneity across clients, adopting apersonalized federated learning method has become an essential choice for thesuccessful deployment of federated learning-based services. Among diversebranches of personalization techniques, a model mixture-based personalizationmethod is preferred as each client has their own personalized model as a resultof federated learning. It usually requires a local model and a federated model,but this approach is either limited to partial parameter exchange or requiresadditional local updates, each of which is helpless to novel clients andburdensome to the client's computational capacity. As the existence of aconnected subspace containing diverse low-loss solutions between two or moreindependent deep networks has been discovered, we combined this interestingproperty with the model mixture-based personalized federated learning methodfor improved performance of personalization. We proposed SuPerFed, apersonalized federated learning method that induces an explicit connectionbetween the optima of the local and the federated model in weight space forboosting each other. Through extensive experiments on several benchmarkdatasets, we demonstrated that our method achieves consistent gains in bothpersonalization performance and robustness to problematic scenarios possible inrealistic services.