Abstract
Federated learning is a distributed machine learning paradigm designed toprotect user data privacy, which has been successfully implemented acrossvarious scenarios. In traditional federated learning, the entire parameter setof local models is updated and averaged in each training round. Although thisfull network update method maximizes knowledge acquisition and sharing for eachmodel layer, it prevents the layers of the global model from cooperatingeffectively to complete the tasks of each client, a challenge we refer to aslayer mismatch. This mismatch problem recurs after every parameter averaging,consequently slowing down model convergence and degrading overall performance.To address the layer mismatch issue, we introduce the FedPart method, whichrestricts model updates to either a single layer or a few layers during eachcommunication round. Furthermore, to maintain the efficiency of knowledgeacquisition and sharing, we develop several strategies to select trainablelayers in each round, including sequential updating and multi-round cycletraining. Through both theoretical analysis and experiments, our findingsdemonstrate that the FedPart method significantly surpasses conventional fullnetwork update strategies in terms of convergence speed and accuracy, whilealso reducing communication and computational overheads.