Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning

  • 2025-01-28 14:08:57
  • Xi Chen, Qin Li, Haibin Cai, Ting Wang
  • 0

Abstract

Federated Learning (FL) empowers multiple clients to collaboratively trainmachine learning models without sharing local data, making it highly applicablein heterogeneous Internet of Things (IoT) environments. However, intrinsicheterogeneity in clients' model architectures and computing capabilities oftenresults in model accuracy loss and the intractable straggler problem, whichsignificantly impairs training effectiveness. To tackle these challenges, thispaper proposes a novel Heterogeneity-aware Personalized Federated Learningmethod, named HAPFL, via multi-level Reinforcement Learning (RL) mechanisms.HAPFL optimizes the training process by incorporating three strategiccomponents: 1) An RL-based heterogeneous model allocation mechanism. Theparameter server employs a Proximal Policy Optimization (PPO)-based RL agent toadaptively allocate appropriately sized, differentiated models to clients basedon their performance, effectively mitigating performance disparities. 2) AnRL-based training intensity adjustment scheme. The parameter server leveragesanother PPO-based RL agent to dynamically fine-tune the training intensity foreach client to further enhance training efficiency and reduce stragglinglatency. 3) A knowledge distillation-based mutual learning mechanism. Eachclient deploys both a heterogeneous local model and a homogeneous lightweightmodel named LiteModel, where these models undergo mutual learning throughknowledge distillation. This uniform LiteModel plays a pivotal role inaggregating and sharing global knowledge, significantly enhancing theeffectiveness of personalized local training. Experimental results acrossmultiple benchmark datasets demonstrate that HAPFL not only achieves highaccuracy but also substantially reduces the overall training time by20.9%-40.4% and decreases straggling latency by 19.0%-48.0% compared toexisting solutions.

 

Quick Read (beta)

loading the full paper ...