Abstract
In Federated Learning (FL), the limited accessibility of data from diverselocations and user types poses a significant challenge due to restricted userparticipation. Expanding client access and diversifying data enhance models byincorporating diverse perspectives, thereby enhancing adaptability. However,challenges arise in dynamic and mobile environments where certain devices maybecome inaccessible as FL clients, impacting data availability and clientselection methods. To address this, we propose an On-Demand solution, deployingnew clients using Docker Containers on-the-fly. Our On-Demand solution,employing Deep Reinforcement Learning (DRL), targets client availability andselection, while considering data shifts, and container deploymentcomplexities. It employs an autonomous end-to-end solution for handling modeldeployment and client selection. The DRL strategy uses a Markov DecisionProcess (MDP) framework, with a Master Learner and a Joiner Learner. Thedesigned cost functions represent the complexity of the dynamic clientdeployment and selection. Simulated tests show that our architecture can easilyadjust to changes in the environment and respond to On-Demand requests. Thisunderscores its ability to improve client availability, capability, accuracy,and learning efficiency, surpassing heuristic and tabular reinforcementlearning solutions.