Abstract
The Space-Air-Ground Integrated Network (SAGIN) plays a pivotal role as acomprehensive foundational network communication infrastructure, presentingopportunities for highly efficient global data transmission. Nonetheless, givenSAGIN's unique characteristics as a dynamically heterogeneous network,conventional network optimization methodologies encounter challenges insatisfying the stringent requirements for network latency and stabilityinherent to data transmission within this network environment. Therefore, thispaper proposes the use of differentiated federated reinforcement learning(DFRL) to solve the traffic offloading problem in SAGIN, i.e., using multipleagents to generate differentiated traffic offloading policies. Considering thedifferentiated characteristics of each region of SAGIN, DFRL models the trafficoffloading policy optimization process as the process of solving theDecentralized Partially Observable Markov Decision Process (DEC-POMDP) problem.The paper proposes a novel Differentiated Federated Soft Actor-Critic (DFSAC)algorithm to solve the problem. The DFSAC algorithm takes the network packetdelay as the joint reward value and introduces the global trend model as thejoint target action-value function of each agent to guide the update of eachagent's policy. The simulation results demonstrate that the traffic offloadingpolicy based on the DFSAC algorithm achieves better performance in terms ofnetwork throughput, packet loss rate, and packet delay compared to thetraditional federated reinforcement learning approach and other baselineapproaches.