Abstract
This paper investigates the use of multi-agent reinforcement learning (MARL)to address distributed channel access in wireless local area networks. Inparticular, we consider the challenging yet more practical case where theagents heterogeneously adopt value-based or policy-based reinforcement learningalgorithms to train the model. We propose a heterogeneous MARL trainingframework, named QPMIX, which adopts a centralized training with distributedexecution paradigm to enable heterogeneous agents to collaborate. Moreover, wetheoretically prove the convergence of the proposed heterogeneous MARL methodwhen using the linear value function approximation. Our method maximizes thenetwork throughput and ensures fairness among stations, therefore, enhancingthe overall network performance. Simulation results demonstrate that theproposed QPMIX algorithm improves throughput, mean delay, delay jitter, andcollision rates compared with conventional carrier-sense multiple access withcollision avoidance (CSMA/CA) mechanism in the saturated traffic scenario.Furthermore, the QPMIX algorithm is robust in unsaturated and delay-sensitivetraffic scenarios. It coexists well with the conventional CSMA/CA mechanism andpromotes cooperation among heterogeneous agents.