Transfer Learning in Multi-Agent Reinforcement Learning with Double Q-Networks for Distributed Resource Sharing in V2X Communication

  • 2021-07-13 15:50:10
  • Hammad Zafar, Zoran Utkovski, Martin Kasparick, Slawomir Stanczak
  • 1


This paper addresses the problem of decentralized spectrum sharing invehicle-to-everything (V2X) communication networks. The aim is to provideresource-efficient coexistence of vehicle-to-infrastructure(V2I) andvehicle-to-vehicle(V2V) links. A recent work on the topic proposes amulti-agent reinforcement learning (MARL) approach based on deep Q-learning,which leverages a fingerprint-based deep Q-network (DQN) architecture. Thiswork considers an extension of this framework by combining Double Q-learning(via Double DQN) and transfer learning. The motivation behind is that DoubleQ-learning can alleviate the problem of overestimation of the action valuespresent in conventional Q-learning, while transfer learning can leverageknowledge acquired by an expert model to accelerate learning in the MARLsetting. The proposed algorithm is evaluated in a realistic V2X setting, withsynthetic data generated based on a geometry-based propagation model thatincorporates location-specific geographical descriptors of the simulatedenvironment(outlines of buildings, foliage, and vehicles). The advantages ofthe proposed approach are demonstrated via numerical simulations.


Quick Read (beta)

loading the full paper ...