Abstract
Classic multi-agent reinforcement learning (MARL) methods require that agentsenjoy global state observability, preventing development of decentralizedalgorithms and limiting scalability. Recent work has shown that, underassumptions on decaying inter-agent influence, global observability can bereplaced by local neighborhood observability at each agent, enablingdecentralization and scalability. Real-world applications enjoying such decayproperties remain underexplored, however, despite the fact that signal powerdecay, or signal attenuation, due to path loss is an intrinsic feature of manyproblems in wireless communications and radar networks. In this paper, we showthat signal attenuation enables decentralization in MARL by considering theillustrative special case of performing power allocation for target detectionin a radar network. To achieve this, we propose two new constrained multi-agentMarkov decision process formulations of this power allocation problem, derivelocal neighborhood approximations for global value function and gradientestimates and establish corresponding error bounds, and develop decentralizedsaddle point policy gradient algorithms for solving the proposed problems. Ourapproach, though oriented towards the specific radar network problem weconsider, provides a useful model for future extensions to additional problemsin wireless communications and radar networks.