Abstract
Multi-agent Deep Reinforcement Learning (MADRL) based traffic signal controlbecomes a popular research topic in recent years. To alleviate the scalabilityissue of completely centralized RL techniques and the non-stationarity issue ofcompletely decentralized RL techniques on large-scale traffic networks, someliterature utilizes a regional control approach where the whole network isfirstly partitioned into multiple disjoint regions, followed by applying thecentralized RL approach to each region. However, the existing partitioningrules either have no constraints on the topology of regions or require the sametopology for all regions. Meanwhile, no existing regional control approachexplores the performance of optimal joint action in an exponentially growingregional action space when intersections are controlled by 4-phase trafficsignals (EW, EWL, NS, NSL). In this paper, we propose a novel RL trainingframework named RegionLight to tackle the above limitations. Specifically, thetopology of regions is firstly constrained to a star network which comprisesone center and an arbitrary number of leaves. Next, the network partitioningproblem is modeled as an optimization problem to minimize the number ofregions. Then, an Adaptive Branching Dueling Q-Network (ABDQ) model is proposedto decompose the regional control task into several joint signal controlsub-tasks corresponding to particular intersections. Subsequently, thesesub-tasks maximize the regional benefits cooperatively. Finally, the globalcontrol strategy for the whole network is obtained by concatenating the optimaljoint actions of all regions. Experimental results demonstrate the superiorityof our proposed framework over all baselines under both real and syntheticdatasets in all evaluation metrics.