Abstract
This paper considers optimal traffic signal control in smart cities, whichhas been taken as a complex networked system control problem. Given theinteracting dynamics among traffic lights and road networks, attainingcontroller adaptivity and scalability stands out as a primary challenge.Capturing the spatial-temporal correlation among traffic lights under theframework of Multi-Agent Reinforcement Learning (MARL) is a promising solution.Nevertheless, existing MARL algorithms ignore effective information aggregationwhich is fundamental for improving the learning capacity of decentralizedagents. In this paper, we design a new decentralized control architecture withimproved environmental observability to capture the spatial-temporalcorrelation. Specifically, we first develop a topology-aware informationaggregation strategy to extract correlation-related information fromunstructured data gathered in the road network. Particularly, we transfer theroad network topology into a graph shift operator by forming a diffusionprocess on the topology, which subsequently facilitates the construction ofgraph signals. A diffusion convolution module is developed, forming a new MARLalgorithm, which endows agents with the capabilities of graph learning.Extensive experiments based on both synthetic and real-world datasets verifythat our proposal outperforms existing decentralized algorithms.