Abstract

Real-world congestion problems (e.g. traffic congestion) are typically verycomplex and large-scale. Multiagent reinforcement learning (MARL) is apromising candidate for dealing with this emerging complexity by providing anautonomous and distributed solution to these problems. However, there are threelimiting factors that affect the deployability of MARL approaches to congestionproblems. These are learning time, scalability and decentralised coordinationi.e. no communication between the learning agents. In this paper we introduceResource Abstraction, an approach that addresses these challenges by allocatingthe available resources into abstract groups. This abstraction creates newreward functions that provide a more informative signal to the learning agentsand aid the coordination amongst them. Experimental work is conducted on twobenchmark domains from the literature, an abstract congestion problem and arealistic traffic congestion problem. The current state-of-the-art for solvingmultiagent congestion problems is a form of reward shaping called differencerewards. We show that the system using Resource Abstraction significantlyimproves the learning speed and scalability, and achieves the highest possibleor near-highest joint performance/social welfare for both congestion problemsin large-scale scenarios involving up to 1000 reinforcement learning agents.

 

Introduction (beta)

None

 

Conclusion (beta)

None