Abstract
Multi-agent reinforcement learning has recently shown great promise as anapproach to networked system control. Arguably, one of the most difficult andimportant tasks for which large scale networked system control is applicable iscommon-pool resource management. Crucial common-pool resources include arableland, fresh water, wetlands, wildlife, fish stock, forests and the atmosphere,of which proper management is related to some of society's greatest challengessuch as food security, inequality and climate change. Here we take inspirationfrom a recent research program investigating the game-theoretic incentives ofhumans in social dilemma situations such as the well-known tragedy of thecommons. However, instead of focusing on biologically evolved human-likeagents, our concern is rather to better understand the learning and operatingbehaviour of engineered networked systems comprising general-purposereinforcement learning agents, subject only to nonbiological constraints suchas memory, computation and communication bandwidth. Harnessing tools fromempirical game-theoretic analysis, we analyse the differences in resultingsolution concepts that stem from employing different information structures inthe design of networked multi-agent systems. These information structurespertain to the type of information shared between agents as well as theemployed communication protocol and network topology. Our analysis contributesnew insights into the consequences associated with certain design choices andprovides an additional dimension of comparison between systems beyondefficiency, robustness, scalability and mean control performance.