Scaling adaptive traffic-signal control involves dealing with combinatorialstate and action spaces. Multi-agent reinforcement learning attempts to addressthis challenge by distributing control to specialized agents. However,specialization hinders generalization and transferability, and thecomputational graphs underlying neural-networks architectures---dominating inthe multi-agent setting---do not offer the flexibility to handle an arbitrarynumber of entities which changes both between road networks, and over time asvehicles traverse the network. We introduce Inductive Graph ReinforcementLearning (IG-RL) based on graph-convolutional networks which adapts to thestructure of any road network, to learn detailed representations oftraffic-controllers and their surroundings. Our decentralized approach enableslearning of a transferable-adaptive-traffic-signal-control policy. After beingtrained on an arbitrary set of road networks, our model can generalize to newroad networks, traffic distributions, and traffic regimes, with no additionaltraining and a constant number of parameters, enabling greater scalabilitycompared to prior methods. Furthermore, our approach can exploit thegranularity of available data by capturing the (dynamic) demand at both thelane and the vehicle levels. The proposed method is tested on both roadnetworks and traffic settings never experienced during training. We compareIG-RL to multi-agent reinforcement learning and domain-specific baselines. Inboth synthetic road networks and in a larger experiment involving the controlof the 3,971 traffic signals of Manhattan, we show that differentinstantiations of IG-RL outperform baselines.