Abstract
Cooling system plays a critical role in a modern data center (DC). Developingan optimal control policy for DC cooling system is a challenging task. Theprevailing approaches often rely on approximating system models that are builtupon the knowledge of mechanical cooling, electrical and thermal management,which is difficult to design and may lead to sub-optimal or unstableperformances. In this paper, we propose utilizing the large amount ofmonitoring data in DC to optimize the control policy. To do so, we cast thecooling control policy design into an energy cost minimization problem withtemperature constraints, and tap it into the emerging deep reinforcementlearning (DRL) framework. Specifically, we propose an end-to-end coolingcontrol algorithm (CCA) that is based on the actor-critic framework and anoff-policy off-line version of the deep deterministic policy gradient (DDPG)algorithm. In the proposed CCA, an evaluation network is trained to predict anenergy cost counter penalized by the cooling status of the DC room, and apolicy network is trained to predict optimized control settings when gave thecurrent load and weather information. The proposed algorithm is evaluated onthe EnergyPlus simulation platform and on a real data trace collected from theNational Super Computing Centre (NSCC) of Singapore. Our results show that theproposed CCA can achieve about 11% cooling cost saving on the simulationplatform compared with a manually configured baseline control algorithm. In thetrace-based study, we propose a de-underestimation validation mechanism as wecannot directly test the algorithm on a real DC. Even though with DUE theresults are conservative, we can still achieve about 15% cooling energy savingon the NSCC data trace if we set the inlet temperature threshold at 26.6 degreeCelsius.