Abstract
The growing computational demands of deep reinforcement learning (DRL) haveraised concerns about the environmental and economic costs of traininglarge-scale models. While algorithmic efficiency in terms of learningperformance has been extensively studied, the energy requirements, greenhousegas emissions, and monetary costs of DRL algorithms remain largely unexplored.In this work, we present a systematic benchmarking study of the energyconsumption of seven state-of-the-art DRL algorithms, namely DQN, TRPO, A2C,ARS, PPO, RecurrentPPO, and QR-DQN, implemented using Stable Baselines. Eachalgorithm was trained for one million steps each on ten Atari 2600 games, andpower consumption was measured in real-time to estimate total energy usage,CO2-Equivalent emissions, and electricity cost based on the U.S. nationalaverage electricity price. Our results reveal substantial variation in energyefficiency and training cost across algorithms, with some achieving comparableperformance while consuming up to 24% less energy (ARS vs. DQN), emittingnearly 68% less CO2, and incurring almost 68% lower monetary cost (QR-DQN vs.RecurrentPPO) than less efficient counterparts. We further analyze thetrade-offs between learning performance, training time, energy use, andfinancial cost, highlighting cases where algorithmic choices can mitigateenvironmental and economic impact without sacrificing learning performance.This study provides actionable insights for developing energy-aware andcost-efficient DRL practices and establishes a foundation for incorporatingsustainability considerations into future algorithmic design and evaluation.