Abstract
We introduce GasRL, a simulator that couples a calibrated representation ofthe natural gas market with a model of storage-operator policies trained withdeep reinforcement learning (RL). We use it to analyse how optimal stockpilemanagement affects equilibrium prices and the dynamics of demand and supply. Wetest various RL algorithms and find that Soft Actor Critic (SAC) exhibitssuperior performance in the GasRL environment: multiple objectives of storageoperators - including profitability, robust market clearing and pricestabilisation - are successfully achieved. Moreover, the equilibrium pricedynamics induced by SAC-derived optimal policies have characteristics, such asvolatility and seasonality, that closely match those of real-world prices.Remarkably, this adherence to the historical distribution of prices is obtainedwithout explicitly calibrating the model to price data. We show how thesimulator can be used to assess the effects of EU-mandated minimum storagethresholds. We find that such thresholds have a positive effect on marketresilience against unanticipated shifts in the distribution of supply shocks.For example, with unusually large shocks, market disruptions are averted moreoften if a threshold is in place.