Abstract
The emergence of abundant electricity time series (ETS) data provides ampleopportunities for various applications in the power systems, includingdemand-side management, grid stability, and consumer behavior analysis. Deeplearning models have advanced ETS modeling by effectively capturing sequencedependence. Nevertheless, learning a generic representation of ETS data forvarious applications remains challenging due to the inherently complexhierarchical structure of ETS data. Moreover, ETS data exhibits intricatetemporal dependencies and is suscepti ble to the influence of exogenousvariables. Furthermore, different instances exhibit diverse electricityconsumption behavior. In this paper, we propose a foundation model PowerPM tomodel ETS data, providing a large-scale, off-the-shelf model for power systems.PowerPM consists of a temporal encoder and a hierarchical encoder. The temporalencoder captures both temporal dependencies in ETS data, considering exogenousvariables. The hierarchical encoder models the correlation between hierarchy.Furthermore, PowerPM leverages a novel self-supervised pretraining frameworkconsisting of masked ETS modeling and dual-view contrastive learning, whichenable PowerPM to capture temporal dependency within ETS windows and aware thediscrepancy across ETS windows, providing two different perspectives to learngeneric representation. Our experiments involve five real world scenariodatasets, comprising private and public data. Through pre-training on massiveETS data, PowerPM achieves SOTA performance on diverse downstream tasks withinthe private dataset. Impressively, when transferred to the public datasets,PowerPM maintains its superiority, showcasing its remarkable generalizationability across various tasks and domains. Moreover, ablation studies, few-shotexperiments provide additional evidence of the effectiveness of our model.