Abstract
The Transformer model has shown leading performance in time seriesforecasting. Nevertheless, in some complex scenarios, it tends to learnlow-frequency features in the data and overlook high-frequency features,showing a frequency bias. This bias prevents the model from accuratelycapturing important high-frequency data features. In this paper, we undertookempirical analyses to understand this bias and discovered that frequency biasresults from the model disproportionately focusing on frequency features withhigher energy. Based on our analysis, we formulate this bias and proposeFredformer, a Transformer-based framework designed to mitigate frequency biasby learning features equally across different frequency bands. This approachprevents the model from overlooking lower amplitude features important foraccurate forecasting. Extensive experiments show the effectiveness of ourproposed approach, which can outperform other baselines in different real-worldtime-series datasets. Furthermore, we introduce a lightweight variant of theFredformer with an attention matrix approximation, which achieves comparableperformance but with much fewer parameters and lower computation costs. Thecode is available at: https://github.com/chenzRG/Fredformer