NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

  • 2021-02-10 18:36:11
  • Kai Chen, Guang Chen, Dan Xu, Lijun Zhang, Yuyao Huang, Alois Knoll
  • 39

Abstract

Although Transformer has made breakthrough success in widespread domainsespecially in Natural Language Processing (NLP), applying it to time seriesforecasting is still a great challenge. In time series forecasting, theautoregressive decoding of canonical Transformer models could introduce hugeaccumulative errors inevitably. Besides, utilizing Transformer to deal withspatial-temporal dependencies in the problem still faces tough difficulties.~Totackle these limitations, this work is the first attempt to propose aNon-Autoregressive Transformer architecture for time series forecasting, aimingat overcoming the time delay and accumulative error issues in the canonicalTransformer. Moreover, we present a novel spatial-temporal attention mechanism,building a bridge by a learned temporal influence map to fill the gaps betweenthe spatial and temporal attention, so that spatial and temporal dependenciescan be processed integrally. Empirically, we evaluate our model on diversifiedego-centric future localization datasets and demonstrate state-of-the-artperformance on both real-time and accuracy.

 

Quick Read (beta)

loading the full paper ...