Abstract
How to best develop foundational models for time series forecasting remainsan important open question. Tokenization is a crucial consideration in thiseffort: what is an effective discrete vocabulary for a real-valued sequentialinput? To address this question, we develop WaveToken, a wavelet-basedtokenizer that allows models to learn complex representations directly in thespace of time-localized frequencies. Our method first scales and decomposes theinput time series, then thresholds and quantizes the wavelet coefficients, andfinally pre-trains an autoregressive model to forecast coefficients for theforecast horizon. By decomposing coarse and fine structures in the inputs,wavelets provide an eloquent and compact language for time series forecastingthat simplifies learning. Empirical results on a comprehensive benchmark,including 42 datasets for both in-domain and zero-shot settings, show thatWaveToken: i) provides better accuracy than recently proposed foundation modelsfor forecasting while using a much smaller vocabulary (1024 tokens), andperforms on par or better than modern deep learning models trained specificallyon each dataset; and ii) exhibits superior generalization capabilities,achieving the best average rank across all datasets for three complementarymetrics. In addition, we show that our method can easily capture complextemporal patterns of practical relevance that are challenging for other recentpre-trained models, including trends, sparse spikes, and non-stationary timeseries with varying frequencies evolving over time.