Abstract
With the rapid development of artificial intelligence, long short term memory(LSTM), one kind of recurrent neural network (RNN), has been widely applied intime series prediction. Like RNN, Transformer is designed to handle the sequential data. AsTransformer achieved great success in Natural Language Processing (NLP),researchers got interested in Transformer's performance on time seriesprediction, and plenty of Transformer-based solutions on long time seriesforecasting have come out recently. However, when it comes to financial timeseries prediction, LSTM is still a dominant architecture. Therefore, thequestion this study wants to answer is: whether the Transformer-based model canbe applied in financial time series prediction and beat LSTM. To answer this question, various LSTM-based and Transformer-based models arecompared on multiple financial prediction tasks based on high-frequency limitorder book data. A new LSTM-based model called DLSTM is built and newarchitecture for the Transformer-based model is designed to adapt for financialprediction. The experiment result reflects that the Transformer-based modelonly has the limited advantage in absolute price sequence prediction. TheLSTM-based models show better and more robust performance on differencesequence prediction, such as price difference and price movement.