This thesis serves three primary purposes, first of which is to forecast twostocks, i.e. Goldman Sachs (GS) and General Electric (GE). In order to forecaststock prices, we used a long short-term memory (LSTM) model in which weinputted the prices of two other stocks that lie in rather close correlationwith GS. Other models such as ARIMA were used as benchmark. Empirical resultsmanifest the practical challenges when using LSTM for forecasting stocks. Oneof the main upheavals was a recurring lag which we called "forecasting lag". The second purpose is to develop a more general and objective perspective onthe task of time series forecasting so that it could be applied to assist in anarbitrary that of forecasting by ANNs. Thus, attempts are made fordistinguishing previous works by certain criteria so as to summarise thoseincluding effective information. The summarised information is then unified andexpressed through a common terminology that can be applied to different stepsof a time series forecasting task. The last but not least purpose of this thesis is to elaborate on amathematical framework on which ANNs are based. We are going to use theframework introduced in the book "Neural Networks in Mathematical Framework" byAnthony L. Caterini in which the structure of a generic neural network isintroduced and the gradient descent algorithm (which incorporatesbackpropagation) is introduced in terms of their described framework. In theend, we use this framework for a specific architecture, which is recurrentneural networks on which we concentrated and our implementations are based. Thebook proves its theorems mostly for classification case. Instead, we provedtheorems for regression case, which is the case of our problem.