Deep Learning Transformer for Forecasting Sumatran Light Crude Oil Price

Time series are often used to project cases from time to time. To get a good forecast, historical data is needed. There are two different approaches to modeling data for forecasting which are the parametric and nonparametric approaches. In parametric, there are some assumptions needed for modeling the data. It makes this method difficult to be applied to real data. Nonparametric models, on the other hand, do not require fulfillment of assumptions like parametric methods and can be an alternative to solve these problems. Currently, deep learning methods are most often used to predict and forecast various cases. The transformer is one such deep learning method. It is a Recurrent Neural Network (RNN) based method that was first used for NLP cases and has been developed to be able to be employed for data series, especially for forecasting. Self-attention was used in this algorithm to capture the long sequence data dependencies and integrate them into the forecasting modeling. Even though the information is significant in its long position in the previous period and spreads in time order, transformers can still involve it in the modeling. This is because the transformer is able to maintain a direct connection to all significant earlier timestamps. In this study, transformer model will be used to forecast Sumatran Light Crude Oil (SLC) price. This method will employ a different number of heads and layers, although the dimension of the model and the number of input and output windows are the same. The success of this method in SLC price forecasting performs optimally with 16 heads and 4 layers and gives a model RMSE of 8.19818.