Optimizing Long Short-Term Memory to Predict Currency Rates

As a travel destination, Saudi Arabia attracts individuals worldwide, including tourists, investors, and immigrant workers, for various purposes, including trip planning, investment decisions, and remittance transfers. Indonesia and Pakistan are the biggest countries that send Umrah and Hajj pilgrims. We need to predict currency rates in 3 pairs of currencies that are frequently used by travel agencies, Hajj and Umrah pilgrims, such as the Saudi Riyal (SAR) against the Pakistani Rupee, the SAR against the Indonesian Rupiah (IDR), and the United States Dollar (USD) against the IDR. This study utilizes Long Short-Term Memory (LSTM) models, the machine learning approach for predicting currency pairs exchange rates. Previous studies succeeded in predicting USD/IDR rates using the LSTM time series-machine learning approach, but the root mean square error (RMSE) value was the worst 271. The research aims to optimize the LSTM to predict the currency rate in the future using historical data obtained from investing.com. We use Python to predict the currency rate pairs, following an experimental investigation with adjustments to the batch size, epoch, and prediction days. The experimental results show that SAR/PKR has a smaller mean square error (MSE) of 0.94, RMSE of 0.97, and MAE of 0.61, while SAR/IDR and USD/IDR Excel with Models 2 and 1 have smaller MSEs of 317.79 and 6654.41, RMSEs of 17.82 and 81.57, and MAEs of 10.54 and 50.12, respectively.


I. INTRODUCTION
As one of the world's most visited countries, Saudi Arabia attracts tourists, investors, and a significant immigrant workforce.Indonesian and Pakistani pilgrims are the biggest countries that visit Makkah and Madinah for Umrah and Hajj [1].SAR and US dollars are important currencies for travellers and travel agents that use transactions.Travel agencies do not have a reference to help them decide on a good strategy for buying SAR or USD for payment for travel accommodation.For that reason, in this study, we need to predict currency rates in 3 pairs of currencies that transactions frequently Hajj and Umrah pilgrims, such as the SAR/PKR, SAR/IDR, and USD/IDR.
An exchange rate means the rate of one currency can be swapped for another currency.It plays a crucial role in global trade and the flow of funds across borders [2].Research results [3] have applied the Markov Chain method to forecast the trade price between the SAR and IDR currencies.The research uses time series data from Bank Indonesia's official website to predict the future currency rate.The study uses a model that combines the Fuzzy Time Series and Markov Chain techniques to predict the exchange price.The research results show that the model has a small error, with an AFER of 0.827% and an MAE of Rp32.96 [3].During the COVID-19 pandemic, researchers [4] attempted to forecast USD/IDR exchange rates.Their findings indicate that the LSTM model is the most effective model for long-term prediction, even though it did not account for COVID-19 events.The model projected a decline in the Rupiah's value against the US dollar, amplified by the pandemic's impact.The RMSE showed a lower value, registering at 271 during the testing phase, and the algorithm had 7 epochs and 5 neurons.The best result is the LSTM algorithm.The weakness of this model is that it only works in the COVID-19 period [4].Researchers [5] chose the Euro against the USD as the data for financial foreign exchange.The experimental outcomes of Utilizing RNN and LSTM network models for forecasting exchange rate prices reveal that the LSTM model's predictive performance surpasses the RNN's.This underscores that the distinctive LSTM algorithm is more adept at comprehending historical exchange rate data and unravelling the intricate relationships within time series data.The authors optimized LSTM models through manual adjustment of hyperparameters.The parameters along the model training are modified, including layer number, dropout, cell number, learning rate, epoch, batch size, and time step.The experimental results show that the LSTM algorithm has a low RMSE value of 75.17 and a mean absolute error of MAE of 57.54 than the RNN network algorithm [5].
In 2023, research results [6] discovered that LSTM outperformed other methods in predicting EUR/USD.encompassed preprocessing Forex data and Developing an LSTM algorithm utilizing widely recognized deep learning frameworks.The LSTM algorithm outperformed the other evaluated models, achieving the best results after 90 epochs.This model achieved the lowest MSE value of 0.0001794, the lowest MAE value of 0.01127, and the highest R-squared value of 0.9779.The researchers optimized LSTM models by adjusting various parameters to improve their performance in Forex prediction.These parameters encompass the number of neurons in the LSTM layer(s), the count of LSTM layers, the dropout rate, the learning rate, and the optimizer [6].The research discusses using the LSTM algorithm for forecasting foreign swap rates during the COVID-19 pandemic.Various factors influence the foreign exchange market and are difficult to predict accurately.The study implemented LSTM models to forecast the Euro against the USD exchange rate in 2020 using hourly and daily timeframes.The high-performance hyperparameters for the models were identified by assessing the root mean square error (RMSE) evaluation results.The predictions for 2020 were then compared to those for 2018 and 2019.The LSTM model demonstrates its ability to yield favourable results in the 2020 prediction, as evidenced by the RMSE result of 0.00135.The authors optimized the LSTM models by selecting the best hyperparameters derived from the root mean square error (RMSE) evaluation results.The hyperparameters considered were the number of hidden layers, the unit of neurons, and the presence or absence of a dropout layer.The weakness of this model is only working in COVID-19 [7].
In 2020, Wijesinghe focused on comparing the performance of LSTM neural networks with other models in predicting exchange rates of currencies.The study uses three different datasets of foreign exchange rates, including GBR/USD, USD/CAD, and AUS/USD.The author optimizes the LSTM model by determining the quantity of hidden layers and cells in each layer.The author also experiments with different values of a scaling factor.In this case, they find that a scaling factor of 8 performs the best by using metrics like RMSE, MAE, and MAPE to assess the performance and determine the superiority of the LSTM model [8].Yadav et al. conducted a study to discuss optimizing LSTM models for time series forecasting in the Indian stock exchange.The LSTM algorithm is enhanced by comparing stateless and stateful models and fine-tuning the number of hidden layers.The stateless approach means it does not maintain any memory of previous batches of data.
In contrast, the stateful approach in LSTM maintains the state or memory of the model between batches.The study's limitations include the limited computational resources and the need for further experiments with larger datasets and longer training periods [9].In 2015, Sun Rich conducted a study in China, employing BP neural networks, RNN neural networks, and LSTM neural networks to predict short-term stock prices.They also assessed the model's accuracy.Furthermore, they verified the model's practicality and precision, considering the distinct characteristics of both the Chinese and American stock markets [10].LSTM provides a better way to forecast time series cases like currency rates or stock markets [11,12].

II. METHODS
The study methodology used for data collection in this research is quantitative.The research design in Fig. 1 contains the stages of planning that will be conducting this research.There are several stages to be carried out, including data collection, scaling, entering variables, adjusting lookback, hyperparameter tuning, running the model, evaluation metrics, testing the model predicted, calculating the metrics and predicted exchange rates.The dataset used in this study was obtained from Investing.com, a reputable financial data provider.The dataset includes historical time series daily data on the closing rates (price) for the three currency pairs: SAR/PKR, SAR/IDR, and USD/IDR.The data spans a specific period in Table I.The dataset period for training is 80%, testing is 20%, and evaluating the machine learning models.These features represent past observations of exchange rates at various time intervals, allowing the models to learn from historical trends, like price, open, high, low, change %, and date.The primary objective of this paper is to forecast the closing price at time t+1.We divided the dataset into two subsets, one designated as the training set and the other as the testing set.These sets are utilized to train and test the algorithms, respectively.For more detail, Fig. 2 is the SAR/PKR currency pair training data, Fig. 3. is the SAR/PKR currency pair testing data from investing.com,and so on for other currencies, then enter the data into Python (Google Colab).In the data preparation phase, we used feature selection, reshaping data sets, normalization, and partitioning data to train and test datasets.We chose the daily price of all currency pairs for training models as the feature.An 80% portion of the data spanning almost 20 years was allocated for model training, while the remaining 20% (almost five years) served as the testing set.The MinMaxScaler function was employed for scaling, ensuring currency pair prices were scaled to the range [0,1].Feature scaling is a preprocessing technique employed in machine learning and data analysis to standardize all input features, ensuring they share a consistent scale [13].
The primary parameters fine-tuned during model training are batch size, epochs, and prediction days (lookback/sliding window).We are preparing training data of historical data and corresponding target values to learn how to predict the value for the next day based on the previous days of data.A hyperparameter approach is used in this study to improve the model's forecasting outcome.
We employed LSTM with 1 to 4 hidden layers to initiate our analysis.These models incorporate 50 neurons within each hidden layer.The training procedure employs the Adam optimization algorithm, and parameter initialization remains consistent across models.We have three models for each currency pair; model 1 is prediction days obtained 30, the batch size is 20, and it undergoes 50 training iterations (epochs).The three models will apply to every currency pair.For more details, we can see the model type in Table II.The batch size determines the sample days utilized in every training epoch.
In contrast, the iteration (epochs) quantity signifies how frequently the entire data set is passed through the model.The number of prediction days, iteration, and batch size are prioritized in this research to determine the best model.This improves the algorithm's efficiency and the training procedure's reliability.After getting the prediction model, we can test the model with testing data, see the performance metrics of the model, and then predict the prices.

A. Long Short-Term Memory (LSTM)
LSTM is an effective time series model for handling sequential data.The architecture includes input, LSTM, and output layers with appropriate activation functions [14].The first step involves determining which information to omit from the last cell state Ct−1.This is done through a sigmoid layer denoted as ft, taking the previous hidden state ht−1 and the current input x-tas input.The sigmoid function outputs values between 0 and 1 for each element in the cell state Ct−1.This operation is defined using Equation (1) [15].
Here, σ represents the sigmoid activation function, Wf and bf are the weight and bias parameters for the overlook gate.Input Gate Layer: The subsequent step involves deciding what new information to save in the cell state.It involves two parts: • The input gate layer (it), determines what values to renew.
• A tanh layer that produces a vector of recent candidate rate, Ct.
The input gate and candidate rate are defined as Equation ( 2) and (3).
Renew to the Cell State: With the input gate and new candidate values in place, this is the time to upgrade the last cell state Ct−1 to the recent Ct.This is performed as Equation ( 4).
Output Gate Layer: The final step determines what to output.The output follows the cell state Ct but in a filtered form.First, a sigmoid layer represented as ot determines which component of the cell state to output.After that, the cell state is passed beyond a tanh function, and the result of the sigmoid gate ot augments the result.The output is defined as Equation ( 5) and (6).

B. Evaluation
The model's performance was evaluated using various metrics such as mean absolute error (MAE), mean squared error (MSE), and root mean squared error (RMSE).These metrics were calculated on the validation dataset to assess each model's accuracy and generalization capabilities.
1) The MSE, where yactual represents the actual humidity, yforecast is the forecasted humidity, and n is the data number.Hence, accuracy is often calculated using Equation ( 7) [16].
2) The RMSE measures the standard deviation of residuals.The Equation ( 8) of recall [16].
3) The MAE measures the average magnitude of errors within a set of forecasting, irrespective of their way.With MAE, we can compute the mean absolute deviation between the forecast result and the actual price, providing a measure to assess the outcome of a regression model [17].The MAE value is calculated using Equation ( 9).

III. RESULTS AND DISCUSSION
The LSTM model underwent training through a supervised learning method involving the provision of input and output/result sequences.The algorithm was then trained to make predictions on the target sequences, guided by the input sequences.Performance evaluations use several different metrics to determine the efficacy of the model.These metrics included MSE, RMSE, and MAE.Table III Comparisons of evaluation indices of currencies set.A tabular representation was generated to display the outcomes of various LSTM models, each trained with different models.These models were assessed based on their performance in terms of MAE, RMSE, and MSE values.MSE quantifies the mean squared error between predicted price and existing prices, where a smaller MSE signifies a more outperform.RMSE is the square root of the MSE between predicted and existing prices.It indicates how much the predictions deviate from the existing data.MAE computes the average absolute deviation between predicted and existing prices, with a smaller rate signifying the best match.
Based on the data presented in the table, it is evident that Model 1 stands out as the top performer and is suitable for currencies pairs USD/IDR if we compare other models, especially only in currency USD/IDR with MAE 50.12,Model 2 with 51, and Model 3 are 103.22.Model 2 outperforms the other models, making it well-suited for the SAR/IDR currency pair.In direct comparison, Model 2 exhibits a lower Mean Absolute Error (MAE) of 10.54 for SAR/IDR, surpassing Model 1 with an MAE of 13.68 and Model 3 with a higher MAE of 23.65.
Additionally, distinctive observations emerge when considering the SAR/PKR currency pairs, while Model 1 showcases favorable performance metrics in Table IV.In the context of SAR/IDR, SAR/PKR, and USD/IDR, Model 2 and Model 1 demonstrate predictions that closely align with the actual prices.The graphical representations in Fig. 8, Fig. 9, and Fig. 10 for the SAR/PKR currency pair support the results of training the model.Likewise, in the case of SAR/IDR represented in Fig. 11, Fig. 12, and Fig. 13, as well as USD/IDR represented in Fig. 14, Fig. 15, and Fig. 16, the outputs indicate the graphical pattern of both currency pairs is same in the actual prices and the prediction.The value of a country's currency is subject to constant fluctuations, both rising and falling, concerning another country's currency.These variations are driven by multiple factors, with one notable influence being the economic principle of supply and demand in Saudi Riyal currency transactions [19].Notably 2022, from September to December, there was a notable surge in visitor pilgrims during the Umrah season, leading to an increased demand for Riyal currency compared to the period from April to June.    4, on 2023-10-23, the actual prices were 15950 for USD/IDR, and the predicted were 15847.The actual price was 4247 for SAR/IDR 4219 for the predicted prices, 74.12 for the actual price of SAR/PKR, and 61.95 for the predicted price.The experimental outcomes, as shown in Table III, consistently demonstrate the superior predictive accuracy of the LSTM model when forecasting exchange rate prices.Overall, the study's results underscore the effectiveness of the LSTM algorithm in foreign exchange forecasting and the potential advantages of employing deep learning processing in such predictive applications.the preprocessing of currency rates historical data, the LSTM algorithm applying prevalent deep learning frameworks, and the appraisal of its performance via metrics such as mean squared error, root mean squared error, and mean absolute error.We can optimize the LSTM model without changing the learning rate, neurons, or hidden layers by experimenting with sliding windows, epochs, and batch size changes.

IV. CONCLUSION
The experimental findings prove the effectiveness of the LSTM model in currency trade rate forecasting.The algorithm's aptitude for capturing intricate relationships within currency trade rates and outperforming positions it as a favorable tool for predicting future currency rates.For travel agencies or individual travelers, particularly those embarking on Umrah and Hajj journeys, it is highly advisable to consider purchasing SAR/IDR and USD/IDR between April and June.During this period, the prices tend to be lower.Additionally, the current trend for SAR/PKR is upward.It is recommended to purchase at this time before the trend further escalates.For future research, we recommend conducting a comparative analysis of machine learning models, including LSTM, Gaussian Mixture Model, and Support Vector Regression.This will help determine the most effective algorithm among these models.

Fig. 10 .Fig. 12 .Fig. 14 .Fig. 16 .Fig. 17 ,
Fig.10.Model 3 Predicted Rates Training for SAR/PKR Fig.11.Model 1 Predicted Rates Training for SAR/IDR Fig.17, Fig.18, and Fig.19 depict a comparison among Model 1, Model 2, and Model 3, revealing that Model 3 significantly deviates from the actual prices.Notably, in May 2020, as illustrated in Fig.23, Fig.24, and Fig.25, during Indonesia's struggle with a pandemic, the USD/IDR and SAR/IDR exchange rates in the testing data (Fig.20, Fig.21, and Fig.23) experienced a substantial surge.This observation indicates that the model demonstrated a remarkable ability to predict exchange rates accurately, even amid crises.The value of a country's currency is subject to constant fluctuations, both rising and falling, concerning another country's currency.These variations are driven by multiple factors, with one notable influence being the economic principle of supply and demand in Saudi Riyal currency transactions[19].Notably 2022, from September to December, there was a notable surge in visitor pilgrims during the Umrah season, leading to an increased demand for Riyal currency compared to the period from April to June.For instance, the value of the Riyal currency typically hovers around Rp 3,950 per Riyal from April to June.However, it experienced an upswing to Rp 4,143 per Riyal during the Umrah in Ramadhan and Hajj season from February to March of 2022 and 2023.The trend starts to ascend in August.