Improving long short-term memory (LSTM) networks for arbitrage spread forecasting: integrating cuckoo and zebra algorithms in chaotic mapping space for enhanced accuracy

ICS algorithm

The CS algorithm is derived from modeling the breeding behavior of cuckoos. CS algorithms typically use a real-valued representation scheme, where the position of each cuckoo takes the form of a vector in a continuous search space. For a problem with D dimensions, the position of a cuckoo can be encoded as: $x_i=\left[x_{i1},x_{i2},\dots \dots ,x_{iD}\right]$, where $x_{ij}$ denotes the $j$ th dimension of the first value of the $i$ cuckoo. The algorithm utilized a balanced combination of local randomized wandering and global exploratory randomized wandering, controlled by a switching parameter. The local randomized wandering can be expressed as:

(2)

(3)

(4)

The CS uses random generation to initialize the population, which often results in uneven initialization and fails to cover the entire population. To address these issues, this article proposes using tent chaotic mapping to initialize the population. Tent chaotic mapping introduces randomness, traversal, and sensitivity to initial values, which can accelerate the algorithm’s convergence speed. This ensures that the initial solution is evenly distributed in the solution space. The comparison of the population distribution generated randomly and the population distribution initialized using tent mapping can be found in the Supplemental File. In both scenarios, the population size is set to 100, and after 30 independent experiments. The population distribution tends to be concentrated in one place for populations initialized using random search compared to populations initialized using tent chaotic mapping. Initializing the population using tent mapping leads to a more uniform population distribution. The tent map is a function with a parameter µ and is a segmented linear mapping. It is defined as shown in Eq. (5):

(5)

In the original CS algorithm, the Lévy flight mechanism is used to explore the solution space. As can be seen from Eqs. (3) and (4), the step size of the Lévy flight shows a pattern of frequent short distances and occasional long distances, and its search process allows individuals to easily span a wide range of search regions, resulting in a weak local search capability of the algorithm. When the algorithm needs to search for a local optimal solution, the randomness of the Lévy flight reduces the search efficiency, resulting in the algorithm needing more iterations to converge when approaching the optimal solution.

To solve the shortcomings of the original CS algorithm in terms of local search capability, we introduce ZOA to replace the Lévy flight mechanism while retaining the local search capability of the CS algorithm. ZOA is chosen because it has the following advantages: (1) balanced search strategy, which can strike a better balance between global exploration and local exploitation; (2) adaptive, which can automatically adjust the search strategy according to the current situation; and (3) utilizes group intelligence to guide the search, which can explore the complex solution space more effectively. The specific alternatives are: in the global search phase, the position update mechanism of ZOA is used instead of Lévy flight, so that individuals can move to the global and local optimal solutions more effectively; in the local search phase, the local search mechanism of CS algorithm is retained, which not only retains the advantages of CS algorithm, but also overcomes its deficiencies in local search, and improves the overall performance of the algorithm and the convergence speed.

ZOA also uses a real-valued representation scheme, where the position of each zebra is represented as a vector in a continuous search space: $x_i=\left[x_{i1},x_{i2},\dots \dots ,x_{iD}\right]$. Where $x_{ij}$ denotes the jth dimension of the first value of the $i$ zebra. The zebra search involves two main phases. In the first phase, population members simulate zebra behavior while foraging for food. The most outstanding member of the ZOA acts as the pioneer zebra, guiding others to their positions in the search space. Equations (6) and (7) mathematically model the updating of pioneer zebras’ positions during the foraging phase. Where “r” is a random number between 0 and 1, and “I” is a random value from the set. The second phase models zebra defense strategies against predators. Zebras evade lion attacks using zigzag and random lateral maneuvers. Against smaller predators like hyenas and dogs, zebras adopt a more aggressive strategy, rallying together to confuse and intimidate their attackers. The ZOA considers two scenarios with equal probability: in the first, a zebra hides near its location from a lion (modeled by S1 in Eq. (8)), The search step size of ZOA is adaptively adjusted as the number of iterations increases this allows the algorithm to explore a large area in the early stages and focus more on localized fine-grained search in the later stages; in the second, zebras in the herd gather around an attacked zebra to form a defensive structure (modeled by S2 in reference Eq. (8)). By introducing a probabilistic selection mechanism that switches between different search strategies, it helps to increase algorithmic diversity. In this context, “t” represents the number of iterations, “T” stands for the maximum number of iterations, and “R” is a constant set at 0.01. P is the switching probability of the two strategies. The value is a random number between 0 and 1, and AZ represents the state of the attacked zebra. When updating a zebra’s position, if the zebra has a better objective function value at the new position, that new position is accepted. Modeling this updating condition using Eq. (9).

(6)

(7)

(8)

(9)

To improve the performance of the CS algorithm, in this study, not only did we use tent chaotic mapping to initialize the population, but we also used the idea of the ZOA algorithm in the position update process of the CS algorithm. Updating of the bird’s nest can be like the zebra in the ZOA algorithm, which always pursues the current optimum and the global optimum. This not only maintains the stochasticity required for searching but also reduces the blindness of searching and speeds up the convergence of the population to the optimal solution, and at the same time, the stochastic elimination mechanism of the CS algorithm makes the algorithm able to smoothly escape from the local optimum, thus significantly enhancing its overall performance. The pseudo-code for the algorithm is shown in Box 1. This pseudo-code describes the optimization process of ICS. First, the algorithm receives information about the optimization problem and sets the number of iterations T and the population size N. It initializes the positions of the zebras and evaluates these positions utilizing an objective function. In each iteration, the algorithm updates the locations in two parts: the first part uses ZOA, which simulates the zebra’s foraging behavior and defense strategy, For each nest, the algorithm uses a new location inspired by the ZOA to generate a new location, i.e., the location of the pioneer zebra, and then, the algorithm searches around the pioneer zebra, after which it introduces a probabilistic selection mechanism that balances between two search strategies (when Ps < 0.5) and (when Ps >= 0.5). When Ps < 0.5, a careful local search is performed in the neighborhood of the current solution through an adaptive step size and a random search, and when Ps >= 0.5, an extensive exploration is performed in the whole search space by introducing a random factor and a diverse search; the second part uses CS, which updates the cuckoo’s locations based on the probability of discovery pa (set to 0.25). After each iteration, the current best solution is saved, and the best solution found in all iterations is finally output. The overall time complexity of the ICS algorithm is: $O(T\ast (\mathrm{n}\ast \dim +n\ast f))$, $T$ is the total number of iterations, $n$ is the number of cuckoos, $\dim$ is the positional dimension of each cuckoo, $f$ is the complexity of computing adaptation. In numerous models for hyperparameter optimization, the MSE or MAE of the validation set is typically employed as the fitness function of the model. In this study, we selected MSE as the fitness function.

ICS-LSTM

Structure of ICS-LSTM

The LSTM network is made up of the memory cell and three gates, namely the input gate, forget gate, and output gate. The three gates control the flow of information within the memory cell by determining what information to discard, store, and transmit to the next state. Figure 1 displays the typical structure of an LSTM network with a single hidden layer. The core part of the LSTM unit contained in the hidden layer(s) is the cell state. Let $X_t=\left[x_t^1,x_t^2,x_t^3,\dots \dots x_t^N\right]$ be the N number of inputs, $H_t=\left[h_t^1,h_t^2,h_t^3,\dots \dots {\mathrm{h}}_t^K\right]$ be the K hidden units or hidden states, and $C_t=\left[c_t^1,c_t^2,c_t^3,\dots \dots c_t^K\right]$ be the cell state of the LSTM network at time t and $f_t$, $I_t$ and $O_t$ represent the forget gate, input gate, and output gate, respectively. At each timestamp t, the input $X_t$ along with the previously hidden state $h_{t-1}^1$ is presented to three gates to compute the next hidden state $h_t$ and to update the previous cell state $C_{t-1}$ to compute the new cell state $C_t$. Where $W_f$, $W_i$, $W_c,W_o$ are the weights matrices corresponding to the input $X_t$ and $U_f$, $U_i$, $U_c$, $U_c$ are recurrent weights matrices associated with previously hidden state $h_{t-1}$ and $b_f$, $b_i$, $b_c$ and $b_o$ are the bias vectors for forget gate, input gate, candidate solution, and output gate, respectively. $\sigma \left(x\right)$ is a log-sigmoid activation function and $\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{h}\left(x\right)$ is the hyperbolic tangent activation function.

The ICS algorithm proposed in this article offers strong global search capability, high search accuracy, and fast convergence speed. In this article, we focus on optimizing the hyperparameters of the network model to minimize the influence of human factors on the network model and enhance the predictive ability of the model. Therefore, we consider the time step, discard rate, and the number of neurons in one input layer and two hidden layers of LSTM as the target optimization parameters. Finally, we establish the ICS-LSTM model with the structure shown in Fig. 2 for futures spread prediction.

Test function

We selected the CS and the ZOA, along with the enhanced Cuckoo Algorithm proposed in this article, for testing in the CEC 2019 test function. In CEC 2019, the F1-F3 functions have varying dimension values and ranges, while the F4-F10 functions are 10-dimensional minimization problems. All these optimization benchmarks aim to minimize the target fitness value. Since the search range of CEC-07 is consistently negative during the testing of the three algorithms, which does not align with the requirements of futures spread prediction models, all CEC 2019 function sets except CEC-07 have been chosen. The function introductions of the selected function sets can be found in the Supplemental File. The mean and standard deviation of the three algorithms after 30 independent runs under the nine test functions are shown. From Table 4 it can be seen that the enhanced cuckoo algorithm proposed in this article consistently outperforms the other two algorithms in terms of mean value. The standard deviation of the cuckoo algorithm in the functions F4, F5, F6, and F8 is the smallest among the three algorithms. The ZOA shows the best performance in the functions F2 and F10, while the improved cuckoo algorithm proposed in this article excels in the functions F1, F3, and F9, performing only slightly worse than the other two optimal algorithms in the remaining test functions.

Data processing

The 1-min period K-line fitted spread data of rebar and HRC futures on the Shanghai Futures Exchange from December 34, 2020, to March 15, 2023, was used to evaluate the proposed prediction model. Each period’s price spread data includes eight features: opening price spread, highest price spread, lowest price spread, closing price spread, MACD, DEA, DIF, and price spread fluctuation. The closing price spread is the target for prediction.

Data preprocessing is required before training, and the data preprocessing steps include the following: firstly, data cleaning is performed to remove missing values and error records, and forward or backward padding is used to ensure the continuity of the time series for missing data during non-trading hours; next, min-max normalization is performed on all the features to ensure that the numerical scales of the different features are consistent, to avoid affecting the learning effect of the model; then, the data is divided into a training set (80%), a validation set (10%), and a test set (20%), which are used for model training, hyper-parameter tuning, and final evaluation, respectively; and lastly, technical indicators such as MACD, DEA, and DIF are calculated using the historical price data, which are used as the input features to help the model capture the market trend and the time series pattern. These preprocessing steps ensure the quality and consistency of the data and provide a reliable basis for model training and comparison.

Metrics

To demonstrate the predictive effectiveness of each model, four metrics were used to measure the performance of each model: MSE, MAPE, MAE, and coefficient of determination ( $R^2$). This is calculated by Eqs. (10) to (13).

(10)

(11)

(12)

(13)

(14)

Model implementation

To enhance the efficiency of updating network weights, the Adam optimizer was selected. The Adam optimizer adaptively adjusts the learning rate of each parameter based on historical gradient information, improving the efficiency and stability of the training process. The number of cuckoo populations was set to 10, and the discovery probability for the search was set to 0.25. Five hyperparameters of the LSTM model require optimization: dropout rate, input layer neurons, hidden layer neurons, and time step.

We established a reasonable search range for the parameters to prevent potential issues during the search process. We analyzed research articles (Chen, Zhou & Dai, 2015; Ding & Qin, 2020; Lu et al., 2021) on LSTM models for forecasting the Chinese futures arbitrage market and set the search range of parameters as presented in Table 5. The search range of the time step was set to 3 to 20, the discard rate to 0.002 to 0.99, and the number of neurons for all three layers to 1 to 200. After optimizing the improved cuckoo algorithm, the final optimal parameters are three for the time step, 0.468 for the discard rate, and 109 neurons for the input layer. The number of neurons in the first hidden layer is 107, and the second hidden layer is 98. More on the optimization process can be found in the Supplemental File.

Experimental results and analysis

This section analyzes the capability of ICS-LSTM for predicting spread trends. Models such as multilayer perceptron (MLP), back propagation (BP), LSTM, BiLSTM, and gated recurrent unit (GRU) are widely used in the field of futures arbitrage prediction (Nosratabadi et al., 2021; Zhang et al., 2022; DiPietro & Hager, 2020; Siami-Namini, Tavakoli & Namin, 2019; Farah et al., 2022). MLP and BP are classic neural network models widely used in early time series prediction tasks. However, these models have some limitations. MLP and BP models have a large number of parameters and are prone to overfitting, and they also struggle to capture the long-term dependencies in time series data. In contrast, the ICS-LSTM method introduces the ICS optimization algorithm, which can effectively avoid local optima and improve the model’s search capability and generalization ability. Therefore, the ICS-LSTM method is more suitable for handling complex time series data than MLP and BP methods. LSTM, BiLSTM, and GRU are advanced time series prediction models that can capture the long-term dependencies in the data. However, the optimization process of these models is prone to getting trapped in local optima, leading to poor model performance. The ICS-LSTM method, by introducing the ICS optimization algorithm, can automatically adjust the model’s hyperparameters and improve the model’s search capability and generalization ability. CS-LSTM and ZOA-LSTM are methods that use CS and ZOA algorithms to optimize the LSTM model. These methods can effectively improve the performance of the LSTM model. However, both CS-LSTM and ZOA-LSTM methods have some limitations, and it is difficult to achieve accurate prediction of arbitrage price difference. In contrast, the ICS-LSTM method can automatically adjust hyperparameters and effectively avoid local optima, outperforming other time series forecasting models in the task of arbitrage spread prediction. The main experimental results are shown in Table 6 and Fig. 3.

To be specific, although MLP and BP networks can roughly predict future trends, they are unable to cope with drastic price fluctuations, and there are obvious differences between the prediction curve and the actual curve, especially at the inflection points. The MLP is a basic feed-forward artificial neural network, and it has an MSE of 8.620, a MAPE of 0.016, an MAE of 2.180, and an R2 of 0.989, from the prediction curve and evaluation indexes, it is obvious that the prediction of MLP is not accurate enough; BP is a classical artificial neural network model, and its MSE is 7.760, MAPE is 0.022, MAE is 2.483, and R2 is 0.980, which can be seen that BP’s MSE is better than MLP, but other indexes are not as good as MLP, which indicates that the overall prediction indexes of BP are slightly better than those of MLP. Although the overall prediction index of the BP network is slightly better than that of MLP, the prediction error of the BP network is rather larger in some key data points. The above results show that there are obvious deviations between MLP and BP networks in predicting future trends, especially when dealing with inflection points and drastic fluctuations.

Compared to traditional models like MLP and BP, LSTM excels at capturing time dependencies and reducing prediction errors, thanks to its gating mechanism and memory cells. However, its performance depends heavily on hyperparameter selection, with metrics showing room for improvement: MSE is 14.984, MAPE is 0.017, MAE is 3.410, and R2 is 0.980. In contrast, the error indicators of GRU are 4.218 for MSE, 0.023 for MAPE, 1.574 for MAE, 4.218 for MSE, and 0.994 for R2. It can be seen that the overall prediction indicators of GRU are better than those of LSTM, but the indicator of MAPE is slightly higher, which indicates that the selection of hyperparameters also has a certain effect on the GRU model. However, the combined prediction curves and indicators show that GRU is better than LSTM in terms of prediction accuracy and stability. BiLSTM enhances LSTM by taking into account both forward and backward information in the input sequence to better understand the context. The metrics are as follows: the MSE is 8.922, the MAPE is 0.013, the MAE is 2.397, and the R2 is 0.988. It can be seen that BiLSTM outperforms the LSTM, but its overall error MSE is still somewhat high. After applying the swarm intelligence optimization algorithm to optimize the model parameters, we found that the optimized model has better generalization and fitting ability. The values of MSE, MAPE, MAE, and R2 are 3.705, 0.012, 1.391, and 0.995 for the CS-LSTM. For the ZOA-LSTM, the values are 5.721, 0.017, 1.791, and 0.995 respectively. From the results, it can be seen that both algorithms optimize the LSTM model with significant improvement compared to the baseline LSTM, and the prediction results are more in line with the real curves. Among them, the results of CS-LSTM are more optimized for ZOA-LSTM, indicating that the global search capability of the CS algorithm performs better in the optimization search process.

The MSE, MAPE, and MAE values of the ICS-LSTM model are 3.326, 0.011, and 1.261 respectively. Compared with MLP, BP, LSTM, BiLSTM, GRU, CS-LSTM, and ZOA-LSTM models, the MSE of the ICS-LSTM model is reduced by 61.4%, 57.1%, 77.8%, 62.7%, 21.1%, 10.2%, and 41.8%, respectively. Additionally, the MAPE is reduced by 31.2%, 50%, 35.2%, 15.3%, 52.1%, 8.3%, and 35.2%, respectively, compared to the other seven models. And the MAE is reduced by 42.1%, 49.2%, 63.0%, 47.3%, 19.9%, 9.3%, and 29.6%, respectively. The results for all three indicators indicate that the error between the predicted and actual values of ICS-LSTM is smaller compared to other models, resulting in higher prediction accuracy than other models. In the results of the model evaluation presented in Table 6, the R2 value of ICS-LSTM is the closest to one compared to all other models, suggesting that the model has a stronger fitting ability. All the experimental results show that the predictive ability and prediction effectiveness of the ICS-LSTM model are superior to other methods.

To more rigorously evaluate the performance of the ICS-LSTM model in predicting future arbitrage trends, we introduced the metric of symbol accuracy (SA). The model’s effectiveness in forecasting future price trends is determined by calculating the proportion of predicted values that are symbolically aligned with actual values. Experimental results indicate that the ICS-LSTM model achieves a symbol accuracy of 47.1%, which is notably higher than that of the unoptimized LSTM model (37.4%) and exceeds the performance of other comparative models. This suggests that the ICS-LSTM model effectively captures both upward and downward market movements. By accurately predicting price trends, investors can better conduct arbitrage trades and maximize returns. At the same time, the high symbol accuracy of the ICS-LSTM model can also help investors avoid losses due to price fluctuations. Therefore, the application of the ICS-LSTM model in the futures arbitrage market has great potential and value.

Analysis shows that LSTM architectures capture long-term dependencies in time series data due to their memory units and gating mechanisms. This capability allows LSTM to perform well when dealing with data exhibiting long-term trends or repetitive patterns. However, the model’s performance is highly contingent on the selection of hyperparameters, such as the number of neurons, time steps, and dropout rates. The ICS method enhances the traditional cuckoo algorithm by introducing a more flexible search mechanism, thus avoiding local optima in the complex optimization space, which is critical for improving the prediction accuracy of LSTM models. By leveraging the LSTM’s robust handling of time-series data and the efficiency of the improved cuckoo algorithm in hyperparameter optimization, the ICS-LSTM model demonstrates a significant enhancement in futures arbitrage prediction accuracy.