Hybrid parking space prediction model: integrating ARIMA, Long short-term memory (LSTM), and backpropagation neural network (BPNN) for smart city development

Auto regressive integrated moving average (ARIMA)

ARIMA handles the linear components of time series data effectively, modeling trends and seasonality in the short term. The ARIMA model predicts future values of the dataset based on its historical values and observed patterns (Medina-Salgado et al., 2022). ARIMA model uses a statistical method to predict available parking space in the time series dataset. It combines two components, autoregressive (AR) and moving average (MA), to predict future values based on past observations. Equation (1) is used to predict the Time-series dataset:

(1) $$\mathrm{y}(\mathrm{t})=\mathrm{c}+\sum (\phi (\mathrm{i})\ast \mathrm{y}(\mathrm{t}-\mathrm{i}))+\sum (\theta (\mathrm{i})\ast \epsilon (\mathrm{t}-\mathrm{i}))+\epsilon (\mathrm{t}).$$

In Eq. (1), y(t) represents the value of the time series dataset at time t, c is used as constant, φ(i) is used as lags coefficient for autoregressive value, θ(i) is used for moving average and ε(t) is used for error at time t.

Long short-term memory model (LSTM)

LSTM handles nonlinear patterns in the time series dataset and is the ideal choice for predicting available parking spaces. LSTM can retain information for extended periods (Awan et al., 2020). This is due to their proficiency in identifying and handling unknown temporal lags between critical data occurrences (Begum et al., 2022). The three primary gates that make up the fundamental architecture of an LSTM are the input (IN), output (OP), and forget gates. A memory cell plays a crucial role in the LSTM’s ability to store and preserve prior knowledge. It is a hybrid with the ARIMA model because it can handle complex, non-linear data patterns efficiently. It particularly addresses the limitations of the ARIMA model, known for its proficiency with linear data, providing a more sophisticated approach to predicting parking space occupancy (Gupta et al., 2022).

Back propagation neural network (BPNN)

BPNNs consist of an input layer, an output layer, and one or more hidden layers. It propagates errors from the output layer to hidden layers, allowing for weight adjustments and reducing prediction errors. It includes forward and backward propagation, similar to the backward error propagation method (Xiao et al., 2023). Error if occurs is sent back to the system in the opposite direction. This reverse process helps the model learn different aspects of the systems, such as weights and biases, to reduce errors and improve the prediction. This process is repeated until we get the desired result. This process helps the neural network to make accurate predictions.

Proposed hybrid model (ARIMA + LSTM + BPNN)

The proposed hybrid model is designed at two phases. In the first phase, data is preprocessed and fed into the proposed hybrid model that combines the ARIMA and LSTM models. ARIMA handles the linear components of the time series data, and LSTM handles the nonlinear components of the data. The LSTM model is hyper-tuned to get the best prediction from the model. Results obtained from the first phase are fed into the second phase, which uses BPNN to optimize the performance of the proposed hybrid model.

In training phase (Phase 1), linear trends in historical parking data are handled well with the ARIMA model. It allows the model to learn from past values. Further, nonlinear data which frequently appears in time series datasets is handled by LSTM, making it suitable for complex patterns in parking space usage. The predicted outcomes from Phase 1 are then passed to Phase 2 through the BPNN integrated at the output layer to optimize the predictions and fine-tuned the model’s output based on the learned patterns from ARIMA and LSTM. The key features of the proposed hybrid model are listed in Table 2.

The above table highlights the key features of the proposed hybrid model. It highlights how combining all three approaches improves the overall accuracy and robustness of the parking space prediction. The pseudo-code representing the workflow of the proposed hybrid model is shown in the next section.

Pseudo code: proposed hybrid model for parking space prediction

The pseudo-code presented above is used to implement the proposed hybrid model. It provides a step-by-step description of different phases of the proposed hybrid model. The working model’s flow chart is presented in Fig. 1.

The steps of the proposed hybrid model are outlined below:

Step 1: Preprocess the dataset. As we are working on the sensors dataset, it generates a lot of noise, so to get an accurate prediction; the first step is to preprocess the data to remove anomalies from the dataset.

Step 2: Parking space prediction dataset comprises a linear component (Lt) and nonlinear component (NLt). To analyze the dataset, the “statsmodels” package in Python is used to decompose the time-series dataset obtained from IoT sensors.

Decomposition aids in examining the seasonality, trend, and residual values of the feature employed for predicting parking space availability. Mathematical illustration of the raw dataset (yt) for parking space prediction is shown in Eq. (2).

(2) $${\mathrm{Y}}_{\mathrm{t}}={\mathrm{L}}_{\mathrm{t}}+\mathrm{N}{\mathrm{L}}_{\mathrm{t}}.$$

Step 3: After observing that the dataset exhibits seasonality, the linear parts of the time-series dataset fed into the ARIMA model for predicting parking space availability. The predicted result of the ARIMA model, denoted as ( ${\overline{L}}_t)$, is obtained using Eq. (3). The “pmdarima” package is employed to determine the optimal parameters for the ARIMA model in order to improve the accuracy of the parking space predictions.

(3) $${\hat{L}}_t=ARIMA\left(Y_t\right)$$

Step 4: The residual quantities (NL_t) of the ARIMA model are computed using Eq. (4) and then utilized as input for the LSTM model in the parking space prediction. Using ARIMA residuals as input for the LSTM model enhances prediction accuracy by allowing the LSTM to capture unresolved variations and complex patterns, creating a more robust prediction model.

(4) $$N{L}_t=Y_t-{\hat{L}}_t.$$

Step 5: To process the nonlinear values of the time-series dataset, the LSTM model is employed, and Eq. (5) is used to predict more complex patterns in the dataset.

(5) $${\widehat{NL}}_t=LSTM\left(N{L}_t\right).$$

Step 6: The integration of ARIMA and LSTM forecasting values is optimized using BPNN. BPNN plays an important role in combining the linear and nonlinear predictive values represented by the function “f” in Eq. (6). BPNN optimizes the results to propagate error values backward in the neural network and adjust neuron weights in the forward direction, for enhancing the accuracy of parking predictions not achieved by existing work done by the researchers.

(6) $${\hat{Y}}_t=f{\hat{L}}_t+N{L}_t.$$

By utilizing BPNN, the model effectively predicts $\left({\hat{Y}}_t\right)$ values by integrating the predicted results of ARIMA $\left({\hat{Y}}_t\right)$ and LSTM $(N{L}_t)$. Unlike existing models, the proposed hybrid model (ARIMA + LSTM + BPNN) improved the prediction results obtained by the combination of ARIMA and LSTM. Instead, a BPNN is employed to establish the true functional relationship among the forecasted values of ARIMA and LSTM, highlighting the novelty of this existing model.

Dataset used

The Melbourne public dataset used to implement the proposed hybrid model is a comprehensive collection of data spanning two months, from January to February 2018 (City of Melbourne, 2024; https://data.melbourne.vic.gov.au/explore/dataset/on-street-car-parking-sensor-data-2018/information/). It encompasses various aspects of the city, including geographical information and parking sensor data. The dataset provides valuable insights into the parking situation in Melbourne during that time period (Zheng, Rajasegarar & Leckie, 2015).

First, the model is implemented on the Melbourne dataset. Further, to check the generalizability of the proposed hybrid model, it is again implemented on the Harvard dataset (https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YLWCSU), which includes both SFpark sensor data and Simulated Crowd-Sensing data. It collects data from different networks of sensors positioned in parking spaces around the city. The data includes information such as when a parking space is available, location, and occupancy status. Different attributes of the dataset include Block_Id, Block Start_Time, and many more. These two different models are used to check the proposed model’s performance in different environments to check the robustness of the proposed hybrid model.

Data preprocessing

The most important step before implementation is to preprocess the dataset so that the dataset is ready to make accurate predictions. Sensors generate a large amount of data that contains a lot of noise. As time-series dataset is used it need to be preprocessed, identifying the patterns from the dataset and then extracting the relevant features to enhance the accuracy of the proposed model (Kasera & Acharjee, 2022). Mean-filling is used to remove noise and missing values from the dataset. This method efficiently handles the missing values from the dataset. This step is essential for enhancing the accuracy of predicting available parking space, ultimately enhancing the predicting accuracy of the proposed hybrid model. The equation below is used to fill inconsistencies in the dataset.

(7) $$\overline{m}\left(t\right)=1d\sum _{i=1}^d{m}_i\left(t\right).$$

In Eq. (7), $\overline{m}\left(t\right)$ is used to find the average of the time period t, m_i(t) is used as actual value and d as no. of days.

In certain parking slots, sensors generate the time-series data, as they are activated at 7:30 am and deactivated around 6:30 pm. This will make substantial fluctuations in the parking dataset. Therefore, the first and the last data points of each day are pruned to mitigate the problem of fluctuations in the parking dataset, which will ultimately result in more accurate analysis and enhance the predicting capability of the proposed hybrid model.

Time-series datasets contain different values, so the first task is to understand the seasonality, trends, and residuals from the available data. This step is necessary before the analysis and is necessary for accurately predicting available parking. A time-based splitting technique is used to recognize the temporal nature of the dataset. The proposed model’s performance is evaluated by employing systematic training and testing. The time-series dataset is divided into training sets and testing sets, and analysis is done in a different environment.

The parking dataset contains variations to ensure the proposed model’s performance (robustness and scalability) with data variation; both the datasets Melbourne and Harvard are divided into three different training and testing parts for implementation and evaluation purposes. In the first part, the dataset is divided into 70% training dataset and 30% testing dataset. In the second phase, the dataset is divided into different ratios, which are 80% for training the model and 20% for testing the model. This division provides a more extensive training set to the model so it can learn efficiently to predict the result. In the last phase, the dataset is divided into ratios, with 90% used for training and 10% used for testing the proposed hybrid model to predict available parking space.

In both of the datasets mentioned above, the proposed hybrid model consistently demonstrates the best performance with the lowest error values in different training and testing scenarios, indicating its superior predictive accuracy across different environments, as both datasets have different specifications. The main focus is maximizing the data used for training, which could result in a more refined and optimized model. By evaluating the proposed hybrid model across these different scenarios, we can comprehensively understand its performance and assess its effectiveness in predicting parking space availability. Optimizing its performance is essential to ensure the scalability of the proposed hybrid model.

Decomposing the parking dataset is necessary to identify better long-term trends, seasonal fluctuations, random noise, and many more. Firstly, both the parking datasets used to implement the proposed hybrid model must be decomposed. Decomposition helps us to determine the appropriate ARIMA model to use. Figure 2 illustrates the decomposed outcome of the Melbourne parking dataset used in this research. The decomposition analysis detects the presence of seasonality in the dataset, indicating recurring patterns over time but no discernible trend. Also, residual values are evident in the dataset, requiring careful consideration and handling in the modeling process.

Figure 3 provides the same comprehensive view of the Harvard parking dataset as done for the Melbourne dataset. It will help to analyze parking occupancy fluctuates with different times, and it breaks complex data into more interpretable components. This decomposition can facilitate better prediction and help the planners to manage the parking resources. It will provide in-depth knowledge that will help identify the busy periods of different parking slots and guide the development of strategies to manage occupancy effectively, which will improve user experience and optimize resource allocation.

We carefully examined both datasets using the Seasonal Decomposition of Time Series (STL) method. This method will break down the data into trends, seasonality, and residuals. With the STL decomposition, the trend part reveals the long-term shifts or variations in parking occupancy data. It allows us to spot any general trends over time. “When looking at parking occupancy, the seasonal part involves the patterns that happen at certain times, like daily, weekly, or monthly cycles. We use the ARIMA model to manage the sequential data in the seasonal dataset. We input both the testing and training data into this model to identify the fundamental patterns and trends. The results are shown in the next section after providing the training and testing data inputs to the ARIMA model.

Implementation phase

In this section, different phases of the proposed hybrid model implementation are discussed in detail. This section provides step-by-step training and testing results of the implementation of the proposed hybrid model for predicting available parking spaces.

ARIMA model

After decomposing both datasets, linear trends from the dataset are fed into the ARIMA model in Phase 1, as briefly discussed in “Data preprocessing”. To find the right parameters for the ARIMA model, we use the ‘auto arima’ function in the ‘pmdarima’ package. This function picks the best values for the parameters (p, d, q) and (P, Q, D, s) by looking for the lowest Akaike information criterion (AIC) value. Once executed on the training dataset, the “auto arima” function determines the selected parameters (p, d, q), (P, Q, D, s) as (2, 0, 0), (0, 1, [1], 4), with an associated AIC value of 520.056. These parameter values are crucial for accurately modelling and predicting the seasonal variations in the parking occupancy data using the ARIMA model.

Figures 4 and 5 display the ARIMA model’s diagnostic results on different datasets used for implementing the proposed hybrid model. The plot includes a standardized residual plot. The standardized residual plot indicates that the residual errors mostly revolve around zero, indicating the likelihood of error for the linear component of the data.

Figures 6 and 7 represent the predicted results of the ARIMA model after training the model as discussed earlier, which are divided into three different scenarios.

This section provides the predicted results of the ARIMA model. In the next section, LSTM is discussed in detail.

LSTM model

The residuals obtained from the ARIMA model are provided to the LSTM model in Phase 1 of the proposed hybrid model. LSTM model is hyper-tuned to predict the available parking space. It’s important to deal with the difference between the actual and predicted occupancy values to get precise results. The LSTM model bridges the gap and improves prediction accuracy. We split the residuals obtained from the ARIMA model into training, validation, and testing sets before adding them to the LSTM model. After that, we check the model’s performance and how it accurately predicts the available parking space. Additionally, we adjust the leftover numbers using the ‘MinMaxScaler()’ function to make training and prediction work better. MinMaxScaler() function will set the values between 0 and 1, helping the LSTM model understand the trends efficiently. The LSTM model uses adjusted leftover values as a starting point and figures out the hidden patterns to make better results. This method helps close the differences between the actual and estimated values, leading to more accurate parking space prediction.

Table 3 outlines the hyperparameters and settings for an LSTM model. The optimizer algorithm used during training is Adam. The learning rate is set to 0.001, determining the step size for adjusting the model’s weights. Sigmoid is used as the activation function for layer 1, while tanh is used for layer 2. These functions introduce non-linearity to the model’s computations. The dropout rate is set to 0.2, representing the proportion of neurons that dropped out during training to prevent overfitting. The model is trained for 300 epochs, which means the entire dataset is passed by the model 300 times during training. The batch size is set to 64, indicating the no. of samples employed in each training iteration before updating the model’s weights. The LSTM model consists of two layers, allowing it to capture complex patterns in the data. Each LSTM layer has 64 units or cells, enabling the model to learn and encode information. The recurrent dropout rate is set to 0.2, applying dropout specifically to the recurrent connections within the LSTM cells.

Figure 8 depicts the loss that occurs among the training and validation data during the implementation process using the LSTM model for parking space prediction using the Melbourne dataset.

Figure 9 depicts the loss that occurs among the training and validation data during the implementation process using the LSTM model for parking space prediction using the Harvard dataset. As shown in Figs. 8 and 9, applying the LSTM model to the training dataset decreased the loss function. Moreover, it can be observed that the loss continued to decrease as the number of epochs increased, eventually reaching a minimum value. This indicates the success of the LSTM model in capturing patterns and improving the accuracy of the predictions for parking space occupancy.

Figures 10 and 11 illustrate the comparison between the predicted values and the actual values for the nonlinear data points employing the LSTM model.

Despite the reduction in losses, it has been proved that the model could not accurately capture the actual values. Notably, a critical disparity between the actual and predicted values indicates the model’s limitations in accurately forecasting parking space occupancy. In the next section, the predicted results of Phase 1 are fed into BPNN to optimize the error rate.

BPNN model

In Phase 2, the predicted result obtained from Phase 1 is fed into Phase 2 to reduce the errors and make accurate predictions. This phase integrates ARIMA and LSTM models and is optimized through BPNN. This hybrid model allows for the combination of forecasted values from both models employing a nonlinear activation function, effectively handling both linear and nonlinear associations among the outputs.

Table 4 represents the tuning of hyperparameters used for BPNN. The BPNN optimizes the output and minimizes errors by iteratively backpropagating errors throughout the neural network and readjusting the weights. The inputs given to the BPNN are scaled using the “MinMaxScaler()” function to enhance model performance. The training process involves 5,000 epochs, enabling the model to reduce errors and refine predictions progressively. Early stopping is implemented, and validation loss is monitored to prevent the model from learning noise in the training data and to overcome the problem of overfitting. A learning rate of 0.9 regulates the magnitude of weight adjustments during training. The next section will explore the final results of implementing the proposed hybrid model.

Performance metrics

The performance metrics used to evaluate the proposed hybrid model are mean squared error (MSE), mean absolute error (MAE), and root mean square error (RMSE). Regarding time series forecasting, accuracy is typically assessed by contrasting the predicted values to the actual values using various error metrics. In time series forecasting, we generally refer to the error metrics mentioned earlier to calculate the model’s performance and the closeness of its predictions to the actual values. The main focus is on the model’s error rate, which is evaluated as a key parameter to measure the performance of the hybrid model. MSE, MAE and RMSE are employed explicitly as comparison and evaluation parameters in this research (Dia, Ahvar & Lee, 2022). While other evaluation parameters are available, the chosen parameters provide meaningful insights into the model’s performance in error estimation.

Mean squared error: MSE measures the average squared difference among the predicted & actual values, providing a measure of the model’s overall error. It is calculated as Eq. (8) (Nhu et al., 2020).

(8) $$MSE={\displaystyle \frac{1}{n}{\sum }_{i=1}^n(Y_i-{\hat{Y}}_i{)}^2}$$where n represents the count of data items, Yi denotes the actual values, ${\hat{\mathrm{Y}}}_{\mathrm{i}}$ stands the predicted values, and Σ denotes the summation over all data points.

Mean absolute error: When you talk about MAE, you’re basically looking at how close the predicted values are to the actual ones. A lower MAE means that the model’s predictions are more accurate. It is calculated as Eq. (9).

(9) $$MAE={\displaystyle \frac{1}{n}\sum |Y_i-{\hat{Y}}_i|}$$where n represents the count of data items, Yi stands for actual values, ${\hat{\mathrm{Y}}}_{\mathrm{i}}$ stands for predicted values, and Σ denotes the summation of all data points.

Root mean squared error: When predicting parking space availability, the root mean square error (RMSE) helps you understand how close the predicted values are to the actual ones. A lower RMSE means the model fits the data well, leading to more precise predictions of parking space availability. So, a minimum RMSE value signifies better accuracy and reliability in forecasting parking space availability. It is calculated using Eq. (10).

(10) $$RSME=\sqrt[2]{{\displaystyle \frac{1}{n}{\sum }_{i=1}^n{\left(Y_i-{\hat{Y}}_i\right)}^2}}$$where n represents the count of data items, Y_i stands for actual values, ${\hat{\mathrm{Y}}}_{\mathrm{i}}$ stands for predicted values.

The performance metrics outlined in this section comprehensively evaluate the hybrid model’s predictive accuracy and efficiency. These metrics form the basis for comparing the proposed model with existing approaches in the next section.

Result and discussion

This section presents the results obtained from Phase 2 and discusses them in detail. This section also presented the statistical significance of the proposed hybrid model. Finally, the proposed hybrid model is compared with existing models.

Figure 12 represents the final output of Phase 2, which is used to predict the available parking space using the proposed hybrid model. The proposed hybrid model successfully minimizes the errors generated in Phase 1 and optimizes the results by making accurate predictions of the available parking space using different datasets. Both datasets have different environmental conditions and different load factors. The results proved that the proposed hybrid model reduces errors and predicts accurately. This result demonstrates the superior performance of the proposed hybrid model in accurately predicting parking space occupancy.

Figure 13 represents the performance of the proposed hybrid model tested on two different datasets. It represents consistent scaling behavior for both datasets as the time taken increases prediction with the number of samples consistently maintained. As the data size increases, time slightly increases, but the proposed model can handle larger datasets efficiently.

Table 5 provides the results obtained during the implementation and validation of the proposed model in different training and testing distributions using the Melbourne dataset. Table 5 demonstrates the performance of the proposed hybrid model by minimizing the errors to improve parking prediction availability.

Table 6 provides the results obtained during the implementation and validation of the proposed hybrid model in different training and testing distributions using the Harvard dataset. Tables 5 and 6 demonstrate that the proposed hybrid model performs optimally in different scenarios by using different datasets in different environments and minimizing the error rates. Table 7 represents the comparative analysis of different models used for parking space prediction.

Table 7 highlights the performance metrics used for comparative analysis and the statistical analysis of the proposed hybrid model compared to the existing models, indicating improvements in the prediction accuracy. It presents a comparative analysis of different models used for parking space prediction, including the proposed hybrid model. Table 8 highlights the runtime of the different models, and the proposed hybrid model illustrates its efficiency for real-time applications in smart city development.

Figure 14 represents the comparative analysis of the performance of the proposed hybrid model with different models. It represents the results of Tables 7 and 8 that help to understand the performance and runtime of the proposed hybrid model.

Table 9 represents the performance comparison of the proposed hybrid model with existing models for parking space prediction. The table compares the results of the proposed hybrid model using different datasets with the existing models.

The proposed hybrid model (ARIMA + LSTM + BPNN) demonstrates superior performance with an MSE of 0.32, an MAE of 0.48, and an RMSE of 0.56 compared to the other models tested on the Melbourne dataset. It is again tested on the Harvard dataset and achieves performances with an MSE of 0.31, an MAE of 0.47, and an RMSE of 0.56. Figure 14 compares the proposed hybrid model’s performance with the existing model using performance metrics such as MSE, MAE, and RMSE. The proposed model shows the lowest error rate, indicating its improved performance. Figure 15 represents the performance comparison of the proposed model with existing models.

It is evident from Fig. 14 and Table 9 that the proposed hybrid model (ARIMA + LSTM + BPNN) outperformed by achieving a minimum error rate for predicting available parking space. By combining different forecasting techniques and leveraging the optimization capabilities of BPNN, the proposed model achieves more accurate predictions and outperforms the standalone models and other hybrid models. The proposed hybrid model is generalized and can be implemented on any parking dataset that compromises time series data, which contains linear and nonlinear data.