Evaluation model design of project construction safety level based on bidirectional recurrent neural network (BiRNN) and bidirectional long short-term memory (BiLSTM)

Construction of evaluation indicators

Enforcing a real-name system for workers and establishing authenticated access prevents unauthorized individuals from entering the site. This enhances internal safety management, ensuring project quality and safe production and providing valuable information support for industry policy development. Implementing a real-name system for pre-job training and assessment aids the owner in worker selection. Utilizing the AHP, 19 secondary causal factors are constructed based on the primary condition of operating personnel, the working conditions of organizational and management personnel, on-site management conditions, and the analysis of unsafe behaviors, as depicted in Table 1.

In this study, exclusive attention is directed towards target layer A and criterion layer B. The focus is solely on unraveling the relationship between the target layer (A) and the criterion layer (B), specifically solving for the criterion layer B in relation to the target layer A. Determining weights assigned to the criterion layer concerning the target layer is undertaken. The elements of the B-layer are delineated using a rating scale ranging from 1 to 4 and its reciprocal. Subsequently, a judgment matrix is formulated to portray the relative importance of each element. (1)${\displaystyle B=\left\{b_{ij}\mid i,j=1\sim n\right\}.}$

Assuming B be the single-ordered weights of the layers are w_k, k = 1 ∼ n, and satisfy w_k >0 and ${\sum }_{k=1}^n{w}_k=1$, according to the judgment matrix, Eqs. (2) and (3) should be satisfied.

(2)${\displaystyle b_{ij}=w_i/w_j,i,j=1\sim n}$ (3)${\displaystyle \left|\sum _{k=1}^n\left(b_{ik}{w}_k\right)-n{w}_i\right|=0.}$

In the resolution process, obtaining eigenvectors that precisely adhere to theoretical requirements poses challenges. Leveraging the principles of hierarchical analysis reveals that matrix consistency signifies the quality of eigenvectors (Pant et al., 2022). Consequently, the task of determining eigenvectors can be transformed into the endeavor of resolving the optimal consistency value. (4)${\displaystyle \begin{array}{cc}\hfill \text{min CI}=\hfill & \hfill \sum _{i=1}^n\left|\sum _{k=1}^n\left(b_{ik}{w}_k\right)-n{w}_i\right|/n\hfill \\ \hfill \mathrm{ s}.\mathrm{t}.\mathrm{ }\hfill & \hfill w_k>0,k=1\sim n\hfill \\ \hfill \hfill & \hfill \sum _{k=1}^n{w}_k=1.\hfill \end{array}}$

According to the definition of consistency ratio, when $\mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}<0.1$, the judgment matrix is considered to have satisfactory consistency.

BiRNN

The complete sequence of features undergoes encoding using a BiRNN network, which records the context vector for each output vector. The current state is forecasted by the forward RNN, capturing the before-mentioned relationships, while an inverse RNN predicts the character based on the context. Ultimately, the prediction results from both RNNs are linearly combined to generate the output of the final bidirectional RNN.

The forward network of the BiRNN iterates from time step t = 1 to T and the backward network iterates from time step t = T to 1. This involves simultaneous computation of the forward hidden sequence h_f and the backward hidden sequence h_b. The two sequences are summed to update the output sequence y, following the specifications in Eqs. (5), (6) and (7) (Hernandez-Matamoros, Fujita & Perez-Meana, 2020).

(5)${\displaystyle h_f=W_{x{h}_f}{x}_t+W_{h_f}{h}_{f\left(t-1\right)}+b_{h_f}}$ (6)${\displaystyle h_b=W_{x{h}_b}{x}_t+W_{h_b}{h}_{b\left(t-1\right)}+b_{h_b}}$

where W_xhf and W_xhb represent the weight matrices that transform the input x_t into the hidden state space for the forward and backward networks, respectively. W_hf and W_hb are weight matrices that connect the hidden states from the previous time step to the current hidden state. The bias vectors b_hf and b_hb are added to the hidden states to allow the model to learn an offset for better fitting the data.

At each time step t, the forward and backward hidden states h_ft, and h_bt are combined to produce the output sequence y_t as follows. (7)${\displaystyle y_t=W_{h_f}{h}_{ft}+W_{h_b}{h}_{bt}+b_y.}$

Wh_f and W_hb are weight matrices that project the forward and backward hidden states into the output space, and b_y isthe bias vector applied to the output. The final output y_t is thus a weighted combination of the forward and backward hidden states, adjusted by the bias.

BiLSTM

Various design structures exist for the state memory function of sequences. LSTM, employed in this study, represents one such design for RNN units. LSTM captures connections between sequence contexts, learns probabilistic relationships between contexts, and finds extensive applications in natural language processing (Su & Kuo, 2019). Its unique ability to regulate information flow is achieved by introducing a gate structure, automatically determining whether information should be retained or discarded. This design also addresses the challenges of gradient explosion and gradient vanishing encountered during the training of RNN networks.

BiLSTM, an extension of LSTM, enhances the model’s capacity to consider historical and future information when processing sequence data. While standard LSTM processes input sequence information from left to right, BI-LSTM concurrently processes left-to-right and right-to-left information (Wu et al., 2023). The computational procedure of BiLSTM is outlined in Eqs. (8) and (9):

(8)${\displaystyle \overrightarrow{h_t}=\overrightarrow{\text{LSTM}}\left(x_t,h_{t-1}\right)=\sigma \left(W\cdot \left[h_{t-1},x_t\right]+b\right)}$ (9)${\displaystyle \stackrel{⃖}{h_t}=\stackrel{⃖}{\text{LSTM}}\left(x_t,\stackrel{⃖}{h_{t+1}}\right)=\sigma \left(\mathrm{W}\cdot \left[\stackrel{⃖}{h_{t+1}},x_t\right]+\mathrm{b}\right)}$

where h_t is the time step t of the hidden state, $\overrightarrow{h_t}$ is the output of forward LSTM, and $\stackrel{⃖}{h_t}$ is the output of the inverse LSTM. The two merged outputs in the sequence process are shown in Fig. 1:

Model construction

This study employs a sophisticated model architecture integrating BiLSTM and BiRNN layers to enhance prediction accuracy. The process begins with data preparation, where all input features and labels are normalized based on the training dataset. This normalization is crucial for reducing the influence of individual sample data and ensuring consistent scaling across the model, which ultimately contributes to more reliable predictions.

Following data normalization, the processed data is input into the RNN layer. The RNN layer’s output undergoes regularization through a Dropout layer to address the overfitting issue. This layer randomly drops units during training, which helps prevent the model from becoming overly dependent on specific nodes and improves generalization. The regularized output is then forwarded to the LSTM layer, which similarly applies Dropout to its output to maintain regularization throughout the network.

Subsequently, the output from the Dropout layer is passed to the BiLSTM layer. The BiLSTM layer processes the data in both forward and backward directions, more comprehensively capturing temporal dependencies. The data then progresses to the BiRNN layer, further enhancing the model’s ability to learn from sequential information by processing data in both directions.

The final stage consolidates the predictions through a fully nonlinear superposition, integrating the outputs from the BiLSTM and BiRNN layers. This consolidated output is then processed through a fully connected Dense layer, which generates the final prediction results. After training, the model’s parameters are saved, and test data is normalized and input into the model. The predictions obtained from the model are subsequently inverse-normalized to produce the final outcomes.

This multi-layered approach thoroughly integrates recurrent layers, enhancing the model’s predictive performance. Carefully structuring these layers and applying regularization techniques collectively improve the model’s ability to generalize and deliver accurate predictions.

The process of constructing the model is depicted in Fig. 2.

Data sources

This study gathers real-name list information for over 1,000 construction and organization management personnel from over 40 enterprises (https://zenodo.org/records/7930450, doi: 10.5281/zenodo.7930450). To enhance the effectiveness of the real-name system, construction enterprises are encouraged to adopt information-based management approaches. This comprehensive approach aims to improve efficiency across all facets of the real-name system, encompassing contract signing, work attendance, payroll management, safety protocols, and skills training.

Experimental setup

The algorithm simulation is conducted within the computer hardware and software environment, as detailed in Table 2.

The algorithm operation parameters are set as shown in Table 3.

Training results

The model in this study conducts crisis prediction on the training set, yielding the confusion matrix illustrated in Table 4.

Analyzing the prediction outcomes of the training samples reveals two misclassifications out of 31 positive samples and three discrimination errors in 32 negative samples, totaling five classification errors across 63 samples. Consequently, the deduced accuracy of the model in this study is 95.24%, and the Recall is 96.55%. Figure 3 depicts the classification accuracy curve of the BiRNN-BiLSTM model at 150 iterations on the training set.

The model in this paper demonstrates the robustness and generalization capabilities by converging and stabilizing after 18 iterations. The convergence at an early stage signifies efficient learning and optimal performance on the given training set. This outcome highlights the model’s adaptability to different data instances, showcasing its ability to generalize effectively beyond the training data. The convergence and stability at a relatively low iteration count also suggest that the model successfully captures the underlying patterns and relationships within the dataset, providing confidence in its reliability for real-world applications and predictive tasks.

Model comparison

Utilizing prediction accuracy as the evaluation metric, the prediction results of different models are depicted in Fig. 4. Notably, the traditional LSTM network exhibits a 4.59% improvement in accuracy compared to RNN, showcasing the enhanced performance of LSTM networks in analyzing and predicting time series. Additionally, introducing BiLSTM cells and the Dropout mechanism further improves the accuracy of the RNN network in model prediction. Specifically, the prediction accuracy of the BiLSTM model increases by 1.60%, the introduction of Dropout improves accuracy by 2.72%, and incorporating both mechanisms simultaneously enhances the prediction accuracy by 4.61%. This underscores the positive impact of these enhancements on the model’s predictive capabilities.

Furthermore, the Attention-based Time-Incremental Network (ATIN), RNN-LSTM, BiGRU, and TCN are included to compare with the model proposed in this study. The results are illustrated in Figs. 5 and 6.

BiRNN-BiLSTM (Ours) demonstrates superior performance in terms of mean squared error (MSE) and root mean squared error (RMSE) with values of 0.48 and 0.69, respectively. This signifies a significant advantage in minimizing squared errors and efficiently capturing the difference between predicted and actual values. The ATIN model also displays favorable results with relatively low MSE (0.51) and RMSE (0.71), indicating good performance regarding both squared and relative magnitude of the error. Conversely, the BiGRU model exhibits higher MSE (0.78) and RMSE (0.88), suggesting comparatively poorer performance in error minimization.

Our focus centers on the model’s capacity to capture the absolute and relative percentages of prediction error accurately. In this context, BiRNN-BiLSTM (Ours) once again excels, demonstrating the lowest mean absolute error (MAE) at 0.54 and the lowest mean absolute percentage error (MAPE) at 3.36%. The ATIN model also performs well, yielding a low MAE of 0.56 and MAPE of 3.58%. Conversely, the BiGRU model exhibits a higher MAE of 0.68 and a higher MAPE of 4.14%, indicating relatively weaker performance.

The outstanding performance of BiRNN-BiLSTM (Ours) can be attributed to the advantageous features embedded in its model architecture. The combination of BiRNN and BiLSTM enables the model to capture long-term dependencies more effectively in time-series data. This bidirectional recursive structure enhances the model’s understanding of contextual information within sequences during training, improving its ability to learn complex patterns in time-series data. Additionally, the bidirectional structure facilitates a more comprehensive utilization of historical information, rendering the model adaptive and robust.

Case analysis

The study focuses on three construction projects—Project A, Project B, and Project C—situated in the power industry sector in the Qingpu District of Shanghai. These projects were assessed based on their safety performance, which was categorized into five distinct levels: V = [V1, V2, V3, V4, V5] = [(0,30), (30,60), (60,70), (70,80), (90,100)]. These levels correspond to “very poor”, “poor”, “average”, “good”, and “excellent”, respectively.

The data collection process involved several systematic steps to ensure accuracy and comprehensiveness. Initially, safety performance data were systematically gathered from each of the three construction projects. This data collection included direct observations, safety reports, and incident records. Observations were conducted on-site to capture real-time safety practices and compliance with safety regulations. Safety reports provided detailed accounts of safety performance over time, while incident records offered insights into any safety breaches or accidents that occurred. The results are depicted in Fig. 7.

Figure 7 shows that Project C outperforms Project A and Project B in overall construction safety evaluation. Project B lags, particularly regarding unsafe behaviors, while Project A exhibits a lower evaluation value in on-site management, creating a discernible gap with Project C. The variations in scale and composition among these projects contribute to differences in their construction safety evaluation levels. These distinctions underscore the efficacy of this study’s index system and model as an effective measure for comprehensive construction safety evaluation. Despite these differences, the three selected projects consistently receive high scores in evaluating operating personnel, attributed to implementing the real-name system and ensuring information security management and unity among engineering personnel. The analysis primarily focuses on the basic situation of the operating personnel, as illustrated in Fig. 8.

Firstly, examining the data on operational staff performance reveals that Project C outshines the others with a score of 90.5, significantly surpassing Project A (85.4) and Project B (88.3). Secondly, regarding site management, Project C demonstrates superior operational and management efficiency with a score of 80.4. In contrast, Project A (65.6) and Project B (70.4) lag, suggesting that Project C exhibits higher organizational coordination and site management. At the organizational management personnel level, Project C again excels with a score of 92.5, markedly higher than Project A (79) and Project B (81.4), indicating superior leadership and organizational coordination capabilities. However, the data on unsafe behaviors reveal that Project C has a relatively high score (77.8), potentially signifying a higher frequency of unsafe behaviors. In contrast, Project A (60.4) and Project B (54.6) score lower, indicating better safety practices. This aspect requires attention, prompting a more in-depth safety assessment and implementing improvement measures.

Figure 9 presents the predicted construction safety evaluation scores of Projects A, B, and C. The scores show that Project C’s construction safety evaluation score gradually improves as the project progresses. In contrast, Projects A and B experienced a decline, with Project A showing a noticeable decrease, indicating potential safety hazards in the later stages of the project.

Considering the results from Figs. 7 and 8, it is evident that Project C excels in organizational management, which primarily contributes to its higher construction safety evaluation score. The outcomes of this assessment demonstrate the model’s accuracy in analyzing the dynamic safety evaluation scores of various project sites. This insight is valuable for managers, providing guidance to facilitate timely rectification and effective management of project sites.

Conclusions

In this study, we quantify construction safety capability using the AHP method, establish an evaluation system across multiple abstract dimensions, and conduct construction safety evaluations by combining BiRNN and BiLSTM models. Validation results demonstrate that the proposed model effectively captures sudden and anomalous sequence changes through its bidirectional structure. It comprehensively captures the temporal characteristics of the construction process by considering historical and future information. The use of BiRNN and BiLSTM for construction safety evaluation enhances the handling and utilization of complex datasets, improving the robustness and generalization ability of the model across different construction scenarios. This reduces the likelihood of accidents and enhances emergency response efficiency, ensuring worker safety and minimizing potential losses.

Limitations

Both the BiRNN and BiLSTM models face challenges with gradient vanishing and exploding, mainly when dealing with long sequences. Although BiLSTM incorporates gating mechanisms to address these issues, they can still affect the model’s stability and predictive accuracy in real-world construction safety evaluations. This limitation hinders the effective processing of extensive time series data, potentially impacting the reliability of safety assessments.

Future research could focus on developing advanced gradient management techniques or incorporating alternative architectures that further mitigate gradient-related issues in BiRNN and BiLSTM models. Exploring variations such as attention mechanisms or transformer models might offer improved performance in handling long sequences. Moreover, research into optimizing the computational efficiency of BiLSTM models is needed to address the high resource requirements. Techniques such as model pruning, quantization, or hybrid architectures that balance accuracy and computational demands could be investigated to enhance the feasibility of real-time safety monitoring systems.

Addressing the challenge of data imbalance could involve integrating synthetic data generation methods, anomaly detection techniques, or cost-sensitive learning approaches to improve the model’s ability to detect rare safety events. Future studies should explore strategies to better represent and learn from sparse safety incident data.