Spatio-temporal causal graph attention network for traffic flow prediction in intelligent transportation systems

Graph convolutional network

Graph convolutional networks (GCN) (Welling & Kipf, 2016) have gained significant attention in recent years due to their ability to process graph-structured data for various tasks, such as graph classification, node classification, and link prediction. GCN can be classified into two main categories based on their clustering method: Spectral-based and Spatial-based (Wu et al., 2020). Spectral-based techniques in graph signal processing use filters to define convolutions on graphs, and they extend these convolutions to the spectral domain by identifying the corresponding Fourier bases. This allows for efficient computation of convolutions on graphs and enables the use of tools from spectral graph theory to analyze graph signals. Some primary examples of spectral-based methods include Chebyshev Spectral CNN (ChebNet) (Monti et al., 2017) and adaptive graph convolution network (AGCN) (Li et al., 2018b). Spatial-based approaches define graph convolutions by aggregating feature information from the neighborhood and using the spatial relationship of the nodes. Some of the most prominent spatial-based methods include GAT and gated attention network (GAAN) (Zhang et al., 2018).

Time-dependent modeling

In the past, traffic forecasting projects relied on multivariate time series analysis methods such as history average (HA) (Duan et al., 2016), support vector regressor (SVR) (Hao, Leixiao & Hui, 2020), and vector autoregression (VAR) (Dissanayake et al., 2021). However, these techniques assume ideal smoothness and do not account for the nonlinear correlation in traffic data. In recent years, deep learning has gained popularity due to its ability to create sophisticated models and learn autonomously. Recurrent neural network (RNN)-based models have been widely used to capture temporal dependencies, but their long-loop structure is time-consuming and can lead to the gradient disappearance and explosion phenomenon. To overcome this issue, temporal convolutional network (TCN) (Bai, Kolter & Koltun, 2018; Zhang, Liu & Zheng, 2019) has been introduced to process time series in parallel and capture more information through a larger field of perception. Additionally, researchers have developed variant models based on Transform (Vaswani et al., 2017) that perform well on long time series prediction tasks (Xu et al., 2020; Zhou et al., 2021; Wu et al., 2021). These approaches solely take into account the temporal patterns of traffic flow data, disregarding the spatial interdependencies among the road nodes.

Spatio-temporal dependence modeling

Several studies have utilized regular two-dimensional grids to represent traffic networks, and employed CNN to capture the spatial correlations in the traffic data. These approaches then use RNN or additional CNN to model the temporal dependencies in the traffic time series data (Li et al., 2020; Zheng et al., 2020; Ma, Song & Li, 2020). However, CNN’s are not always applicable in traffic road networks of non-Euclidean nature. To solve the problem, many researchers are turning to GCN-based traffic flow models. For example, DCRNN (Li et al., 2017) models spatial dependencies by wandering in both directions on the traffic road topology graph and then uses GRU to capture temporal correlations. ASTGCN (Guo et al., 2019) models the Spatial-temporal dependence of traffic data with spatial attention and temporal attention. STSGCN (Song et al., 2020) captures the local spatio-temporal correlation by combining spatial and temporal blocks through a local spatio-temporal synchronized graph convolution module. STFGNN (Li & Zhu, 2021) learns both local and global spatio-temporal dependencies by processing data-driven spatial and temporal graphs at different moments in parallel. STGODE (Fang et al., 2021) draws on the Dynamic Time Warping (DTW) (Li, Liang & Wang, 2018a) used by STFGNN to generate semantic adjacency matrices for traffic road topology maps to capture deeper spatio-temporal correlations. In recent works, Z-GCNETs (Chen, Segovia & Gel, 2021) introduce the concept of zigzag persistence in the traffic network diagram structure and integrate it into GCNs to enhance the stability of the model. TAMP-S2GCNets (Chen et al., 2021) proposes the Euler Poincare surface for learning the topological structure of traffic network through topological features under the multi-parameter persistence of traffic flow data, and constructs a hypergraph convolution network to model the spatio-temporal dependence of traffic flow. STG-NCDE (Choi et al., 2022) designs two neural control differential equations dealing with temporal and spatial dependencies, respectively, and integrates both to capture spatio-temporal dependencies simultaneously. DSTAGNN (Lan et al., 2022) proposes a spatio-temporal perception distance to dynamically learn Spatial-temporal dependence.

Compared to the spatio-temporal prediction models mentioned above, our proposed STCGAT utilizes a node adaptive learning graph attention network to model the spatial dependency without relying on the topological structure information of the traffic network. Additionally, a bidirectional spatio-temporal component is introduced, which recursively integrates the causal spatio-temporal correlations at different moments within the traffic network. Simultaneously, it captures the global spatio-temporal dependencies present in the overall time series.

Problem formulation

Traffic flow prediction uses historical traffic flow on the road to predict traffic condition information in the future period.

Definition 1: The graph $G=(V,E)$ is utilized in this study to represent the topological information of the traffic roads. Where $V=\{v_1,v_2,\cdots ,v_N\}$ is the set of all nodes of graph G, N is the number of all road nodes, and E denotes the set of connected edges of all nodes of graph G.

Definition 2: The historical traffic flow information of a time length T is represented in this study by a feature matrix $X=\{X_1,X_2,\cdots ,X_N\}\in {\mathbb{R}}^{N\times T\times F}$. Where F denotes the feature dimension, $X_t=\{{\overrightarrow{x}}_{t:1},{\overrightarrow{x}}_{t:2},\cdots ,{\overrightarrow{x}}_{t:N}\}\in {\mathbb{R}}^{N\times F}$ denotes the set of traffic information of all road nodes at any $t$ time, and ${\overrightarrow{x}}_{t:i}\in {\mathbb{R}}^F$ then denotes all the feature vectors of node $v_i$.

Traffic flow prediction aims to use the G and X to predict the traffic flow $Y^{\mathrm{\prime }}=[X_{t+1}^{\mathrm{\prime }},X_{t+2}^{\mathrm{\prime }},\cdots ,X_{t+T}^{\mathrm{\prime }}]$ in the following T moments by learning a function $f(\cdot )$.

(1) $$Y^{\mathrm{\prime }}=f(G;(X_{t-T},X_{t-(T-1)},\cdots ,X_t))$$

Spatial dependency modeling

In the spatial dimension, the traffic condition data of different road nodes are strongly and dynamically interconnected. However, traditional graph neural networks rely on predefined adjacency matrices to perform graph convolution operations based on factors like connectivity or distance between graph nodes. Although these matrices intuitively represent the positional relationship between nodes, they cannot capture the dynamic spatial correlation between road nodes at different moments. To address this issue, as shown in Fig. 1B, a node adaptive learning mechanism is adopted in this study. This mechanism effectively learns the dynamic correlation information among road nodes at different moments to generate the traffic subgraph $G_{ad}$ specific to the corresponding moment. As shown in the following equation, this mechanism generates the adjacency matrix $\widetilde{A^t}\in {\mathbb{R}}^{N\times N}$ for any moment $t$.

(2) $${\tilde{A}}^t=softmax(ReLU(E_{At}\cdot E_{At}^T))$$where $E_{At}\in {\mathbb{R}}^{N\times d}$ is the embedding dictionary encoding each traffic road node, $d$ is the embedding dimension, $E_{At}^T$ is the transpose matrix of $E_{At}$. $ReLU$ is the nonlinear activation function, and $softmax$ is the normalization function.

To capture the adaptive and dynamic spatial dependencies among nodes in the spatial dimension, a NAL-GAT model is proposed in this study. This model integrates a node-adaptive learning mechanism with GAT. Specifically, at any moment $t$, NAL-GAT computes attention coefficients of neighboring nodes for the node correlation information generated by the node-adaptive learning mechanism. This enables the model to extract the spatial features of traffic roads and aggregate spatial dependencies among the nodes on the graph at moment $t$. As shown in the equation below, the attention coefficient $e_{ij}^t$ between node $v_i$ and its neighbor node $v_j$ is computed in a vertex-wise manner.

(3) $$e_{ij}^t=a(W{\overrightarrow{x}}_{t:i},W{\overrightarrow{x}}_{t:j})$$where $a$ is the computational function of the attention mechanism, and $W\in {\mathbb{R}}^{F\times F^{\mathrm{\prime }}}$ is the graph’s weight matrix of all nodes. The attention coefficient ${\alpha }_{ij}^t$ of the graph’s attention layer is then generated by normalizing the attention coefficients of node $v_i$’s neighbors.

(4) $$\begin{array}{rl}& {\alpha }_{ij}^t=softma{x}_j(e_{ij}^t)\\ & =\frac{exp(e_{ij}^t)}{{\sum }_{k\in {\tilde{A}}_i^t}exp(e_{ik}^t)}\end{array}$$where ${\tilde{A}}_i^t$ denotes all the neighbor nodes of $v_i$.

In addition, it is noted that in the GAT, all nodes share the same parameter space $W\in {\mathbb{R}}^{F\times F^{\mathrm{\prime }}}$. However, this can result in a large graph W when there are more nodes, making the model difficult to optimize. To address this problem, a shared weight pool $W_p\in {\mathbb{R}}^{d\times F\times F^{\mathrm{\prime }}}$ is constructed, which can get the weight matrix $W^{\mathrm{\prime }}=E_{At}\cdot W_p\in {\mathbb{R}}^{F\times F^{\mathrm{\prime }}}$ of each node according to the node’s embedding dictionary $E_{At}$.

(5) $$\begin{array}{rl}& e_{ij}^t=LeakReLu({\overrightarrow{a}}^T[E_{At}\cdot W_p{\overrightarrow{x}}_{t:i}\parallel E_{At}\cdot W_p{\overrightarrow{x}}_{t:i}])\\ & e_{ik}^t=LeakReLu({\overrightarrow{a}}^T[E_{At}\cdot W_p{\overrightarrow{x}}_{t:i}\parallel E_{At}\cdot W_p{\overrightarrow{x}}_{t:k}])\\ & {\alpha }_{ij}^t=\frac{exp(e_{ij}^t)}{{\sum }_{k\in {\tilde{A}}_i^t}exp(e_{ik}^t)}\end{array}$$where $\overrightarrow{a}\in {\mathbb{R}}^{2F^{\mathrm{\prime }}}$ is the weight matrix, $\parallel$ denotes the connection operation, and $LeakReLu$ is the nonlinear activation function.

To capture deeper feature information, this article further uses the multi-head attention to model spatial dependence. As shown in the following equation, Q sets of mutually independent attention mechanisms are invoked.

(6) $${\overrightarrow{x}}_{t:i}^{\mathrm{\prime }}={\parallel }_{q=1}^{Q}LeakReLu({\sum }_{k\in {\tilde{A}}_i^t}{\alpha }_{ik}^{t,q}(E_{At}^q\cdot W_p^q){\overrightarrow{x}}_{t:k})$$where ${\alpha }_{ik}^{t,q}$ is the weight coefficient computed by the attention mechanism of the qth group at time $t$, $E_{At}^q\cdot W_p^q$ is the weight matrix of the corresponding group, and ${\overrightarrow{x_i}}^{\mathrm{\prime }}\in {\mathbb{R}}^{Q{F}^{\mathrm{\prime }}}$ is the new feature representation obtained by passing node $v_i$ through the attention layer of the multi-headed graph.

When employing multi-headed attention, it is important to avoid outputting too many features in the network’s last layer. To address this, a separate self-attentive mechanism is utilized to limit the output feature length of each node. Specifically, a new weight pool $W_p^{\mathrm{\prime }}\in {\mathbb{R}}^{d\times Q{F}^{\mathrm{\prime }}\times F^{\mathrm{\prime }\mathrm{\prime }}}$ is employed to map the node’s output dimension from ${\mathbb{R}}^{Q{F}^{\mathrm{\prime }}}$ to ${\mathbb{R}}^{F^{\mathrm{\prime }\mathrm{\prime }}}$ and obtain the final output result ${\overrightarrow{x}}_{t:i}^{\mathrm{\prime }\mathrm{\prime }}\in {\mathbb{R}}^{F^{\mathrm{\prime }\mathrm{\prime }}}$.

(7) $${\overrightarrow{x}}_{t:i}^{\mathrm{\prime }\mathrm{\prime }}=LeakReLu({\sum }_{k\in {\tilde{A}}_i^t}{\alpha }_{ik}^{t\mathrm{\prime }}(E_{At}^{\mathrm{\prime }}\cdot W_p^{\mathrm{\prime }}){\overrightarrow{x}}_{t:k}^{\mathrm{\prime }})$$

After completing the graph attention layer operation for all nodes in the graph, the output features can be obtained as $X_t^{\mathrm{\prime }\mathrm{\prime }}=\{{\overrightarrow{x}}_{t:1}^{\mathrm{\prime }\mathrm{\prime }},{\overrightarrow{x}}_{t:2}^{\mathrm{\prime }\mathrm{\prime }},\cdots ,{\overrightarrow{x}}_{t:N}^{\mathrm{\prime }\mathrm{\prime }}\}\in {\mathbb{R}}^{N\times F^{\mathrm{\prime }\mathrm{\prime }}}$. For the convenience of presentation, this process can be expressed in the following equation.

(8) $$X_t^{\mathrm{\prime }\mathrm{\prime }}=\sigma ({\tilde{A}}^t{X}_t(E_{At}\cdot W_p))$$where $\sigma (\cdot )$ is the computation function for the graph attention layer.

Local causal spatial-temporal dependency modeling

There is a correlation between traffic conditions in the time dimension at various times. As shown in Fig. 1C, the gating unit of GRU is replaced with NAL-GAT in order to model Spatio-temporal correlations. Specifically, the spatially dependent time series data $X_t^{\mathrm{\prime }\mathrm{\prime }}$ at any moment $t$ is used as the input data of the GRU.

(9) $$\begin{array}{rl}& z_t=\sigma ({\tilde{A}}^t[X_t,{\overrightarrow{h}}_{t-1}](E_{At}^z\cdot W_p^z))\\ & r_t=\sigma ({\tilde{A}}^t[X_t,{\overrightarrow{h}}_{t-1}](E_{At}^r\cdot W_p^r))\\ & \widetilde{h_t}=tanh({\tilde{A}}^t[X_t,r_t\odot {\overrightarrow{h}}_{t-1}](E_{At}^{\tilde{h}t}\cdot W_p^{\tilde{h}t})\\ & \overrightarrow{h_t}=z_t\odot h_{t-1}+(1-z_t)\odot \widetilde{h_t}\end{array}$$where ${\overrightarrow{h}}_{t-1}$ is the output at the previous moment, ${\tilde{\mathrm{h}}}_t$ is the candidate hidden layer state, $[\cdot ]$ is the concat operation in the feature dimension, $\overrightarrow{h_t}\in {\mathbb{R}}^{N\times F^{\mathrm{\prime }\mathrm{\prime }}}$ is the output at the moment $t$, and $\odot$ is the multiplication by elements.

It is important to note that as the input time length rises, so does the network depth of the model. However, the deep network may lead to issues such as gradient disappearance and overfitting in the model. Therefore, the residual module is used to connect the layers of the network in order to enhance the model’s capacity for long-term capture.

(10) $$\begin{array}{rl}& z_t=\sigma ({\tilde{A}}^t[X_t,{\overrightarrow{h}}_{t-1}^{\mathrm{\prime }}](E_{At}^z\cdot W_p^z))\\ & r_t=\sigma ({\tilde{A}}^t[X_t,{\overrightarrow{h}}_{t-1}^{\mathrm{\prime }}](E_{At}^r\cdot W_p^r))\\ & \widetilde{h_t}=tanh({\tilde{A}}^t[X_t,r_t\odot {\overrightarrow{h}}_{t-1}^{\mathrm{\prime }}](E_{At}^{\tilde{h}t}\cdot W_p^{\tilde{h}t})\\ & \overrightarrow{h_t}=z_t\odot h_{t-1}^{\mathrm{\prime }}+(1-z_t)\odot \widetilde{h_t}\\ & {\overrightarrow{h}}_t^{\mathrm{\prime }}=\epsilon ({\omega }_1\otimes X_t+{\omega }_2\otimes \overrightarrow{h_t})\end{array}$$where ${\omega }_1$ and ${\omega }_2$ are both one-dimensional convolution kernels, $\epsilon$ is the nonlinear activation function, $\otimes$ denotes the convolution operation, and ${\overrightarrow{h}}_t^{\mathrm{\prime }}\in {\mathbb{R}}^{N\times F^{\mathrm{\prime }\mathrm{\prime }}}$ is the output of residual concatenation. Until the completion of the above operations at the Tth time step, the sequence data containing the spatio-temporal dependence can be obtained as ${\overrightarrow{H}}^{\mathrm{\prime }}\in {\mathbb{R}}^{N\times T\times F^{\mathrm{\prime }\mathrm{\prime }}}$.

In addition, traffic data are not always sequentially correlated, and there are complex causal correlations between traffic events. Therefore, bidirectional GRU is utilized to capture the local causal and temporal relationships. The reverse operation is similar to the above operation, and the output results are finally stitched to obtain the output $H\in {\mathbb{R}}^{N\times T\times 2F^{\mathrm{\prime }\mathrm{\prime }}}$.

(11) $$H={\overrightarrow{H}}^{\mathrm{\prime }}\parallel \overleftarrow{H^{\mathrm{\prime }}}$$

Global spatial-temporal dependency modeling

From the above procedure, it is evident that GRU is processed by progressively unfolding along the timeline, which causes the output at the present time to depend on the state at the previous time and so lacks the capacity to capture global temporal dependence. A parallel TCN is deployed along the time axis to enhance the performance of extracting long-term Spatio-temporal dependencies. As shown in the following equation, $H^{\mathrm{\prime }}\in {\mathbb{R}}^{N\times (T\ast 2F^{\mathrm{\prime }\mathrm{\prime }})}$ is utilized as the TCN’s input data. In the time series convolution process, the time series data $H_{i:}^{\mathrm{\prime }}\in {\mathbb{R}}^{(T\ast F^{\mathrm{\prime }\mathrm{\prime }})}$ of any node $v_i$ and a filter $f:\{0,\cdots ,l-1\}\overrightarrow{}R$ are first extended for the elements $s$.

(12) $$F(s)=(H_{i:}^{\mathrm{\prime }}{\ast }_{d}f)(s)={\sum }_{i=0}^{l-1}f(i)\cdot H_{i:}^{\mathrm{\prime }}{(}_{s-d\cdot i})$$where $l$ is the filter size, and $s-d\cdot i$ denotes the direction of the timeline past, $d$ is the dilation factor. When $d=0$, the dilation convolution becomes a regular convolution. In addition, as shown in Fig. 1D, the TCN needs to perform a series of transformations such as Weight Norm and Dropout and use the residual join to obtain the output $o\in {\mathbb{R}}^{N\times (T\ast 2F^{\mathrm{\prime }\mathrm{\prime }})}$ as in the equation below.

(13) $$o=Activation(x+F(x))$$

The prediction layer

Afterward, a two-layer fully connected neural network is utilized to perform a linear transformation specifically tailored to the dimensions of the output sequence.

(14) $$Y^{\mathrm{\prime }}=W_2\cdot \phi (W_1\cdot o+b_1)+b_2$$where $b_1$ and $b_2$ are the bias values, $W_1\in {\mathbb{R}}^{F^{\mathrm{\prime }\mathrm{\prime }\mathrm{\prime }}\times (T\ast 2F^{\mathrm{\prime }\mathrm{\prime }})}$ and $W_2\in {\mathbb{R}}^{(T\times F)\times F^{\mathrm{\prime }\mathrm{\prime }\mathrm{\prime }}}$ are the weight matrices, and $Y^{\mathrm{\prime }}\in {\mathbb{R}}^{N\times T\times F}$ is the prediction result.

During model training, the optimization is performed using the $L1$ loss function and the Adam optimizer in order to minimize the error between the predicted $Y^{\mathrm{\prime }}$ and the labeled values $Y=[X_{t+1},X_{t+2},\cdots ,X_{t+T}]$. In summary, the specific calculation process of STCGAT is shown in Algorithm ??.

(15) $$loss=\frac{1}{T}\sum _{i=1}^T|Y_{t+i}-Y_{t+i}^{\mathrm{\prime }}|$$

Datasets

Extensive experiments were carried out on four publicly available transportation datasets namely PeMS03, PeMS04, PeMS07, and PeMS08 (Guo et al., 2021). These datasets were obtained from Caltrans’ Performance Measurement System (PeMS), which has deployed over 39,000 traffic detectors on California freeways to collect real-time traffic data and aggregates the collected data every 5 min. Details about the datasets are provided below.

• PeMS03: The dataset contains 358 sensors connected by 547 edges, recording 26,208-time steps of traffic data.

• PeMS04: The dataset contains 307 sensors connected by 340 edges, recording 16,992-time steps of traffic data.

• PeMS07: The dataset contains 883 sensors connected by 866 edges, recording 28,224-time steps of traffic data.

• PeMS08: The dataset contains 170 sensors connected by 295 edges, recording 17,856-time steps of traffic data.

Baseline methods

To evaluate the prediction performance of the proposed model, a series of baseline models are selected for comparative experiments in this study. These baseline models encompass the mainstream traffic flow prediction models outlined in this article, namely HA (Duan et al., 2016), LSTM (Karimzadeh et al., 2021), DCRNN (Li et al., 2017), ASTGCN (Guo et al., 2019), STSGCN (Song et al., 2020), STFGNN (Li & Zhu, 2021), STGODE (Fang et al., 2021), Z-GCNETs (Chen, Segovia & Gel, 2021), TAMP-S2GCNets (Chen et al., 2021), STG-NCDE (Choi et al., 2022) and DSTAGNN (Lan et al., 2022). Furthermore, a traffic flow prediction network called AGCRN (Bai et al., 2020), which incorporates an adaptive graph convolution module, is also chosen as a new baseline model.

Experimental settings

The used dataset is divided into 60% training set, 20% validation set, and the remaining 20% test set with Z-score standardization. The partitioned dataset is then processed through a sliding window of length $2T$, where the first T time lengths of serial data are used as historical data, and the last T time lengths of data are used as labeled values. In our experiments, T was set to 12.

The STCGAT model was implemented using the PyTorch deep learning framework, with hyperparameters set as follows: the node embedding feature dimension was set to 10, the hidden layer size was 64, the model used three multi-head attention mechanisms and a convolutional kernel size of two. During the training process, a batch size of 64 was utilized, along with an Adam optimizer and a learning rate of 0.001. The model was trained for a maximum of 300 epochs. The experiments were carried out using a Linux server equipped with an NVIDIA GeForce 2080Ti GPU.

In order to evaluate the predictive accuracy of both the proposed model and other baseline models, the following performance metrics were employed as evaluation criteria.

• Mean absolute error (MAE): $$\mathrm{M}\mathrm{A}\mathrm{E}=\frac{1}{L}\stackrel{L}{\sum _{i=1}}|Y_i-Y_i^{\mathrm{\prime }}|$$

• Root mean squared error (RMSE): $$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{\frac{1}{L}{\sum }_{i=1}^L{(Y_i-Y_i^{\mathrm{\prime }})}^2}$$

• Mean absolute percentage error (MAPE): $$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}=\frac{100\mathrm{\%}}{L}\stackrel{L}{\sum _{i=1}}|\frac{Y_i-Y_i^{\mathrm{\prime }}}{Y_i}|$$

Experiment analysis

On PeMS03, PeMS04, PeMS07, and PeMS08, our model was compared with the twelve representative baseline approaches described previously. Table 1 shows the prediction performance results of STCGAT and other baseline models within 1 h (12 prediction steps). It can be observed that: (1) In traffic flow prediction tasks, the effectiveness of GCN in explicitly modeling spatial correlation is demonstrated by the superior performance of the GCN-based approach compared to the LSTM-based time series forecasting methods. This highlights the significance of incorporating spatial correlation into the prediction models; (2) The performance metrics of our improved GAT-based method on each dataset are almost better than other advanced baseline models, achieving significant results; (3) as shown in Table 2, STCGAT is compared with several advanced spatio-temporal prediction models for the purpose of parameter count and training cost comparison. The results show that STCGAT achieves the best prediction performance with relatively reasonable control of the overall parameter number and training cost compared with the current advanced spatio-temporal prediction models; (4) as shown in Fig. 2, the prediction results are visualized by continuously saving 24 prediction snapshots of STCGAT and other advanced baseline models on the test set. It can be observed that STCGAT demonstrates faster and more accurate recovery of prediction accuracy, particularly in the presence of missing data. The corresponding changes during peak traffic periods are also more accurate; (5) as shown in Fig. 3, which demonstrates the comparison of the prediction power of STCGAT with other spatio-temporal prediction models on different horizon, it can be observed that the performance curve of STCGAT exhibits relatively small oscillations on each dataset. This indicates that the proposed method possesses stable short- and long-term spatio-temporal prediction capabilities.

Ablation study

Four STCGAT-based model variants were created in this study, and STCGAT was contrasted with these variants to understand the influence of STCGAT’s numerous modules on its predictive performance. The differences between these four variants of the model are described below.

• 1. w/o reverse GRU: The model models the time dependence using only positive GRU.

• 2. w/o ResNet: The model eliminates the STCGAT residual connection module.

• 3. w/o node embedding: The model uses a predefined adjacency matrix to replace the self-generated adjacency matrix in STCGAT.

• 4. w/o TCN: The model removes the TCN from STCGAT.

Table 3 presents the experimental results conducted with the aforementioned variants and STCGAT. Combined with the histogram of evaluation metrics for each variant model at different time steps in Fig. 4, the following observations can be made: (1) Each evaluation metric of w/o TCN is at the maximum value and the performance metrics are similar for short term (15 mins) and long term (60 mins). It illustrates the necessity of using temporal convolutional networks to extract the global temporal correlation of traffic flow; (2) the performance metrics of w/o node embedding decrease significantly when the node self-learning module is removed. This proves that learning the dynamic spatial correlation among nodes from the state information of traffic flow at different moments better expresses how the traffic flow dynamics change in reality; (3) the performance metrics values of w/o ResNet compared to STCGAT are significantly larger, i.e., removing the residual module has a more significant impact on both datasets. This indicates that the application of the residual module helps STCGAT to mitigate the problem of overfitting or gradient disappearance caused by the superposition of network layers to a certain extent; (4) the performance metrics of with or without reverse GRU also increase on both the PeMS04 and PeMS08 datasets, indicating that effectively capturing the causality of traffic flow data helps to analyze the spatio-temporal correlation of traffic flow more comprehensively; (5) compared with the four variants, STCGAT has the best performance. This statement highlights two key observations regarding the performance of STCGAT. Firstly, it underscores the significance of individual modules within the STCGAT framework. Secondly, it demonstrates the superior ability of STCGAT to effectively capture and extract spatio-temporal correlations present within traffic flow series.