A bike-sharing demand prediction model based on Spatio-Temporal Graph Convolutional Networks

Spatio-temporal feature construction

The input spatio-temporal feature of ST-BDP includes weather conditions, time series signals and demand characteristics of space station groups.

Demand graph feature

Demand graph feature description

The bike-sharing user demand at various bike-sharing stations in the cities is interdependent. This phenomenon aligns with “Tobler’s First Law” (TFL), which posits that “All things are related to other things, but near things are more related than distant things” (Tobler, 1970). To capture the intricate relationships between bike-sharing stations, the historical demand characteristics are structured into graph features, encapsulating the influence dynamics among stations. By introducing demand graph features, ST-BDP can have the ability to accurately identify the spatio-temporal information in historical demand data.

In the demand graph feature for bike-sharing stations, each station represents a vertex, while the physical distance between stations defines the edges. The weights of the edges represent the influence of users on the demand for shared bikes between stations. The magnitude of bike demand at each station is intrinsically linked to these edges, facilitating the construction of demand graph characteristics for the bike-sharing stations.

The graph features are defined as $G=(V,E,W)$, where V is the vertex of the graph, E is the vertex of the graph and the edge between the vertices, and the relationship between the individual nodes will form a $N\times N$-dimensional matrix, i.e., the adjacency matrix W, $W\in R^{N\times N}$. In the demand graph feature for bike-sharing stations, each vertex value represents the demand at a specific station at a given time. Let T denote the time interval, with each interval $T_i$ corresponding to a specific demand map. Figure 1 illustrates the demand graph across various time periods. At any given time $T_i$, the graph can be represented as $G_{T_i}=(V_{T_i},E,W)$; here, $V_{T_i}$ is the vertex set indicating the demand at $n$ bike-sharing stations. E denotes the edge set, where each edge signifies the relationship between two interconnected stations. Meanwhile, $W\in R^{N\times N}$ stands for the adjacency matrix.

The weights of the adjacency matrix W are computed using Eq. (1). In this equation, $d_{ij}$ represents the distance between stations $i$ and $j$. $w_{ij}$ signifies the weights associated with the edges in set E. ${\sigma }^2$ corresponds to the distribution variance of the matrix W, and $\theta$ is the sparsity threshold value.

The demand graph features of bike-sharing stations are constructed to reflect the relationship between the station locations and their corresponding users’ demand.

(1) $$w_{ij}=\{\begin{array}{l}\exp (-\frac{d_{ij}^2}{{\sigma }^2}),i\ne j\mathrm{a}\mathrm{n}\mathrm{d}\exp (-\frac{d_{ij}^2}{{\sigma }^2})\ge \theta \\ \\ 0,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{s}\end{array}.$$

Demand graph feature construction

We construct demand graph features using shared bike riding data from users in New York City, USA from 2017/01/01 to 2020/12/31. This data was sourced from the Citi Bike sharing system, accessible at https://www.citibikenyc.com/system-data. Table 2 shows the data items of bike-sharing riding data.

Table 2:

The data items of bike-sharing riding data.

Data items	Data type	Unit
Start time	Int	second
Start station ID	String	–
Start station name	String	–
Start station latitude	String	–
Start station longitude	String	–
End time	Int	second
End station ID	String	–
End station name	String	–
End station latitude	String	–
End station longitude	String	–

DOI: 10.7717/peerj-cs.2391/table-2

Using the riding data, the hourly usage of bike-sharing at each station is computed and consolidated into a dataset representing users’ bike-sharing demand. Examples of demand data for various stations are presented in Table 3.

Table 3:

Examples of bike-sharing demand data.

Time	Station A	Station B	Station C	Station D	Station E
2019-07-09 06:00	20	5	5	8	10
2019-07-09 07:00	48	19	18	22	20
2019-07-09 08:00	106	26	26	28	32
2019-07-09 09:00	68	30	22	26	27
2019-07-09 10:00	13	20	11	18	15
2019-07-09 11:00	20	21	15	15	15
2019-07-09 12:00	14	27	19	19	19
2019-07-09 13:00	13	21	19	19	16
2019-07-09 14:00	16	15	23	24	21
2019-07-09 15:00	21	33	22	18	15
2019-07-09 16:00	51	27	30	40	29
2019-07-09 17:00	163	78	111	47	65
2019-07-09 18:00	108	67	137	41	88

DOI: 10.7717/peerj-cs.2391/table-3

To prevent the data from entering the saturation region of the activation function and subsequently causing vanishing gradients, we normalize the demand data. Given that the primary activation functions of the models are the Tanh and ReLU function—which are particularly sensitive to values within the range $[-1,1]$—the demand data is normalized to fall within this range, thereby facilitating more effective model training.

The adjacency matrix of the bike-sharing demand graph feature is calculated by the distance between the bike-sharing stations in the traffic network, and the calculation (Eq. (1)). The time interval of the data set is set to 1 h, and each hour corresponds to one bike-sharing demand graph, i.e., each vertex in the network has 24 data points per day.

Weather feature

Weather feature description

Natural weather determines whether users choose bike sharing as a mode of travel, which in turn affects the demand for bike sharing. Therefore, in order to make accurate demand forecasts for shared bikes, weather factors need to be considered. Combined with spatio-temporal feature equations, a series of processes of data attribute calibration, cleaning, feature encoding and scaling are required in order to extract weather features from the data.

Attribute calibration and data cleansing. The primary attributes of weather data encompass time stamps, temperature, dew point, humidity, and wind speed. These data, collected at regular intervals, are associated with weather indicators for that time period. Due to collection irregularities, there are occasional redundancies and missing values. For integer-type data, linear interpolation is employed for imputation, while string-type attributes such as wind direction and weather type utilize up-filling techniques.

Feature coding. Weather-related data attributes, such as wind direction and weather type, are typically represented as strings. To facilitate their use in ST-BDP, certain feature data undergoes encoding and transformation processes. Specifically, the fixed number of weather types is represented by using one-hot encoding. Meanwhile, string-based wind data is translated into an angular representation, delineating wind direction within the [0, 360] degree range, with 0 degrees denoting due north and encompassing 16 distinct wind directions. Notably, direct angle values present challenges for model inputs, especially when values approach both 0 and 360 degrees, which should be interpreted as closely adjacent. To address this, wind direction and speed are translated into wind vectors, as demonstrated in Eqs. (2) and (3).

(2) $$win{d}_x=speed\times \cos \left(\frac{direction\times \pi }{180}\right)$$

(3) $$win{d}_y=speed\times \sin \left(\frac{direction\times \pi }{180}\right).$$

Feature scaling. Weather data exhibits a long-tailed distribution, where samples are unevenly distributed, and there’s a substantial variance in data values. This imbalance can adversely affect subsequent ST-BDP training and degrade the overall model performance. To mitigate this, we introduce Eq. (4), a feature normalization approach for weather attributes. In the equation, $x$ represents the input feature, $x_{mean}$ denotes its mean, and $x_{std}$ stands for its variance. By employing this normalization, we can alleviate issues arising from the skewed sample distribution, harmonize feature value disparities, and markedly enhance the stability of the quality of ST-BDP’s input weather feature.

(4) $$y=\frac{x-{x_m}_{ean}}{{x_s}_{td}}.$$

Weather feature construction

The weather data of LaGuardia Airport in New York, USA was selected to construct the weather feature of ST-BDP for experiments. Table 4 shows the data type and unit of the main data items of the weather data, all from the Weather Underground website (https://www.wunderground.com).

Table 4:

Data items of weather data.

Data items	Data type	Unit
Timestamp	Int	second
Temperature	Int	F
Dew point	Int	F
Humidity	Int	%
Wind speed	Int	mph
Pressure	Float	in
Precipitation	Float	in
Wind direction	String	–
Condition	String	–

DOI: 10.7717/peerj-cs.2391/table-4

Weather data undergoes data cleaning and encoding procedures to transform wind direction and wind speed into wind vectors. Figure 2A depicts the wind direction and wind speed prior to conversion, and Fig. 2B illustrates the resultant wind vectors post-conversion.

To mitigate the long-tail distribution and achieve a more uniform data spread, the weather feature within the dataset is normalized. Figure 3 displays the violin plots of primary weather attributes both prior to and following the feature scaling process.

Table 5 shows some example results of the weather data used in experiment after feature scaling.

Table 5:

Examples of weather features (T: Temperature; D: Dew point; H: Humidity; W X: Wind vector X; W Y: Wind vector Y).

Time	T	D	H	W X	W Y
06:00	72	65	78	2.583	−5.4156
07:00	72	65	78	3.536	−3.536
08:00	74	65	73	5	0
09:00	78	66	66	1.148	2.772
10:00	81	63	54	0	0
11:00	83	59	44	6.364	−6.364

DOI: 10.7717/peerj-cs.2391/table-5

Upon compression of the weather features, a data analysis was conducted, followed by a linear regression with the demand for bike-sharing. As depicted in Fig. 4, there exists a positive correlation between the demand for bike-sharing and both temperature and dew point. Conversely, humidity and precipitation exhibit a negative correlation with demand. The relationships between demand and wind speed, as well as between demand and air pressure, are not clearly delineated and cannot be succinctly represented as linear associations. This also proves the rationality of ST-BDP model using deep learning methods to learn these nonlinear complex relationships.

Temporal feature

Temporal feature description

Bike-sharing demand exhibits periodic patterns across daily, weekly, and yearly timeframes. Nonetheless, representing time in mere seconds results in a monotonic increase, failing to capture these inherent cyclical tendencies as indicated by Phillips, Parr & Riskin (2003). In light of this, temporal features are constructed using sine and cosine functions, capitalizing on their intrinsic periodic properties to effectively represent temporal data.

Time $t$ in seconds is extracted from the weather and bike-sharing demand data. $t$ is initially transformed into a timestamp, which is then further converted into time signals—specifically, sine and cosine signals. The employed time signals encompass $da{y}_{\sin }$, $da{y}_{\cos }$, $wee{k}_{\sin }$, $wee{k}_{\cos }$, $yea{r}_{\sin }$, and $yea{r}_{\cos }$. These signals capture the periodicities in bike-sharing demand across varying durations: daily, weekly, and yearly. Serving as crucial temporal characteristic, these signals constitute one of the input features for the ST-BDP model.

Equations (5) and (6) facilitate the transformation of daily timestamps into $da{y}_{\sin }$ and $da{y}_{\cos }$. For every hour, a unique temporal point is denoted by the pair $(da{y}_{\sin },da{y}_{\cos })$. This pairing effectively represents a specific hour within a day, with both $da{y}_{\sin }$ and $da{y}_{\cos }$ completing a cycle over a 24-h span, thereby encapsulating the daily periodicity.

(5) $$da{y}_{\sin }=\sin [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24}\right)]$$

(6) $$da{y}_{\cos }=\cos [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24}\right)]$$

Equations (7) and (8) facilitate the transformation of weekly timestamps into $wee{k}_{\sin }$ and $wee{k}_{\cos }$. These functions, $wee{k}_{\sin }$ and $wee{k}_{\cos }$, complete one cycle over a span of 1 week (equivalent to $7\times 24$ h), capturing the weekly periodicity exhibited by the demand for bike-sharing.

(7) $$wee{k}_{\sin }=\sin [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24\times 7}\right)]$$

(8) $$wee{k}_{\cos }=\cos [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24\times 7}\right)]$$

Equations (9) and (10) are employed to transform yearly timestamps into $yea{r}_{\sin }$ and $yea{r}_{\cos }$. These terms, $yea{r}_{\sin }$ and $yea{r}_{\cos }$, cycle annually, with each instance within the year being uniquely represented by the coordinate pair $(yea{r}_{\sin },yea{r}_{\cos })$. They encapsulate the annual periodicity inherent in the demand for bike-sharing.

(9) $$yea{r}_{\sin }=\sin [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24\times 365.2425}\right)]$$

(10) $$yea{r}_{\cos }=\cos [timestamp\times \left(\frac{2\times \pi }{60\times 60\times 24\times 365.2425}\right)].$$

Temporal feature distribution

Daily demand data distribution. The demand for bike-sharing is related to time and exhibits distribution characteristics based on daily, weekly, and yearly cycles. Taking the Pershing Square North station (with a longitude of 40.751873 and a latitude of −73.977706) from the dataset used in this study as an example, its relationship between bike-sharing demand and time within 1 day (from 0–23 h on 2017/2/13) is shown in Fig. 5A. Similarly, the relationship between bike-sharing usage and time over a week (from 0–168 h, between 2017/2/13 and 2017/2/19) is depicted in Fig. 5B. Within a single day, there are two peak demand periods for shared bicycles, which correspond to the morning rush hours from 7:00 to 9:00 and the evening rush hours from 17:00 to 19:00. The usage at night is lower than during the day, and this pattern is consistent every day. This indicates that the demand for bike-sharing follows a daily cyclical distribution pattern.

Weekly demand data distribution. The demand for bike-sharing exhibits a cyclical pattern on a weekly basis. Taking the Pershing Square North station as an example, its line chart of demand over three weeks (from 0–504 h, between 2017/2/13 and 2017/3/6) is shown in Fig. 6A. The results of grouping and summing the demand for bike-sharing by day of the week are presented in Fig. 6B. The demand for bike-sharing on weekends is noticeably lower than on weekdays, and this pattern of demand distribution is consistent each week. This illustrates that the demand for bike-sharing follows a weekly cyclical distribution pattern.

Yearly demand data distribution. The demand for bike-sharing exhibits an annual periodicity. Taking the Pershing Square North station as an example, the relationship between the demand for bike-sharing usage and time over three years is shown in Fig. 7. It can be observed that the bike-sharing usage demand has a similar distribution pattern each year, indicating that the demand for bike-sharing follows an annual cyclical pattern. Furthermore, from December to February of the following year, which constitutes winter in New York, there’s a noticeable dip in the demand for bike-sharing. This highlights that weather conditions also have a significant impact on the long-term demand for bike-sharing.

According to the previous description, the demand for bike-sharing shows a distribution pattern with daily, weekly, and yearly cycles. For the construction of temporal features, the data are processed by sine and cosine functions (Eqs. (5), (6)) which are used to represent the temporal feature information of temporal data. Figure 8 shows the day signal plot for a day, with the horizontal coordinates of time and the vertical coordinates of the values of $da{y}_{\sin }$ and $da{y}_{\cos }$. $da{y}_{\sin }$ and $da{y}_{\cos }$ characterize the periodicity of the demand for bike-sharing usage in terms of days.

Equations (7)–(10) are utilized to convert the timestamp into $wee{k}_{\sin }$ and $wee{k}_{\cos }$ (the period is 7 $\times$ 24 h), and $yea{r}_{\sin }$ and $yea{r}_{\cos }$ (the period is 365 days) to construct temporal features with time periodicity. Table 6 shows some of the time signals in the study data.

Table 6:

Time signals.

Time	Day sin	Day cos	Year sin	Year cos
06:00	−0.5	0.866	−0.110	−0.99392
07:00	−0.259	0.966	−0.111	−0.99384
08:00	−0.751	1	−0.112	−0.99376
09:00	0.259	0.966	−0.112	−0.99368
10:00	0.5	0.866	−0.113	−0.9936
11:00	0.707	0.707	−0.114	−0.11366

DOI: 10.7717/peerj-cs.2391/table-6

Model structure

ST-BDP model operates on an encoder-decoder architectural framework. Within ST-BDP model, the STGCNTCN and STGCN blocks are adeptly employed to distill diverse spatio-temporal characteristics, thereby enhancing the accuracy for bike-sharing demand predictions. The ST-BDP model, architected in an Encoder-Decoder framework, predominantly encompasses four neural networks: STGCN, TCN, Conv1D, and Self-Attention. The structure of ST-BDP model is depicted in Fig. 9.

Spatio-temporal feature construction: The multi-input for ST-BDP adeptly integrates features pertaining to spatio-temporal information, encompassing aspects such as demand graph feature of space station groups (demand-characteristic), weather conditions (weather-characteristic), and time series signals (time-characteristic).

STGCN and TCN-based encoder: ST-BDP’s encoder is designed for feature vector encoding. The TCN blocks are utilized to extract the time-characteristic and weather-characteristic’s feature. The GNN-based STGCN block is utilized to extract demand characteristics’ spatio-temporal feature. Finally, the comprehensive representation of all input features is obtained through the encoder’s feature fusion layer, Con1D, residual cascade, and normalization layer.

Self-Attention-based decoder: The feature vector output by Encoder is transformed into time series data of future time period by Self-Attention and residual connection, and finally dimensional transformation is performed by Conv1D to achieve the prediction of future demand of bike-sharing.

Relative to conventional demand forecasting frameworks, ST-BDP model based on STGCN has the advantage of richer training parameters and a deeper model. Notably, all layers of ST-BDP model are convolutional operations or other structures that can process in parallel. This model structure, devoid of sequential operations, offers a distinct advantage in training time over RNN-based configurations. Leveraging the ResNet network architecture (Targ, Almeida & Lyman, 2016), the Add block within ST-BDP model integrates the dense vector produced by the feature fusion layer with the output vector from the STGCN Block. This integration ensures that the demand characteristics of the spatial station clusters significantly influence the prediction results, while mitigating potential interference from secondary features. Concurrently, this approach addresses the vanishing gradient issue in ST-BDP model, which can arise due to an excessive number of neural network layers.

STGCN and TCN-based FR block

In ST-BDP model, the Encoder component is responsible for encoding feature vector, which is subsequently processed by a feature fusion layer and a residual concatenation layer for enhanced feature learning. This component is composed of modules: the STGCN block, the TCN blocks, feature fusion layer, residual concatenation layer, and a normalization layer. Specifically, the STGCN block focuses on extracting hidden information from the demand graph features, while the TCN blocks are dedicated to capturing temporal and weather-related feature information.

TCN block

Block2 and Block3, which are utilized to extract weather and temporal features, are composed of four stacked TCN residual blocks (Bai, Kolter & Koltun, 2018). The architecture of a TCN residual block is illustrated in Fig. 10. A TCN residual block encompasses a series of transformations. The output of TCN block is obtained by adding its input $x$ to the transformations $F(x)$ made within the residual block, as represented in Eq. (11).

(11) $$o=Activation(x+F(x)).$$

The TCN residual block is composed of two dilated convolution layers (Yu & Koltun, 2015) and a ReLU layer. Weight normalization (Salimans & Kingma, 2016) is employed for normalization. Moreover, a dropout layer (Srivastava et al., 2014) is added after each dilated convolution for regularization purposes. To accommodate varying input and output dimension sizes, beyond the basic $3\times 3$ convolution kernel, an additional $1\times 1$ convolution kernel is utilized for dimension transformation. This transforms the input into a tensor with the same shape as the output, ensuring that the input and output can be added together seamlessly.

STGCN block

STGCN block, as shown in Fig. 11A, is a three-layer structure consisting of two spatio-temporal convolutional blocks (ST-Conv blocks) and one output block. This structure also facilitates the network to fully utilize the bottleneck strategy, achieving dimension compression and feature compression, and adjusting the channel count through the graph convolution layer. Figure 11B illustrates that the ST-Conv block possesses a three-layered structure. The first and third layers are direction temporal gated-Conv layers (Fig. 11C), the second layer is a spatial Graph-Conv layer, the output layer is a convolutional layer, and residual connections and bottleneck strategies are introduced in each block. The input $v_t-M+1,...,v_t$ is uniformly processed by ST-Conv block to capture the spatio-temporal dependencies of the data. Finally, the output layer integrates all features to generate the final prediction result $\hat{v}$.

The input and output of the ST-Conv block are both 3D tensors. The input of block $l$ is $v^l\in R^{M\times n\times C^l}$, and the output is $v^{l+1}\in {\mathbb{R}}^{(M-2(K_t-1))\times n\times C^{l+1}}$. Refer to Eq. (12) for further details.

(12) $$v^{l+1}={\mathrm{\Gamma }}_1^l\times \tau ReLU({\mathrm{\Theta }}^l\times G({\mathrm{\Gamma }}_0^l\times \tau v^l))$$

${\mathrm{\Gamma }}_1^l$ and ${\mathrm{\Gamma }}_0^l$ represent the convolution kernels of the upper and lower Temporal Gated-convs layers of block $l$, ${\mathrm{\Gamma }}_0^l$ represent the convolution kernel of the spatial Graph-Conv layer, $ReLU()$ represent the ReLU activation function. The output block maps the output of the last ST-Conv block into a matrix representing the final prediction result.

The loss function of STGCN is shown in Eq. (13), where $W_{\theta }$ is all the trainable parameters in the model, $v_{t+1}$ is the real label, and $\hat{v}()$ represents the predicted value of the model.

(13) $$L(\hat{v},W_{\theta })=\sum _t{\left||\hat{v}(v_t-M+1,...,v_t,W_{\theta })-v_{t+1}|\right|}^2.$$

The remaining encoder layers

ST-GCN Block1, TCN Block2, and TCN Block3 produce three encoded vectors $x_1$, $x_2$, and $x_3$ corresponding to the input features. These vectors $x_1$, $x_2$, and $x_3$ are merged into a dense vector $\hat{x}$ through a feature fusion layer, comprising a concatenate layer followed by a Conv1D layer. The Concatenate layer stitches the $x_1$, $x_2$, and $x_3$ vector into a vector $\hat{x}$, and Conv1D is used to realize the feature fusion and vector dimension transformation.

When passing through the feature fusion layer, vectors influence each other, potentially resulting in the loss of some important features. In the ST-BDP model, we introduce the residual connection in the ResNet (Targ, Almeida & Lyman, 2016) network and combine the dense vector output of the feature fusion layer with the vector output of STGCN Block1 through the Add block. This ensures that the historical demand feature of bike-sharing significantly influences the prediction result, safeguarding it from interference by less important features. Additionally, this approach prevents the gradient vanishing issue that might arise in the ST-BDP model due to an excessive number of neural network layers.

After multiple layers of processing in ST-BDP, data may deviate from normalization and exhibit an increasing bias. To ensure consistent training behavior, it is pivotal to re-normalize the data such that it exhibits a mean of 0 and a variance of 1. This is especially crucial in prediction task where the spatio-temporal data is diverse in each batch. Given the inherent variability in the data sets, these data may significantly diverge from each other, resulting in minimal correlation between them. To address these challenges, a normalization layer is employed for ST-BDP construction. For a given hidden layer $l$, with H representing the number of hidden neurons, and $a$ being the pre-activation value of a neuron (i.e., $a=wx$), the expectation and standard deviation for normalization are derived as per Eqs. (14) and (15), respectively.

(14) $$u^l=\frac{1}{H}\sum _{i=1}^H{a}_i^l$$

(15) $${\sigma }^l=\sqrt{\frac{1}{H}\sum _{i=1}^H{(a_i^l-u^l)}^2}.$$

Subsequently, the normalized output is computed using Eq. (16), where parameters $g$ and $b$ signify the gain and bias. Importantly, these parameters are adaptable and are fine-tuned during the training process. This normalization ensures that the activations across the features in a layer maintain consistent statistics, facilitating a stable and efficient training regimen.

(16) $$h=\mathrm{R}\mathrm{e}lu(\frac{g^l}{{\sigma }^l}.(a_i^l-u^l)+b).$$

The described structure constitutes ST-BDP’s encoder layers. These layers of the encoder process all inputs and encode them into a feature vector that captures the spatio-temporal characteristics.

Self-attention-based decoder

ST-BDP’s decoder is tasked with interpreting the vectorized output from the Encoder layers, ultimately yielding the projected bike-sharing demand for subsequent time frames. It encompasses a variety of blocks including Self-Attention computing, residual connections, one-dimensional convolutional networks, and normalization.

The decoder of ST-BDP is implemented based on the Self-Attention mechanism. Distinct from RNN-like structures that rely on sequential computations, Self-Attention operates in parallel, thereby offering superior computational efficiency. This mechanism is adept at capturing long-term dependencies without being hampered by the temporal distance between events. Moreover, Self-Attention prioritizes salient features and diminishes the effect of less pertinent ones on the predictions, an attribute that bolsters the precision of the predicted outcomes.

The Self-Attention computation within ST-BDP model is a transformative process where a vector sequence X undergoes encoding to produce another vector sequence Y, and the entirety of this Self-Attention computing is executed in parallel. The Scaled Dot-Product Attention mechanism is given by Eq. (17), where $d_k$ represents the dimension of $k$.

(17) $$Attention(\mathcal{Q},K,V)=softmax\left(\frac{\mathcal{Q}{K}^T}{\sqrt{d_k}}\right)V.$$

The Self-Attention algorithm employs the aforementioned Scaled Dot-Product Attention formula. For Self-Attention, $\mathcal{Q}$, K, and V all emanate from a unified vector. Specifically, within the ST-BDP model, $\mathcal{Q}$, K, and V are derived by the multiplication of the Encoder’s output vector $x$ with three distinct matrices: $w_1,w_2,$ and $w_3$. This process is illustrated in Eq. (18). Notably, matrices $w_1,w_2,$ and $w_3$ undergo training in tandem with ST-BDP model’s other hyper parameters.

(18) $$\mathcal{Q},K,V=w_1(x),w_2(x),w_3(x).$$

After undergoing the Self-Attention layer, the output vector $x$ is transformed into another vector $y$. This vector $y$ is then combined with $x$ to produce the resultant vector $\tilde{y}$. To expedite the training process, $\tilde{y}$ is processed through a Layer Normalization layer. Subsequently, a one-dimensional convolution, Conv1D, is employed to convert the feature dimensions and deliver the predict of future bike-sharing demand. Opting for Conv1D over a linear layer was a strategic choice, as the convolutional operation intrinsic to Conv1D offers fewer parameters, resulting in more efficient training. The Decoder culminates its operations by producing a matrix symbolizing future bike-sharing demand.

Dataset description

The bike-sharing demand dataset comprises demand graph data, weather data, and temporal data. Concerning temporal configurations, both input and output time spans are fixed at 72 h. Consequently, data pertaining to demand, weather specifics, and temporal signals from the preceding 72 h are employed to predict bike-sharing demand for the subsequent 72 h. This equates to a sliding window of 144 h, where the initial 72 h of demand data, weather information, and time signals serve as inputs to ST-BDP model, while the demand for the ensuing 72 h acts as labels. This window is then shifted backward hourly until the entirety of the dataset is processed.

Experimental settings

Experimental environment and model training

The experimental setup employed for the program was configured within the PyCharm IDE, utilizing Tensorflow 2.4 as the deep learning development library and Jupyter Notebook as the development tool. The hardware specifications of the experimental equipment include an Intel(R) Core(TM) i5-9300H CPU @ 2.40 GHz processor, an NVIDIA GeForce GTX 1660 Ti graphics card, and 16GB of DDR4 memory.

For model training, the ST-BDP model employed the Adam optimizer, set with a learning rate of For model training, the ST-BDP model employed the Adam optimizer, set with a learning rate of triggered when the decrease in the loss value is less 19, with its functional representation illustrated in Fig. 12. Given the susceptibility of bike-sharing demand predictions to anomalies—such as special holidays or extreme weather events which can induce unpredictable demand fluctuations—the Log-Cosh function was deemed suitable for this task. This function is twice differentiable everywhere and effectively mitigates the impact of outliers during model training.

(19) $$Loss=\sum _{i=1}^m\log (\cosh (y_i-{\hat{y}}_i)).$$

For the ST-BDP model, the following parameter configurations were employed:

STGCN component:

The ST-Conv block, which is a part of the STGCN, has three layers with channels set to 64, 16, and 64, in sequential order.

Both the graph convolution kernel size (K) and the temporal convolution kernel size ( $K_t$) are configured to 3.

TCN component:

The TCN block within the ST-BDP model comprises four stacked TCN residual blocks. All these blocks utilize convolutional kernels of size 3.

Training details:

Optimizer: Adam optimizer is employed for the model training.

Loss function: The chosen loss function is Log-Cosh.

Time slice: The length of the time slice is set to 72.

Batch size: For each iteration, the batch size is fixed at 50.

Initial learning rate: Set to 0.001.

Early stopping criterion: Training is terminated prematurely if the loss value descends below 0.0002 for three consecutive times.

The influence of various saptio-temporal features

To observe the influence of various features on the prediction results, we constructed a single-layer linear computational model, taking historical demand, weather, and time data as inputs to predict the future usage demand for bike-sharing. By analyzing the model parameters, we derived the influence weight ratio of input features on the output, as shown in Fig. 13. The experimental results demonstrate that the absolute value of the influence weight of historical demand exceeds 0.6, making it the most dominant feature among all the input characteristics. Thus, historical demand is the most influential factor on the prediction results, indicating that it is the most crucial input feature.

From the aforementioned analysis, it is evident that historical demand is the most crucial input feature. Therefore, in BDP model, the vector generated by inputting the historical demand graph feature into STGCN Block1 should be the vector with the highest association weight.

Model structure rationalization experiment

In order to verify the rationality of Self-Attention in ST-BDP model and analyze the influence of different feature inputs on the prediction performance, BDP model, TCN model, and two variants of BDP model (BDP-1 and BDP-2) are selected for performance comparison experiments. The four models are described as follows:

• BDP model: This model shares a structural similarity with ST-BDP model, being mainly bifurcated into the Encoder and Decoder segments. Distinctively, the BDP model does not incorporate demand graph features during its feature construction. Instead, it relies solely on the analysis of historical demand data.

• TCN model: This model is mainly used for time series prediction, including time series prediction, causal convolution, inflated convolution, TCN residual block and other parts. Notably, it does not employ the Self-Attention mechanism, marking a difference from the BDP model.

• BDP-1 model (without temporal feature): This model variation excludes the characteristic information of the time signal. Specifically, it omits the TCN Block2, which processes the input temporal feature in the original BDP model.

• BDP-2 model (without weather feature): BDP-2 abstains from incorporating weather feature information. Within this model, the TCN Block 3, responsible for processing the weather factor, is removed from the BDP structure.

Self-attention performance impact analysis

When comparing TCN model to BDP model, a key distinction is the absence of the Self-Attention mechanism in TCN model. In Table 7, a detailed comparison between the prediction performance of the TCN and BDP models on the bike-sharing demand dataset is presented.

Table 7:

Prediction performance of the TCN and BDP models.

Model	MAE	MAPE	SMAPE	RMSE
TCN	2.11	19.93%	20.42%	3.21
BDP	1.80	18.61%	18.82%	2.67

DOI: 10.7717/peerj-cs.2391/table-7

By analyzing the experimental results in Table 7, it is found that the prediction performance of BDP model is better than that of TCN model, with lower MAE, MAPE, SMAPE, RMSE and higher accuracy. As the models’ prediction results shown in Fig. 14, BDP model’s predictions align more closely with the actual data compared to those from the TCN model. This superior performance of the BDP model demonstrates the efficacy of the Self-Attention mechanism in effectively capturing the long-term dependencies inherent in bike-sharing demand. By emphasizing relevant features and diminishing the impact of less pertinent ones, the Self-Attention mechanism enhances the model’s predictive accuracy.

Multi-spatio-temporal feature performance impact analysis

The performance comparison between the BDP model, BDP-1 model, and BDP-2 model, each having varied feature inputs, on the curated bike-sharing demand dataset, can be observed in Table 8.

Table 8:

Prediction performance of BDP-1, BDP-2 and BDP models.

Model	MAE	MAPE	SMAPE	RMSE
BDP-1	2.36	22.38%	22.92%	3.44
BDP-2	3.07	22.88%	23.62%	5.49
BDP	1.80	18.61%	18.82%	2.67

DOI: 10.7717/peerj-cs.2391/table-8

Figure 15 illustrates the predictive performance of the BDP model and its two variants, bike-sharing demand prediction for the subsequent 72 h. BDP-1 model showcases a higher prediction accuracy compared to BDP-2 model. This suggests that weather features play a more substantial role in influencing prediction accuracy than the time signals. This is likely because the input historical demand data inherently functions as a time series, inherently capturing some of the temporal feature. However, the accuracy of BDP-1 model falls short when compared to the primary BDP model. This is attributed to the fact that the temporal features embedded within the historical demand series do not adequately capture the cyclic nature of bike-sharing demand. In contrast, explicit temporal signals provide a more comprehensive representation of this cyclic behavior.

Baseline comparison experiments

ST-BDP model’s predictive performance is benchmarked against an array of baseline models to understand its robustness and accuracy in bike-sharing demand prediction. These baseline models encompass traditional statistical methods, machine learning algorithms, and deep learning architectures:

• Historical average (HA): This model capitalizes on historical data, simply utilizing the average of previous periods as its predicted value.

• Autoregressive Integrated Moving Average model (ARIMA): A stalwart in the time-series forecasting realm, ARIMA models data in a time series based on historical patterns. Its limitation lies in its inability to aptly fit irregular historical data. Furthermore, it primarily looks at time dependence, sidelining spatial network attributes, which could influence demand prediction.

• Multilayer perceptron (MLP): A feedforward neural network that can solve nonlinear problems by stacking multiple layers of neurons and introducing nonlinear activation functions. However, for high-dimensional data, this network has high requirements for feature engineering and data preprocessing.

• Recurrent neural network (RNN): This network model have internal states that allows them to remember contextual information from sequences, capturing dependencies effectively. This makes them well-suited for processing and predicting sequence data. However, RNNs struggle with capturing long-term dependencies in long sequences, often encountering issues such as vanishing or exploding gradients.

• Long short-term memory (LSTM): This network model can effectively capture and retain long-term dependencies by introducing memory units and gating mechanisms. However, due to its time step dependency, this model is difficult to train in parallel.

• Bidirectional LSTM (BiLSTM) (Schuster & Paliwal, 1997): Bidirectional LSTM can more accurately capture the bidirectional dependencies in the time series by processing the forward and backward information of the sequence simultaneously, thereby improving the prediction performance.

• Gated recurrent unit (GRU): The GRU controls the writing of new information by merging the input gate and the forget gate into an update gate. However, it is difficult to perform well in complex prediction tasks because it is difficult to control the flow of information in a fine-grained manner.

• Convolutional neural network (CNN): A deep learning model that can efficiently process grid-structured data through local connections, weight sharing, and hierarchical feature extraction. However, the network model is not robust enough to translation, rotation, and scale changes of the input data, and its effectiveness depends on the quality of data annotation.

• Linear support vector regression (LSVR): A machine learning algorithm that is commonly used in binary classification problems, but can also be used to solve time-series prediction problems. It is mainly used to fit numerical values and is generally applied to scenarios with sparse features and a small number of features.

• Feed-forward neural network (FNN) (Zhang et al., 2022): A basic neural network model with high applicability. The model can be considered as a function that implements a complex mapping from the input space to the output space through multiple compositions of simple nonlinear functions.

• Graph convolutional GRU (GCGRU): A kind of graph convolutional neural network designed for temporal prediction with good prediction results, but the computation process is sequential computation of GRU.

• Spatio-Temporal Graph Convolutional Networks (STGCN): A graph neural network improved variant model with spatio-temporal convolutional (TCN) blocks that can capture temporal and spatial features of the demand data, and the computation process is parallel. Compared with ST-BDP, the input features of STGCN do not introduce weather and periodic time features, and STGCN is not designed as an encoder-decoder model structure.

Table 9 shows the prediction performance of the ST-BDP model and the baseline model on the experimental dataset, where the time window of the LSTM model is 24 time steps, each time step is 1, and the grid search method is used to optimize the hyperparameter settings:

$$lr\in [0.001,0.0005,0.0001]$$

$$batch\mathrm{\_}size\in [16,32,64]$$

$$unit\mathrm{\_}number\in [64,128,256].$$

The bidirectional LSTM model selects the same parameter settings as the LSTM model.

Through comparison of experimental results, it is found that the demand forecasting model based on deep learning (LSTM, RNN, GRU, GRU, CNN, GCGRU, STGCN, ST-BDP) has better prediction accuracy and lower prediction error rate than the demand forecasting model based on statistical learning methods (HA, ARIMA) and machine learning methods (MLP, LSVR, FNN).

When benchmarked against other models, graph neural network-based models (GCGRU, STGCN, ST-BDP) have excellent performance in demand forecasting. While traditional statistical learning methods and machine learning-based demand forecasting models can effectively address short-term forecasting challenges, their over-reliance on historical patterns, error accumulation, and insufficient spatial data integration affect the accuracy and effectiveness of their long-term forecasts. Although models such as LSTM, RNN, BiLSTM, and BDP can capture dependencies in time series data, they only rely on temporal features. Therefore, they cannot explain the complex spatial relationships and grid structures inherent in certain forecasting tasks, resulting in low forecasting accuracy. In contrast, the ST-BDP model built on the basic STGCN model adopts STGCN and TCN blocks to encapsulate additional features (demand graph, temporal signals, and weather information). This infusion of diverse feature information, especially the demand graph feature, renders the ST-BDP model exceptionally proficient, evidenced by its commendable performance across all four evaluation criteria: MAE, MAPE, SMAPE, and RMSE.

The prediction results of the ST-BDP model and other baseline models for the shared bicycle demand in the next 72 h are shown in Figs. 16 and 17. The baseline models shown in Fig. 17 are prediction models that only consider time characteristics. By observing the experimental indicators in the experimental results, STGCN model and ST-BDP model capture the trend of bike-sharing demand during the peak time period more accurately than the other models. This shows that the models based on efficient graph convolution and stacked temporal convolution structures, can quickly respond to dynamic demand changes in the bike-sharing network without overrelying on historical averages as most recurrent neural networks do.

In terms of model prediction efficiency, Fig. 18 shows the prediction time of the GCGRU model, STGCN model, and ST-BDP model on some data in the test set. In the experiment, the GCGRU model contains three layers, each with 64, 64, and 128 units. The setting of the STGCN model is basically the same as that of the ST-BDP model. The average prediction time of the three models on the test set obtained in the experiment is 0.571, 0.077, and 0.102s, respectively. Experimental results show that the prediction efficiency of the ST-BDP model is 5.6 times that of the GCGRU model. The ST-BDP model uses a graph convolutional network (GCN) to process spatial dependencies and a temporal convolutional network to process temporal dependencies. This model architecture allows the model to parallelize convolution operations in spatial and temporal dimensions, improving model efficiency. Compared with the STGCN model, the ST-BDP model adds components specifically for processing weather information and time signals, which makes the ST-BDP model structure more complex and slightly increases the prediction time. Thanks to the special design of ST-Conv Blocks and TCN Block, the ST-BDP model excels in capturing spatiotemporal features and processing multi-source data, while effectively balancing time consumption and parameter settings.

In short, the ST-BDP model exhibits superior precision in pinpointing the demand periods. This is because ST-BDP adds two TCN blocks to learn weather features and time periodic features in different dimensions. Furthermore, ST-BDP model is architectured with an encoder-decoder design that incorporates Self-Attention mechanism, enhancing the model’s efficacy even more.