A dynamic graph Hawkes process based on linear complexity self-attention for dynamic recommender systems

Hawkes process on dynamic graph

The Hawkes process on the dynamic graph can model the dynamic embedding of users and items. Specifically, whether the user u or the item v participates in the formation of the interaction $\left(u,v\right)$ at the time t can be quantified by the conditional intensity of this event. (2)${\displaystyle {\lambda }_u\left(t\right)={\mu }_u\left(t\right)+\alpha \sum _{\left(u,v^{\prime },t^{\prime }\right)\in H_u\left(t\right)}{\gamma }_{h{v}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)+\beta \sum _{\left(u^{\prime },v,t^{\prime }\right)\in C_u\left(t\right)}{\gamma }_{c{u}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)}$ (3)${\displaystyle {\lambda }_v\left(t\right)={\mu }_v\left(t\right)+\alpha \sum _{\left(u^{\prime },v,t^{\prime }\right)\in H_v\left(t\right)}{\gamma }_{h{u}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)+\beta \sum _{\left(u,v^{\prime },t^{\prime }\right)\in C_v\left(t\right)}{\gamma }_{c{v}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)}$ where μ_u(t) and μ_v(t) are the basic intensity of the interaction events formed by nodes u or v at the time t, respectively, which are not influenced by historical events on u or v. The basic intensity determines the basic level of the current event, and it can also be understood as the expected probability of the event at the time t. The reasonable setting of the basic intensity can improve the prediction ability and explanation of the model. α and β are positive constants that control the influence of the basic intensity and historical events. If these two parameters are larger, then the impact of historical events on subsequent events will be more lasting. In practical application, we can balance the importance of historical events and the latest events by adjusting α and β.

H_u(t) = {(u, v′, t′) ∈ I:t′ ≤ t} and H_v(t) = {(u′, v, t′) ∈ I:t′ ≤ t} are the sets of historical interaction events of nodes u or v with respect to time t, respectively. Where v′ is the historical interaction neighbor of u, and u′ is the historical interaction neighbor of v. γ_hv′(t′) and γ_hu′(t′) respectively represent the influence degree of the historical interaction neighbor v′ or u′ on the current event at time t′.C_u(t) = {(u′, v, t′) ∈ I:t′ < t} and C_v(t) = {(u, v′, t′) ∈ I:t′ < t} are the sets of historical collaborative interaction events of node u or v with respect to time t, respectively. Where u′ is the historical cooperative interaction neighbor of u, v′ is the historical cooperative interaction neighbor of v. γ_cu′(t′) and γ_cv′(t′) respectively represent the influence degree of the historical cooperative interaction neighbor u′ or v′ on the current event at time t′. In the Hawkes process, the time decay effect of historical events on current events is modeled by kernel functions. The kernel function describes the decay process of the excitation effect of historical events with time. Therefore, the influence of historical events on the interaction probability of current events can be explained as the weighted sum of the excitation effect of historical events in time, in which the weight of time is determined by the kernel function. In the DGHP-LISA model, we use the kernel function κ(⋅) to model the time decay effect of historical events on current events, which can control the decay rate of historical events with time.

Next, combined with dynamic graph representation, the conditional intensity in Eqs. (2) and (3) is materialized. LISA(⋅) is a self-attention module with linear complexity, which can effectively model the importance between update mechanisms by dynamically allocating weights to different embedding update mechanisms. The conditional intensity can be generated from a transfer function f (Mei & Eisner, 2017; Panzarasa, Opsahl & Carley, 2009), i.e., (4)${\displaystyle {\lambda }_u\left(t\right)=f\left(LISA\left(u\left(t^{-}\right),\Delta t_u^{-},u_h\left(t^{-}\right),u_c\left(t^{-}\right)\right)\right)}$ (5)${\displaystyle {\lambda }_v\left(t\right)=f\left(LISA\left(v\left(t^{-}\right),\Delta t_v^{-},v_h\left(t^{-}\right),v_c\left(t^{-}\right)\right)\right)}$ where u(t⁻) and v(t⁻) are the latest embeddings of u or v before time t, respectively. $\Delta t_u^{-}$ and $\Delta t_v^{-}$ respectively represent the time interval between current time t and previous interaction time t⁻ of the node u or v. For dynamic graph, the node to be updated firstly inherits the influence of the previous state and elapsed time, which is used as the basic intensity to learn the embedding of users and items.

u_h(t⁻) and v_h(t⁻) represent the aggregate embedding of the historical interaction neighbors of u or v before the time t, respectively. The proposed model establishes a dynamic bipartite graph to simulate the interaction between user and item nodes, which means that the user’s historical interaction neighbors are the items that he or she has interacted with, and vice versa. In dynamic recommendation scenarios, the item that a user interacts with reflects the user’s recent interest to some extent. Correspondingly, users who are interested in a specific item can be regarded as a part of the item’s properties. Therefore, it is necessary to use the historical interaction neighbors of the user or item to learn its embedding.

u_c(t⁻) and v_c(t⁻) represent the aggregate embedding of the historical collaborative interaction neighbors of u or v before the time t, respectively. In the model, the historical collaborative interaction neighbors of the user or item node are the historical interaction neighbors of the other node involved in the interaction, which capture the collaborative relationship between users and items. Specifically, for a specific item, it may have been purchased by multiple users before the current interaction, and now a new user has purchased the item. It can be assumed that this new user has a collaborative relationship with the users who previously purchased the specific item. Therefore, the update of node embedding in dynamic graph takes into account not only the self-information and historical interaction of nodes, but also the structural information between nodes.

To sum up, the historical interaction aggregation embedding and historical collaborative interaction aggregation embedding of users or items before time t reflect the impact of historical events on current events, which is the key to simulate the excitation effect caused by historical events, combined with the basic intensity to learn the embedding of users and items.

Neighbor aggregation mechanism

It can be seen from Eqs. (4) and (5) that using the Hawkes process on the dynamic graph to model the dynamic embedding of users and items needs to consider the aggregate information of their historical interaction neighbors and historical collaborative interaction neighbors. To aggregate such information, let: (6)${\displaystyle u_h\left(t^{-}\right)=\partial \left(v\left(t^{-}\right),{H^{\prime }}_u\left(t\right)\right)}$ (7)${\displaystyle u_c\left(t^{-}\right)=\partial \left(u\left(t^{-}\right),{C^{\prime }}_u\left(t\right)\right)}$ (8)${\displaystyle v_h\left(t^{-}\right)=\partial \left(u\left(t^{-}\right),{H^{\prime }}_v\left(t\right)\right)}$ (9)${\displaystyle v_c\left(t^{-}\right)=\partial \left(v\left(t^{-}\right),{C^{\prime }}_v\left(t\right)\right)}$ where ∂(⋅) is the aggregator function. H′_u(t) and H′_v(t) represent the historical interaction neighbor embedding set of nodes u or v with respect to time t, respectively. C′_u(t) and C′_v(t) represent the historical collaborative interaction neighbor embedding set of nodes u or v with respect to time t, respectively. (10)${\displaystyle {H^{\prime }}_u\left(t\right)=\left\{{v^{\prime \prime }}_h\left(t^{\prime }\right)|{v^{\prime \prime }}_h\left(t^{\prime }\right)=LISA\left(v^{\prime }\left(t^{\prime }\right)+h\left(t^{\prime }\right)\right),\left(u,v^{\prime },t^{\prime }\right)\in H_u\left(t\right)\right\}}$ (11)${\displaystyle {H^{\prime }}_v\left(t\right)=\left\{{u^{\prime \prime }}_h\left(t^{\prime }\right)|{u^{\prime \prime }}_h\left(t^{\prime }\right)=LISA\left(u^{\prime }\left(t^{\prime }\right)+h\left(t^{\prime }\right)\right),\left(u^{\prime },v,t^{\prime }\right)\in H_v\left(t\right)\right\}}$ (12)${\displaystyle {C^{\prime }}_u\left(t\right)=\left\{{u^{\prime \prime }}_c\left(t^{\prime }\right)|{u^{\prime \prime }}_c\left(t^{\prime }\right)=LISA\left(u^{\prime }\left(t^{\prime }\right)+h\left(t^{\prime }\right)\right),\left(u^{\prime },v,t^{\prime }\right)\in C_u\left(t\right)\right\}}$ (13)${\displaystyle {C^{\prime }}_v\left(t\right)=\left\{{v^{\prime \prime }}_c\left(t^{\prime }\right)|{v^{\prime \prime }}_c\left(t^{\prime }\right)=LISA\left(v^{\prime }\left(t^{\prime }\right)+h\left(t^{\prime }\right)\right),\left(u,v^{\prime },t^{\prime }\right)\in C_v\left(t\right)\right\}}$

where h(t′) represents the time embedding of time t′.v^′′_h(t′) and u^′′_h(t′) represent the historical interaction neighbor embedding of nodes u or v at time t′ after time encoding, respectively. u^′′_c(t′) and v^′′_c(t′) represent the historical collaborative interaction neighbor embedding of nodes u or v at time t′ after time encoding, respectively. As shown in Fig. 2, in order to enable the model to make full use of time information and capture the time decay effects of different historical events on current events, we add time embedding to all neighbor embedding and model the time correlation of different historical events through the LISA module.

For aggregator function ∂(⋅), we provide the following two candidate aggregator functions for use in neighbor aggregation:

1. Mean aggregator: It is a simple operator to calculate the average value of central node embedding and neighbor node embeddings, which can be regarded as a variant of GCN method (Hamilton, Ying & Leskovec, 2017; Kipf & Welling, 2016). (14)${\displaystyle u_h\left(t^{-}\right)=\frac{1}{1+\left|{H^{\prime }}_u\left(t\right)\right|}\left(v\left(t^{-}\right)+\sum _{{v^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_u\left(t\right)}{v^{\prime \prime }}_h\left(t^{\prime }\right)\right)}$ (15)${\displaystyle u_c\left(t^{-}\right)=\frac{1}{1+\left|{C^{\prime }}_u\left(t\right)\right|}\left(u\left(t^{-}\right)+\sum _{{u^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_u\left(t\right)}{u^{\prime \prime }}_c\left(t^{\prime }\right)\right)}$ (16)${\displaystyle v_h\left(t^{-}\right)=\frac{1}{1+\left|{H^{\prime }}_v\left(t\right)\right|}\left(u\left(t^{-}\right)+\sum _{{u^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_v\left(t\right)}{u^{\prime \prime }}_h\left(t^{\prime }\right)\right)}$ (17)${\displaystyle v_c\left(t^{-}\right)=\frac{1}{1+\left|{C^{\prime }}_v\left(t\right)\right|}\left(v\left(t^{-}\right)+\sum _{{v^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_v\left(t\right)}{v^{\prime \prime }}_c\left(t^{\prime }\right)\right)}$

2. Attention aggregator: Inspired by the GAT (Veličković et al., 2017) model, it uses the attention mechanism to weighted summation of neighbor nodes. (18)${\displaystyle u_h\left(t^{-}\right)=\sum _{{v^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_u\left(t\right)}{\alpha }_h\left(t^{\prime }\right){v^{\prime \prime }}_h\left(t^{\prime }\right)}$ (19)${\displaystyle {\alpha }_h\left(t^{\prime }\right)=\frac{\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[v\left(t^{-}\right)\left|\right|{v^{\prime \prime }}_h\left(t^{\prime }\right)\right]\right)\right)}{\sum _{{v^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_u\left(t\right)}\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[v\left(t^{-}\right)\left|\right|{v^{\prime \prime }}_h\left(t^{\prime }\right)\right]\right)\right)}}$ (20)${\displaystyle u_c\left(t^{-}\right)=\sum _{{u^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_u\left(t\right)}{\alpha }_c\left(t^{\prime }\right){u^{\prime \prime }}_c\left(t^{\prime }\right)}$ (21)${\displaystyle {\alpha }_c\left(t^{\prime }\right)=\frac{\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[u\left(t^{-}\right)\left|\right|{u^{\prime \prime }}_c\left(t^{\prime }\right)\right]\right)\right)}{\sum _{{u^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_u\left(t\right)}\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[u\left(t^{-}\right)\left|\right|{u^{\prime \prime }}_c\left(t^{\prime }\right)\right]\right)\right)}}$ (22)${\displaystyle v_h\left(t^{-}\right)=\sum _{{u^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_v\left(t\right)}{{\alpha }^{\prime }}_h\left(t^{\prime }\right){u^{\prime \prime }}_h\left(t^{\prime }\right)}$ (23)${\displaystyle {{\alpha }^{\prime }}_h\left(t^{\prime }\right)=\frac{\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[u\left(t^{-}\right)\left|\right|{u^{\prime \prime }}_h\left(t^{\prime }\right)\right]\right)\right)}{\sum _{{u^{\prime \prime }}_h\left(t^{\prime }\right)\in {H^{\prime }}_v\left(t\right)}\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[u\left(t^{-}\right)\left|\right|{u^{\prime \prime }}_h\left(t^{\prime }\right)\right]\right)\right)}}$ (24)${\displaystyle v_c\left(t^{-}\right)=\sum _{{v^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_v\left(t\right)}{{\alpha }^{\prime }}_c\left(t^{\prime }\right){v^{\prime \prime }}_c\left(t^{\prime }\right)}$ (25)${\displaystyle {{\alpha }^{\prime }}_c\left(t^{\prime }\right)=\frac{\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[v\left(t^{-}\right)\left|\right|{v^{\prime \prime }}_c\left(t^{\prime }\right)\right]\right)\right)}{\sum _{{v^{\prime \prime }}_c\left(t^{\prime }\right)\in {C^{\prime }}_v\left(t\right)}\text{exp}\left(\text{LeakyRelu}\left(W_{\alpha }\left[v\left(t^{-}\right)\left|\right|{v^{\prime \prime }}_c\left(t^{\prime }\right)\right]\right)\right)}}$

where || is the concatenation operation and W_α ∈ ℝ^2d is a weight matrix.

In practical dynamic recommendation scenarios, due to the huge amount of data, neighbor aggregation needs to cost a high computational cost. Therefore, we select a fixed number of neighbors for aggregation and call the number of neighbor nodes selected as aggregator size.

Self-attention mechanism with linear complexity

Let E_i ∈ ℝ^m×d_in and E_o ∈ ℝ^m×d_out represent the input and output embedding of the LISA module respectively. m represents the number of embedding. d_in and d_out represent the dimensions of input and output embedding respectively. Let Q = E_iW_q, K = E_iW_k and V = E_iW_v denote the Query, Key and Value matrices generated by linear transformation on input embedding, respectively. Where W_q ∈ ℝ^d_in×d_k, W_k ∈ ℝ^d_in×d_k and W_v ∈ ℝ^d_in×d_out are all weight matrices. The LISA module generates output embedding using the following self-attention operations based on linear complexity: (26)${\displaystyle E_o=Q\left(\rho \left(K^{\top }\right)V\right)}$ where K^⊤ denotes the matrix transpose of K, and ρ(⋅) denotes the operation of applying softmax normalization for each row separately. The self-attention operation can be interpreted as first aggregating the features in the V matrix into d_k context vectors using the weights in the ρ(K^⊤), and then reassigning the context vectors back to m using the weights in the Q matrix. The computational and memory complexities of this operation are O(m).

This self-attention operation is similar to the attention operation used by Chen et al. (2018) or Shen et al. (2021), but we do not use softmax standardization for Q matrix. Normalized Q matrix constrains the output embedding to a convex combination of context vectors, which may limit the expression ability of the self-attention mechanism. Therefore, we remove the softmax normalization of Q matrix, which allows the output embeddings to span the entire subspace of the d_k global context vectors.

Transfer function

The key to materialize the conditional intensity is to fit a transfer function f to the Hawkes process on the dynamic graph. In the past, softplus function and its variants are generally used as transfer function in the study of Hawkes process. To ensure that the output of the transfer function is positive, we instantiate f on the basis of Eqs. (7) and (8) as: (27)${\displaystyle {\lambda }_u\left(t\right)=ReLU\left(SUM\left(LISA\left(u\left(t^{-}\right),\Delta t_u^{-},u_h\left(t^{-}\right),u_c\left(t^{-}\right)\right)\right)\right)}$ (28)${\displaystyle {\lambda }_v\left(t\right)=Re\text{LU}\left(SUM\left(LISA\left(v\left(t^{-}\right),\Delta t_v^{-},v_h\left(t^{-}\right),v_c\left(t^{-}\right)\right)\right)\right)}$ where the activation function uses ReLU. SUM(⋅) means that the features generated by different embedding update mechanisms are fused in a simple addition way.

The connection between transfer function and conditional intensity

A well chosen transfer function f , taking the dynamic graph representations of the node as input, is equivalent to the conditional intensity of the Hawkes process in Eqs. (2) and (3). In this section we formally show the connection.

First, we define the base intensity as a function of the self-information: (29)${\displaystyle {\mu }_u\left(t\right)=f_u\left(u\left(t^{-}\right),\Delta t_u^{-}\right)}$ (30)${\displaystyle {\mu }_v\left(t\right)=f_v\left(v\left(t^{-}\right),\Delta t_v^{-}\right).}$ Next, we define the influence of different historical events on current events as a function of historical interaction neighbors and historical collaborative interaction neighbors. (31)${\displaystyle \sum _{\left(u,v^{\prime },t^{\prime }\right)\in H_u\left(t\right)}{\gamma }_{h{v}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)=f_{\gamma }\left(u_h\left(t^{-}\right)\right)}$ (32)${\displaystyle \sum _{\left(u^{\prime },v,t^{\prime }\right)\in C_u\left(t\right)}{\gamma }_{c{u}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)=f_{\gamma }\left(u_c\left(t^{-}\right)\right)}$ (33)${\displaystyle \sum _{\left(u^{\prime },v,t^{\prime }\right)\in H_v\left(t\right)}{\gamma }_{h{u}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)=f_{\tau }\left(v_h\left(t^{-}\right)\right)}$ (34)${\displaystyle \sum _{\left(u,v^{\prime },t^{\prime }\right)\in C_v\left(t\right)}{\gamma }_{c{v}^{\prime }}\left(t^{\prime }\right)\kappa \left(t-t^{\prime }\right)=f_{\tau }\left(v_c\left(t^{-}\right)\right).}$ Given these building blocks, we rewrite the conditional intensity in Eqs. (2) and (3) as: (35)${\displaystyle {\lambda }_u\left(t\right)=f_{\lambda }\left(u\left(t^{-}\right),\Delta t_u^{-},u_h\left(t^{-}\right),u_c\left(t^{-}\right)\right)}$ (36)${\displaystyle {\lambda }_v\left(t\right)=f_{\pi }\left(v\left(t^{-}\right),\Delta t_v^{-},v_h\left(t^{-}\right),v_c\left(t^{-}\right)\right)}$ where f_λ is a composite function of f_u, f_γ and the summation, and f_π is a composite function of f_v, f_τ and the summation. By choosing the right transfer function f , we further rewrite f_λ and f_π as the composition of f and the LISA module.

Finally, the conditional intensity can be expressed as follows: (37)${\displaystyle {\lambda }_u\left(t\right)=\left(f\circ LISA\right)\left(u\left(t^{-}\right),\Delta t_u^{-},u_h\left(t^{-}\right),u_c\left(t^{-}\right)\right)=f\left(LISA\left(u\left(t^{-}\right),\Delta t_u^{-},u_h\left(t^{-}\right),u_c\left(t^{-}\right)\right)\right)}$ (38)${\displaystyle {\lambda }_v\left(t\right)=\left(f\circ LISA\right)\left(v\left(t^{-}\right),\Delta t_v^{-},v_h\left(t^{-}\right),v_c\left(t^{-}\right)\right)=f\left(LISA\left(v\left(t^{-}\right),\Delta t_v^{-},v_h\left(t^{-}\right),v_c\left(t^{-}\right)\right)\right).}$

Project and predict the next item embedding

As shown in Fig. 1, in dynamic recommendation, user u interacts with item v at time t, and then interacts with item i at time t + Δt_u. Our task is to predict items that the user are most likely to interact with before time t + Δt_u, which is an analogy to link prediction problem in dynamic graph. Specifically, after updating the user and item embedding of time t by using the Hawkes process on the dynamic graph, the DGHP-LISA calculates the projected embedding $\hat{u}\left(t+\Delta t_u\right)$ of user u and the predicted embedding $\tilde{i}\left(t+\Delta t_u\right)$ of item i through the projection layer and prediction layer in turn. Then, we calculate the L 2 distance between the predicted item embedding and all other item embeddings, and then recommend items with the smallest distance to the predicted item embedding.

1. Projection layer: According to the method suggested in LatentCross (Beutel et al., 2018), based on the embedding u(t) of user u at time t and the elapsed time Δt_u, we incorporate time into the projected embedding via Hadamard product. The formula is as follows: (39)${\displaystyle \hat{u}\left(t+\Delta t_u\right)=u\left(t\right)\odot \left(1+W_{pro}\Delta t_u\right)}$ where W_pro is used to convert Δt_u into a time-context vector. We initialize W_pro by a 0-mean Gaussian. The vector 1 + W_proΔt_u is used to scale the past user embedding. When Δt_u = 0, the projected embedding is the same as the input embedding vector. The larger the value of Δt_u, the more the projected embedding differs from the input embedding and the projected embedding drifts over time.

2. Prediction layer: After obtaining the projected embedding $\hat{u}\left(t+\Delta t_u\right)$ of user u, we use the prediction layer function to learn the future embedding of item i represented as $\tilde{i}\left(t+\Delta t_u\right)$. We make this prediction based on the current user u and its interaction item v immediately before time t + Δt_u. The formula is as follows: (40)${\displaystyle \tilde{i}\left(t+\Delta t_u\right)=W_{pre}\left[\hat{u}\left(t+\Delta t_u\right)\left|\right|\bar{u}\left|\right|v\left(t\right)\left|\right|\bar{v}\right].}$ As shown in Eq. (40), we use both the static and dynamic embeddings to predict the static and dynamic embedding of item i at time t + Δt_u, and W_pre represents the weight matrix.

Loss function

DGHP-LISA is trained to minimize the L2 distance between the predicted item embedding and the ground truth item’s embedding at every interaction. The loss function is as follows: (41)${\displaystyle L=\sum _{\left(u,v,t\right)\in I}{∥\tilde{v}\left(t\right)-\left[v\left(t^{-}\right)\parallel \bar{v}\right]∥}_2+{\lambda }_u{∥u\left(t\right)-u\left(t^{-}\right)∥}_2+{\lambda }_v{∥v\left(t\right)-v\left(t^{-}\right)∥}_2.}$

The first loss term minimizes the predicted embedding error. Since the items’ and users’ properties tend to be stable in a short time, the last two terms are added to regularize the loss and prevent the consecutive dynamic embeddings of a user and item to vary too much, respectively. I is a sequence of user-item interaction events in chronological order. λ_u and λ_v are scaling parameters, which are used to ensure the losses are in the same range.

Evaluation metrics

We use the following two evaluation metrics for experiments:

1. Mean Reciprocal Rank (MRR) is the average of the reciprocal rank of the first positive example in all user recommendation lists. This indicator can measure the performance of the model with respect to the ranking list of items. Higher MRR score means the ground truth item tends to have higher rank positions in the predicted item lists. In some scenarios, such as ranking search results for specific users, the improvement of MRR can significantly improve the user experience, especially for items with high importance in ranking results. To calculate MRR, we use equation: (42)${\displaystyle MRR\text{=}\frac{1}{\left|I\right|}\sum _{i\in I}\frac{1}{ran{k}_i}}$ where i ∈ I represents traversing all interactions and rank_i represents the location of the ground truth item in the recommendation list for the i-th interaction.

2. Recall@k is the fraction of interactions in which the ground truth item is ranked in the top k. This metrics pays more attention to the integrity and diversity of recommendation results, and it is more suitable to be used in the recommendation system. Because the recommendation system needs to ensure that it covers as many items as possible in the user’s area of interest, so as to improve user satisfaction and participation. In some scenarios, such as e-commerce and news recommendation, the improvement of Recall@k can significantly improve the actual effect of the recommendation system, increase sales or user stickiness and so on. Formally, Recall@k is defined as: (43)${\displaystyle Recall@k\text{=}\frac{n_{hit}}{\left|I\right|}}$ where n_hit is the number of ground truth items that are among the top-k recommendation list, and $\left|I\right|$ is the number of all test cases.

Parameter setting

We use Pytorch to implement DGHP-LISA, and the hyper-parameters are determined by the performance of the model on the verification set. For all algorithms, we use 128-dimensional dynamic embeddings and randomly initialize user and item embeddings with a Gaussian distribution with mean 0 and variance 1. Adam optimizer with learning rate 1e−3, L2 penalty 1e−5 is adopted in our model. Static embeddings all use one-hot vectors. The scaling parameters λ_u and λ_v in loss function are set to 1. All algorithms run for 50 rounds, and the corresponding test set is selected according to the best verification set. All experiments are run independently in the same experimental environment with Intel(R) Xeon(R) Gold 5118 host and NVIDIA Tesla V100-SXM2-32GB GPU. For comparison methods, we mostly use the default hyperparameters of the original paper.

Ablation study

To verify the effectiveness of each module in the model, we implemented several variants of DGHP-LISA and conduct the next item prediction task on three datasets. In the experiment, DGHP-LISA uses a attention aggregator with an aggregator size of 20. First of all, we use only the basic intensity in the node embedding update framework based on the dynamic graph Hawkes process, and then gradually add the excitation effect of historical events, the time embedding of neighbor nodes, and the LISA module to form DGHP-LISA. The specific process is described as follows:

• variant a (basic intensity): We only use basic intensity to model in the node embedding update framework based on dynamic graph Hawkes process.

• variant b (+excitation effect of historical events): We add the modeling of the influence of different historical interaction events and historical collaborative interaction events on the current events.

• variant c (+time embedding of neighbor nodes): We add time embedding to all historical interaction neighbors and historical collaborative interaction neighbors.

• variant d (+ LISA): LISA module is added to the time encoding of neighbor nodes and the feature fusion of node embeddings.

Table 3 presents that the key modules in DGHP-LISA are effective for the next item prediction task. Specifically, modeling the influence of historical events on current events in the node embedding update framework based on dynamic graph Hawkes process is helpful to capture the excitation effects of different historical events. We add time embedding to all historical interaction neighbors and historical collaborative interaction neighbors, which is helpful to capture the time decay effect of different historical events on current events. We add the LISA module to the time encoding of neighbor nodes, which effectively captures the time correlation of different historical events. At the same time, we add the LISA module to the feature fusion of node embeddings , which effectively models the importance between update mechanisms. In conclusion, the ablation experiment presents the effectiveness of each module in our model.

Impact of different aggregator functions

In order to study the impact of different aggregator functions on the performance of the model, we test the effectiveness of mean and attention aggregator functions respectively. In the experiment, the size of the aggregator is 20. It can be seen from Table 4 that the attention aggregator is better than the mean aggregator on three datasets. It is proved that the attention aggregator can update the user or item embedding by selecting the information which is more beneficial to the prediction task in multiple historical interaction neighbors or historical collaborative interaction neighbors. However, the mean aggregator has the same weight when aggregating neighbor information, which may ignore the most influential nodes.

Impact of different aggregator sizes

In order to verify the impact of different aggregator sizes on the model performance, we use attention aggregator to set different aggregator sizes to evaluate the model. In the experiment, DGHP-LISA uses a attention aggregator, and the aggregator size is set to 20, 40, 60 and 80, respectively. As shown in Table 5, as the size of the aggregator increases, the performance of the model on three datasets decreases at first and then increases. When the size of aggregator is 20, the performance of the algorithm is the best. This shows that the increase of aggregator size may bring a lot of neighbor information redundancy, which is not helpful to modeling. Therefore, the accuracy of the model and the speed of training can be improved by reducing the size of the aggregator.