Intent-aware knowledge graph-based model for electrical power material recommendation

Session-based recommendations

Session-based recommendation focus on predicting the next item for a user based on their past interactions, such as a series of purchases or clicks (Xu et al., 2019). Unlike traditional user-item recommendation frameworks, SR emphasizes sequential patterns and utilizes a limited set of user interactions within a brief timeframe, referred to as a session. Early research (Shani et al., 2005; Rendle, Freudenthaler & Schmidt-Thieme, 2010) commonly employed Markov decision processes (MDPs) to forecast a user’s subsequent actions based on their prior behaviors. For instance, Rendle, Freudenthaler & Schmidt-Thieme (2010) introduced a model that captures the sequential nature of consecutive clicks, improving the accuracy of predictions for each sequence. Nonetheless, a significant limitation of Markov chain-based models is their assumption that past elements are independent, which can hinder prediction accuracy.

In recent years, a considerable amount of research (Quadrana et al., 2017; Zhao et al., 2020) has focused on harnessing deep learning techniques, particularly through the development of various recurrent neural networks (RNNs) and attention-based models for session-based recommendations, which have shown promising results. For instance, Quadrana et al. (2017) proposed an innovative approach to personalize RNN models by transferring information across sessions and introduced a hierarchical RNN model that refines and propagates the latent hidden states of RNNs throughout user sessions. Additionally, Li et al. (2017) employed the embedding from the last-click to represent the user’s current interests and built an attention model on this foundation to effectively capture the user’s short-term intent. Recently, there has been an increasing interest in graph neural networks (GNNs), which effectively merge node information with topological structures, positioning themselves as an innovative approach within the field of recommender systems (Wei et al., 2022; Zhang et al., 2022). Several studies (Wu et al., 2019; Zhang et al., 2020; Zhu et al., 2023) have adapted GNNs for session-based recommendation tasks, improving performance by modeling each session as a graph. For instance, the SR-GNN model (Wu et al., 2019) employs gated graph neural networks (GGNN) to understand the intricate relationships between item transitions in session-based recommendations. Nonetheless, numerous methods struggle to effectively differentiate the impacts of various historical sessions on the current session. To tackle this problem, Zhang et al. (2020) proposed a personalized graph neural network that incorporates an attention mechanism to better represent how past sessions influence the current one. Despite these advancements, the GNN-based recommendation methods discussed often lack interpretability, making it difficult to assure users of the reliability of the recommended outcomes.

GNN-based recommendations

In recent years, there has been a rapid rise in the exploration of various GNN variants to tackle a wide range of challenges in graph-related tasks like graph embedding, node classification, and link prediction (Raza et al., 2024). These studies have demonstrated significant potential in handling complex graph data (Gao et al., 2021; Wu et al., 2020; Ma et al., 2022; Luo et al., 2022). Leveraging the ability of GNNs to process graph-structured data and extract structural information, a mass of GNN-based recommender systems has emerged to address diverse challenges across different recommendation scenarios (Wang et al., 2021b, 2021a). Motivated by the achievements of collaborative filtering (CF) models that utilize neural networks, researchers have delved into graph-based CF techniques (Berg, Kipf & Welling, 2018; Wang et al., 2019d; Hu et al., 2021). For instance, models like graph convolutional matrix completion (GCMC) (Berg, Kipf & Welling, 2018) and neural graph collaborative filtering (NGCF) (Wang et al., 2019d) apply graph convolution and embedding propagation mechanisms to enhance the representation of users and items. However, these methods often overlook latent relations between users and items. Addressing this limitation, Hu et al. (2021) propose a neural graph personalized ranking (NGPR) model to explicitly capture these connections by utilizing the user-item interaction graph along with nonlinear interaction modeling. Although these collaborative filtering methods for graph-structured data provide considerable advantages, they may not fully consider the dynamic evolution of user preferences and consumption motivations, particularly in sequential recommendation scenarios. Models like HOP-Rec (Yang et al., 2018) strive to enrich user-item interactions through random walks. The inherent structure of graphs naturally aligns with many data types in recommender systems, such as user-item interactions represented as bipartite graphs and user clickstreams modeled as session graphs (Ding et al., 2021).

To tackle the challenges mentioned above, several existing approaches (Wu et al., 2019; Qiu et al., 2019; Xu et al., 2019) have constructed sequential graphs that are generated by the item sequences within the same session. They then employed GNNs to capture the intricate transitions among items, which were deemed complex and challenging for traditional sequential methods (Ding et al., 2021). GGNN (Li et al., 2015), as a modification of GNNs, integrated GRUs to comprehensively capture internode relations by iteratively integrating influences from adjacent nodes within the graph. Subsequently, SR-GNN (Wu et al., 2019) employed GGNN to represent intricate item transition dynamics in session-based recommendation contexts. However, many of these approaches did not adequately distinguish the effects of different historical sessions on the current session. To tackle this challenge, a personalized graph neural network with an attention mechanism (A-PGNN) (Zhang et al., 2020) was introduced to address this issue by modeling the influence of historical sessions on the current session. Nevertheless, prior models typically focused on modeling item-to-item relations within each session, neglecting the opportunity to globally characterize relations across different sessions for enhanced representations of items. To overcome this limitation, Wang et al. (2021b) introduced the LP-MRGNN model, which successfully constructed and modeled a multi-relational item graph to extract valuable insights from various sessions and diverse types of user behavior. Motivated by the diverse GNN-based models mentioned above, this study presents a new GNN-based model designed to learn item representations from co-occurrence graphs based on four attributes for session-based recommendations. As far as we know, no previous research has aimed to concurrently model item connections within session graphs alongside item-item transitions in multi-view attribute co-occurrence graphs.

Intent-aware recommendations

Recently, researchers have strived to embed intent-awareness into recommendation systems in a more direct and unambiguous manner, as noted in the work by Jannach & Zanker (2024). Notably, the results from practical industrial applications underscore the substantial potential of customizing recommendations based on the anticipated intentions of users. For example, in the context of session-based recommendation, the MCPRN model, as presented by Wang et al. (2019b), introduced mixture-channel purpose routing networks. These networks were designed to adaptively learn the diverse purchase intentions of users for each item across different channels (sub-sequences). Meanwhile, Liu et al. (2020) put forward a multi-intent translation graph neural network, which aimed to uncover multiple user intents by taking into account the correlations among these intents. Additionally, Pan et al. (2020) devised an intent-guided neighbor detector in their ICM-SR model, which was used to retrieve the appropriate neighbor sessions for the representation of neighbors. Distinct from session-based recommendation approaches, another stream of research efforts is centered on modeling the sequential changes in users’ interaction behaviors over a more extended time period. Ma et al. (2020) proposed the DSSRec model, which featured a seq2seq training strategy. This strategy employed multiple future interactions as a form of supervision and incorporated an intent variable derived from a user’s historical and future behavior sequences. The intent variable served to capture the mutual information between an individual user’s past and future behavior sequences. It’s worth noting that two users with similar intents might have representations that are quite distant in the representation space. In contrast to this approach, in our work, the intent variable is learned from all users’ sequences and is utilized to maximize the mutual information among different users who have similar learned intents. Moreover, ASLI (Tanjim et al., 2020) utilized a temporal convolutional network along with side information (such as user action types like click, add-to-favorite, etc.,) to capture user intent. Subsequently, the learned intents were employed to guide the session recommendation (SR) model in predicting the next item.

Unlike the aforementioned studies, our proposed IKG-EMR models user embedding from both two perspectives by utilizing knowledge graphs and user behavior sequences, respectively, i.e., user preference and user intent. In addition, IKG-EMR designs a GNN to produce user intent and item embeddings from the tripartite graph of “User-Item-Topic”, which can alleviate the problem of sparsity faced by recommendation systems.

Overview

To implement the task of electrical power material recommendation, this article proposes a intent-aware knowledge graph-based mode, named IKG-EMR. As shown in Fig. 3, the input of IKG-EMR contain the tripartite graph of “User-Item-Topic” and the behavior sequence of user. IKG-EMR mainly consists of four modules: user intent embedding and item embedding, user preference embedding, adaptive fusion network, and making recommendation. Specifically, we first use a graph neural network to generate user intent embedding and item embedding from the tripartite graph of “User-Item-Topic”, and a multi-head Transformer to generate user preference embedding, respectively (“Generating User and Item Embedding”). Then, a cohesive representation of users can be achieved by implementing an adaptive gated fusion method, which integrates the intent and preference embeddings of the users (see “Adaptive Fusion Approach”). For recommendation purposes, IKG-EMR computes the purchase probabilities for each item based on the representations of the specified target user and items through interaction modeling. The items with the highest top- $k$ purchase probabilities will constitute the user’s recommendation list (refer to “Making recommendation and model training”).

Generating user and item embedding

In this subsection, we detail how to model user’s embedding from tripartite graph of “User-Item-Topic” and behavioral sequence of users, respectively.

Adaptive fusion approach

Both short-term and long-term components have strengths and weaknesses. It is necessary to accommodate these two components. Instead of using a naive way to combine them, e.g., ${\mathbf{r}}_u={\mathbf{r}}_u^I+{\mathbf{r}}_u^P$, we devise an adaptive fusion network to evaluate the relevance of user intent and preferences, and aggregate information based on that assessment. Initially, we embed the identifiers (ID) of user $u$ into a representation vector, which serves as the user’s profile vector ${\mathbf{r}}_u^Q$. Given the embedding vectors of user intent ${\mathbf{r}}_u^I$, user preference ${\mathbf{r}}_u^P$, and user’s profile vector ${\mathbf{r}}_u^q$ as inputs, the gate vector ${\mathbf{F}}_u$ is designed to regulate the influence of long-term and short-term preferences, as defined in Eq. (15).

(15) $${\mathit{F}}_u=sigmoid(W_Q{\mathit{r}}_u^Q+W_I{\mathit{r}}_u^I+W_P{\mathit{r}}_u^P+b_u),$$where $W_q$, $W_s$, $W_l$, $b_u$ are projection parameters.

Finally, the output of preference vector ${\mathit{r}}_u$ of user $u$ can be calculated by Eq. (16).

(16) $${\mathit{r}}_u=(1-{\mathit{F}}_u)\odot {\mathit{r}}_u^I+{\mathit{F}}_u\odot {\mathit{r}}_u^P,$$where $\odot$ denotes element-wise multiplication.

Making recommendation and model training

The “user-item” interaction modeling layer aims to model the preference level between users and items. Specifically, in the recommendation model framework of IKG-EMR, the interaction score between user $u$’s representation vector $r_u$ and item $i$’s representation vector $r_i$ to predict user $u$’s interaction score with item $i$ as defined in Eq. (18).

(17) $${\mathit{z}}_{ui}=\mathrm{\Sigma }({\mathit{r}}_u^T{\mathit{r}}_i),$$where $\mathrm{\Sigma }(\cdot )$ is the activation function. Here, we also choose ReLU as the activation function. The final output of the “user-item” interaction modeling layer is the interaction score of user $u$ with item $i$, denoted as $z_{ui}$.

Given the interaction score $z_{ui}$ between user $u$ and item $i$, we use the sigmoid function to obtain the model’s output (the probability of user $u$ purchasing item $i$), as given in Eq. (18).

(18) $$\hat{y}(u,i)=sigmoid({\mathit{z}}_{ui}),$$where $\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}(\cdot )=\frac{1}{1+e^{-(\cdot )}}$.

During the model training phase, in terms of recommending items to users, the positive labels are the set of items $i$ that user $u$ actually purchased (interaction exists), denoted as $Y^{+}$. The negative labels are formed by log-uniform sampling from the item set I excluding the positive labels (interaction does not exist), denoted as $Y^{-}$. We use binary cross-entropy loss, widely used in recommendation systems, as the loss function for IKM-EMR, as defined in Eq. (19).

(19) $$\mathcal{L}(\hat{\mathit{y}})=-\sum _{(u,i)\in Y^{+}\cup Y^{-}}(y_{u,i}\log ({\hat{y}}_{u,i})+(1-y_{u,i})log(1-{\hat{y}}_{u,i}))+\lambda ||\mathrm{\Theta }|{|}_2^2,$$where $\mathrm{\Theta }$ denotes all the learnable parameters in this model, $||\mathrm{\Theta }|{|}_2^2$ represents L2-regularizer, and $\lambda$ is the regularization coefficient. $y_{u,i}$ represents the probability distribution of user $u$ actually purchasing item $i$. Specifically, if $(u,i)\in Y^{+}$, then $y_{u,i}=1$; otherwise, $y_{u,i}=0$.

We utilize the Adam optimizer (Zhu et al., 2023) to minimize the loss function $\mathcal{L}(\hat{y})$ to optimize the parameters in the IKM-EMR model. Compared to existing training schemes, this approach efficiently extracts valuable insights from negative samples, thereby lowering the computational expenses associated with model training.

Experimental setup

Datasets

We evaluate the proposed model using a real-world electric power materials dataset provided by Jiangsu Electric Power Materials (http://www.js.sgcc.com.cn), one of China’s largest provincial power grid companies. The time interval of this dataset spans from March 18, 2021, to May 9, 2023. The statistics of the preprocessed dataset is summarized in Table 2, and the dataset is available at https://github.com/zgx881205/electric-dataset.

Table 2:

Statistics of the electric power materials dataset.

Dataset	#Users	#Items	#Interactions
Electric	80	2,195	16,598

DOI: 10.7717/peerj-cs.3023/table-2

Note:

“#” represents the quantity of someone object.

In practice, knowledge graph of real-world electric power materials dataset faces sparsity and noise, including long-tail entity distributions and irrelevant item-entity links, which limit KGR effectiveness. Using this real-world electrical dataset as an illustration, as depicted in Fig. 4, most entities in knowledge graphs display a long-tail distribution. Accurately modeling semantic transitions in the knowledge graph using Trans-based method necessitates a sufficient number of triplets associated with each entity, posing challenges for precisely capturing relationships between items.

Overall performance comparison

Table 3 presents a comprehensive comparison of the overall performance between our IKG-EMR approach and the baseline methods. The table is divided into three sections: the first part includes traditional methods, i.e., User-KNN and SVD; the second part comprises neural-based collaborative filtering and matrix factorization methods, i.e., CDL and DeepFM; and the third part showcases GNN-based models, i.e., GCMC, SR-GNN and KGCN. Overall, IKG-EMR consistently outperforms all baselines across the three datasets, as indicated by all evaluation metrics (Recall@ $k$, Coverage@ $k$ and NDCG@ $k$). More specifically, several key observations can be made:

• The neural-based collaborative filtering (CF) and matrix factorization (MF) techniques, such as CDL and DeepFM, demonstrate a marked improvement in performance compared to earlier traditional methods like User-KNN and SVD. One potential reason for this is that neural-based approaches are capable of learning more effective representations of users and items than their traditional counterparts.

• GNN-based methods, such as SR-GNN, which treats behavior sequences as a directed graph-structured data, outperform GAE, which utilizes a bipartite interaction graph (e.g., GCMC), as well as the neural-based CF and MF techniques, e.g., CDL and DeepFM. This suggests that the design of gated GNNs is particularly well-suited for session-based recommendation scenarios. Consequently, we also employ a multi-head Transformer to model user preferences derived from behavioral sequences.

• IKG-EMR consistently outperforms leading personalized GNNs, including SR-GNN, in the context of session-based recommendation tasks, such as SR-GNN, as well as KG-based recommendation systems. This enhancement can be attributed to two main factors: (1) leveraging the strengths of GNNs, IKG-EMR is capable of extracting high-quality representations of user intent; (2) ; and (3) the adaptive fusion mechanism combined with an attention network in IKG-EMR effectively integrates the correlational embedding information of users’ intent and preferences.

• IKG-EMR consistently outperforms leading personalized GNNs, including SR-GNN, in the context of session-based recommendation tasks, such as SR-GNN, as well as KG-based recommendation systems. This enhancement can be attributed to three main factors: (1) leveraging the strengths of GNNs, IKG-EMR is capable of extracting high-quality representations of user intent; (2) IKG-EMR utilizes a Transformer-based architecture to capture sequential dependencies in user behavior sequences; and (3) the adaptive fusion mechanism combined with an attention network in IKG-EMR effectively integrates the correlational embedding information of users’ intent and preferences.

Ablation study

Here, we conduct ablation studies to assess the effectiveness of the key components within IKG-EMR by creating three variants of the model: IKG-EMR (NoP), IKG-EMR (NoI) and IKG-EMR (NoA). Specifically, IKG-EMR (NoP) excludes the user preference encoder module, meaning it does not take into account the informativeness of user preferences while learning their representations. IKG-EMR (NoI) omits the user intent encoder from the User Intent and Item Encoder module, thereby disregarding the embedding of user intent in the representation learning process. Finally, IKG-EMR (NoA) removes the adaptive fusion mechanism with the attention network, resulting in a simple combination of the embedding vectors of user intent and preferences without recognizing the differing significance of these two types of embeddings. The results of these ablation studies are listed in Table 4.

First, we compare IKG-EMR with its three variants: IKG-EMR (NoP), IKG-EMR (NoI), and IKG-EMR (NoA). As observed, IKG-EMR exhibits significantly better performance than these three variants, indicating that the user intent encoder, user preference encoder, and the adaptive fusion mechanism with the attention network are critical for the task of electrical power recommendation.

Second, the notable performance decline of IKG-EMR (NoP) demonstrates that modeling user preferences is the most important factor in electrical power recommendation. This is primarily because incorporating user preferences serves as a fine-grained encoder, enabling the recommendation of more items that users actually like, in contrast to the coarse-grained approach provided by the user intent encoder.

Hyper-parameter analysis

Here, Recall@20, Coverage@20 and NDCG@20 are selected as evaluation metrics, and then we further examine how key parameters, including the number of GNN layers and the message dropout rate, affect IKG-EMR.

Effect of number of GNN layers

To demonstrate the specific effects of GNNs in IKG-EMR, we explore how varying the number of GNN layers ( $d_g$) used for graph representation propagation impacts performance. Figure 5 illustrates the results for GNN layer values ( $d_g$) ranging from 1 to 3. Notably, both Recall@20 and NDCG@20 show marked improvement when the GNN depth increases from 1 to 2. A 2-layer GNN achieves the best performance, while the performance of the 1-layer GNN is the lowest. This discrepancy in performance is due to the 1-layer GNN’s inability to capture higher-order relationships. Generally, deeper GNNs are anticipated to gather more extensive information from both long-term and short-term behavioral graphs. However, when $d_g$ exceeds 3, the node representations can become less distinguishable, which increases noise within the model. As a result, this can impede further improvements in recommendation performance.

Effect of dropout

To mitigate overfitting in IKG-EMR and investigate the impact of different dropout rates, the technique of message dropout is utilized here. Figure 6 illustrates the Recall@20 and NDCG@20 results, demonstrating how the message dropout ratio affects performance across two datasets. As indicated, consistent with prior research (Berg, Kipf & Welling, 2018), messages are independently dropped out, which enhances the robustness of the embeddings against the variability of individual edges. It is evident that an appropriately chosen dropout ratio allows IKG-EMR to achieve commendable results, while settings that are too low or too high can readily result in underfitting or overfitting issues.

Effect of learning rate

To investigate how the learning rate affects the performance of IKG-EMR, we tune the learning rate amongst [ ${10}^{-1}$, ${10}^{-2}$, ${10}^{-3}$, ${10}^{-4}$, ${10}^{-5}$]. The corresponding results are illustrated in Fig. 7. From Fig. 7, we can clearly observe that the performance of IKG-EMR is sensitive to the set of learning rate. The optimal options of learning rate for IKG-EMR is ${{10}^{-}}^3$.

Effect of batch size

To investigate how the number of batch size affects the performance of IKG-EMR, we tune the batch size amongst [16, 32, 64, 128, 256]. The corresponding results are illustrated in Fig. 8. It is obvious that when batch size is set to 64, IKG-EMR can achieve the best performance.

Effect of transformer heads

We investigate how the number of transformer heads affects the performance of IKG-EMR. We vary different numbers of transformer heads from 2 to 12 with a step size 2 in the experiment. The corresponding results are illustrated in Fig. 9. As can be seen, the optimal options of the number of transformer heads for IKG-EMR is 8. As the number of heads increases from 8 to 10, the performances descend due to over-parameterization.