A temporal knowledge graph reasoning model based on recurrent encoding and contrastive learning

Temporal knowledge graph reasoning

TKG reasoning is segmented into interpolation reasoning and extrapolation reasoning based on different timestamps. Interpolation reasoning refers to inferring facts that are missing in historical timestamp. For example, Temporal TransE (TTransE) (Leblay & Chekol, 2018) incorporates timestamp vectors to represent temporal transformations between entities. Hyperplane-based Temporally aware knowledge graph Embedding (HyTE) introduced by Dasgupta, Ray & Talukdar (2018) represents time as a hyperplane projects entities and relationships onto it to encode temporal information. TNTComplEx (Lacroix, Obozinski & Usunier, 2020) employed tensor factorization, to model events with added temporal information as fourth-order tensors. However, these methods have poor ability to capture the evolution of facts and are not suitable for predicting entities in future timestamps, thus prompting research into extrapolation reasoning. The objective of extrapolation reasoning is to forecast facts in future timestamps. Know-Evolve by Trivedi et al. (2017) models events using time point processes and predicts future occurrences based on the conditional probabilities. RE-NET (Jin et al., 2019) uses neighborhood aggregators and recurrent event encoders to encode historical facts. RE-NET uses graph structure information but this model only considers local structures. xERTE (Han et al., 2020) establishes an interpretable prediction model based on subgraph search. CyGNet (Zhu et al., 2021) applies repeated facts in historical sets to predict high-frequency entities. TANGO (Han et al., 2021) models the temporal knowledge graph using neural ordinary differential equations with continuous-time reasoning capabilities. TIme Traveler (TITer) by Sun et al. (2021) utilizes reinforcement learning techniques to predict target entities based on path search. Similar to TITer, Clue Searching and Temporal Reasoning (CluSTeR) (Li et al., 2021a) also uses reinforcement learning to search for possible target entities. RE-GCN (Li et al., 2021b) and its extended CEN (Li et al., 2022a) model the evolution of entities and relationships at each timestamp to obtain local historical dependencies, and they introduce static attributes to improve prediction results. However, they do not fully utilize long-term information. EvoExplor (Zhang et al., 2022a) achieves entity prediction by capturing complex evolutionary theories and community structures of historical facts. Graph Hawkes Transformer (GHT) (Sun et al., 2022) proposes a Transformer based point-to-point process model for capturing structural and temporal information. TLogic (Liu et al., 2022) and TLmod (Bai et al., 2023) use temporal logic regulations collected from temporal knowledge graphs to predict entities. HiSMatch (Li et al., 2022b) incorporated two structural encoders to compute representations of query-related history and candidate-related history, and integrates background knowledge into entity representations. TEemporal logiCal grapH networkS (TECHS) (Lin et al., 2023) introduces a temporal graph encoder and a logical decoder for TKG reasoning, and puts forward a forward message-passing mechanism. The Pre-trained Language Model with Prompts for temporal Knowledge graph completion (PPT) (Xu et al., 2023a) transforms the quadruple into input for the pre-trained model and converts the time intervals between different timestamps into prompts to constituting coherent sentences with semantic information. Lee et al. (2023) proposes a model to use in-context learning with large language models for TKG reasoning. The proposed model transformed relevant historical facts into prompts and earned prediction results through token probabilities. GenTKG (Liao et al., 2024) capitalizes on a retrieval tactic on the ground of temporal logic rules and valid fine-tuning with few-shot parameters for TKG reasoning. TiRGN (Li, Sun & Zhao, 2022) considers the sequential, reduplicative and periodic patterns of historical facts. TiRGN fully recognizes local and global history information. But TiRGN ignores the interference of repeated history on entity prediction. CENET (Xu et al., 2023b) utilizes the frequency of historical events and contrastive learning to obtain correlations between historical and unhistorical events in order to predict matching entities. However CENET does not consider the evolution of historical facts, resulting in poor performance.

Contrastive learning

Contrastive learning has been broadly utilized across several territories, such as computer vision (Feng et al., 2023; Lin et al., 2023; Wu et al., 2023b), muti-modal learning (Xie et al., 2023), audio processing (Hu et al., 2023) and natural language processing (Rethmeier & Augenstein, 2023; Zhao et al., 2023). Recently, contrastive learning has been applied in static knowledge graph reasoning. SimKGC model (Wang et al., 2022) introduced three types of negative samples for improving the efficiency of contrastive learning. SimRE (Zhang et al., 2024) employed comparative learning to mimic the head and body of rules, and incorporates rule features into the model through simple addition. However contrastive learning has been less applied in TKG reasoning. The CENET model is an example of using contrastive learning for TKG reasoning. This model utilizes contrastive learning to recognize historical and non-historical facts, reducing the interference of historical facts on entity prediction.

Relational graph convolutional network

To adapt GCNs for relational data, the relational graph convolutional network (R-GCN) was developed. R-GCN effectively aggregate local neighborhood information with effect. It is a model that can be used for message passing.

R-GCN was initially applied to static knowledge graph reasoning to predict missing information in the knowledge graph. The node features of each layer in R-GCN are calculated from the node features of the previous layer and the relationships between nodes, and so as to hold on to the information of the nodes themselves, R-GCN also incorporates self-connection.

(1) $$h_i^{\left(l+1\right)}=\sigma \left(\sum _{rϵ\mathcal{R}}\sum _{j\in N_i^r}{\displaystyle \frac{1}{c_{i,r}}{W}_r^{\left(l\right)}{h}_j^{\left(l\right)}+W_0^{\left(l\right)}{h}_i^{\left(l\right)}}\right)$$where ${\mathrm{N}}_{\mathrm{i}}^{\mathrm{r}}$ denotes neighboring nodes with r-type edges connected to node $\mathrm{i}$. ${\mathrm{c}}_{\mathrm{i},\mathrm{r}}$ is a constant and it is broadly used to represent the in degree of node $\mathrm{i}$. ${\mathrm{W}}_{\mathrm{r}}^{\left(\mathrm{l}\right)}$ and ${\mathrm{W}}_0^{\left(\mathrm{l}\right)}$ are learnable parameters. $\mathrm{l}$ denotes the number of layers. ${\mathrm{h}}_{\mathrm{i}}^{\left(\mathrm{l}\right)}$ denotes the embedding representation of node $\mathrm{i}$ in layer $\mathrm{l}$. ${\mathrm{W}}_0^{\left(\mathrm{l}\right)}{\mathrm{h}}_{\mathrm{i}}^{\left(\mathrm{l}\right)}$ denotes self-connection.

Contrastive learning

The learning framework of contrastive learning is based on discriminative representation of contrastive thinking, primarily used for unsupervised or self-supervised representation learning. The notion of contrastive learning is to compare a given sample with similar positive samples and dissimilar negative samples (Chen et al., 2020). The use of contrastive learning model and loss function, representations corresponding to positive samples are closer in the representation space, whereas representations corresponding to negative samples are further away in the representation space. In contrastive learning, a small batch of randomly selected N samples is used to obtain enhanced samples. Given a pair of positive samples (i, j), optimize the following loss function using the original and augmented samples.

(2) $${\mathcal{L}}_{i,j}=-log{\displaystyle \frac{exp\left({\mathit{z}}_i\cdot {\mathit{z}}_j/\tau \right)}{\sum _{k=1,k\ne i}^{2N}exp\left({\mathit{z}}_i\cdot {\mathit{z}}_k/\tau \right)}}$$where $2\mathrm{N}$ is the sum of the number of the original and augmented samples. ${\mathrm{z}}_{\mathrm{i}}$ is the projection embedding of the sample $\mathrm{i}$ and $\mathrm{\tau }$ is a temperature parameter facilitating the model learn from hard negatives, and $\cdot$ is dot product that is utilized for calculating the similarity of samples between different views.

Temporal knowledge graph preliminaries

A TKG $\mathcal{G}$ is formalized as a sequence of KG snapshots, i.e., $\mathcal{G}=\left\{{\mathcal{G}}_1,{\mathcal{G}}_2,...,{\mathcal{G}}_{\mathrm{T}}\right\}$. Each snapshot ${\mathcal{G}}_{\mathrm{t}}=\left(\mathcal{E},\mathcal{R},{\mathcal{F}}_t\right)$ can be seen as a directed multi-relational graph at timestamp t. A fact in ${\mathcal{F}}_t$ is denoted as a quadruple (s, r, o, t), where $\mathrm{s},\mathrm{o}\mathrm{ϵ}\mathcal{E}$ and $\mathrm{r}\mathrm{ϵ}\mathcal{R}$. It indicates that there is a relationship r between s and o at time t.

The intention of TKG extrapolation is to forecast missing subject entity or object entity in given query (s, r, ?, t) or (?, r, o, t) according to previous historical KG snapshot $\left\{{\mathcal{G}}_1,{\mathcal{G}}_2,...,{\mathcal{G}}_{\mathrm{t}-1}\right\}$. For simplicity, this article adds the quadruple of the inverse relation (o, ${\mathrm{r}}^{-1}$, s, t) in TKG. Therefore the extrapolation task is simplified to predict the object entity. For each prediction task (s, r, ?, t) at time t, We employ the TKG snapshot sequence of m timestamps before time t as ${\mathcal{G}}_{\mathrm{t}-\mathrm{m}:\mathrm{t}-1}$.

Model overview of the proposed TRLC model

The TRLC model proposed in this article consists of four modules. The first module takes the sequential information of k snapshots before the current timestamp t as input to the recurrent encoder, obtaining the embedding of entities and relations. The second module is to establish a global historical matrix, with the aim of integrating all repeated historical information into knowledge graph reasoning. The third module is to use the loss function of contrastive learning to diminish the effect of irrelevant historical information on the model’s prediction of future events. The fourth module uses a time decoder to capture periodic information, and ultimately this module can obtain the predicted results for each entity. The entire framework of the proposed TRLC model is shown in Fig. 2.

Recurrent encoder

Many models, such as RE-GCN (Li et al., 2021b) CEN (Li et al., 2022a), and TiRGN (Li, Sun & Zhao, 2022), have certified the excellent performance of the R-GCN model in obtaining representations of entities and relations. Unlike RE-GCN, which simply adds representations of entities and relations, TiRGN uses one-dimensional convolution to better combine the representations of entities and relations. We apply the one-dimensional convolutional R-GCN introduced by TiRGN to encode entities and relationships at each timestamp. The specific aggregation formula is as follows:

(3) $$h_{o,t}^{l+1}=f\left({\displaystyle \frac{1}{c_0}\sum _{\left(s,r,oϵ{\mathcal{F}}_t\right)}{w}_1^l\left(\phi \left(h_{s,t}^l,r_t\right)\right)+w_2^l{h}_{o,t}^l}\right)$$where $h_{s,t}^l$, $h_{o,t}^l$ and $r_t$ denote the embedding representation of entities s, o, and relation r in the ${\mathrm{l}}^{\mathrm{t}\mathrm{h}}$ layer at time t. $w_1^l$ and $w_2^l$ are learnable parameters for aggregating features and self-loop in the ${\mathrm{l}}^{\mathrm{t}\mathrm{h}}$ layer. $f(\cdot )$ means the Relu activation function. $\phi (\cdot )$ denotes the one-dimensional convolution operator. ${\mathrm{c}}_0$ represents the in-degree of entity o, it is a normalizing constant. It is worth noting that when an entity has no relation with other entities in a snapshot, a self-loop edge is still needed to update the representation of that entity.

In order to insure that each query can contain the sequential dependencies of snapshots at the previous timestamp, this article employs the gated recurrent unit (GRU) to gradually renew the representations of entities and relations for each query, ultimately obtaining entity and relation representations that incorporate temporal information from different snapshots. The formula for updating the embeddings of entities in sequence of the snapshots is as follows:

(4) $$H_t=\mathrm{G}\mathrm{R}\mathrm{U}\left({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{H}}_{\mathrm{t}-1}^{\mathrm{R}\mathrm{G}\mathrm{C}\mathrm{N}}\right)$$where $H_t$, $H_{t-1}ϵ{\mathbb{R}}^{\left|\epsilon \right|\times d}$ are the d-dimensional entity embedding matrices at timestamp t and $\mathrm{t}-1$, $H_{t-1}^{RGCN}ϵ{\mathbb{R}}^{\left|\epsilon \right|\times d}$ is the entity embedding matrix after encoding on the KG snapshots at $\mathrm{t}-1$. ${\mathrm{H}}_{\mathrm{t}}$ is obtained by calculating ${\mathrm{H}}_{\mathrm{t}-1}$ and ${\mathrm{H}}_{\mathrm{t}-1}^{\mathrm{R}\mathrm{G}\mathrm{C}\mathrm{N}}$ using the GRU model.

Resemble the formula for updating the embeddings of entities, the formula for updating the embeddings of relations in sequence of the snapshots is as follows:

(5) $$r_t^{\prime }=\left[\mathrm{p}\mathrm{o}\mathrm{o}\mathrm{l}\mathrm{i}\mathrm{n}\mathrm{g}\left({\mathrm{H}}_{\mathrm{t}-1},{\mathrm{V}}_{\mathrm{r},\mathrm{t}}\right);\mathrm{r}\right]$$

(6) $$R_t=GRU\left(R_{t-1},R_t^{\prime }\right)$$where $V_{r,t}$ represents the embedding matrices of entities correlated with relationship r at t. $R_t^{\prime }$ is composed of all relations $r_t^{\prime }$. $R_t$, $R_{t-1}ϵ{\mathbb{R}}^{\left|\mathcal{R}\right|\times d}$denote the embedding matrices of relations at timestamp t and $\mathrm{t}-1$. $R_t$ is gained by calculating $R_{t-1}$ and $R_t^{\prime }$ using the GRU model.

Historical matrix

Considering that certain historical events occur repeatedly, this article establish a historical matrix of entity sets to record whether the current query has appeared in history, in order to provide historical constraints for contrastive learning and time decoder. Specifically, for each query (s, r, ?, t), traverse all snapshots ${\mathcal{G}}_{0:\mathrm{t}-1}$ before t to obtain the corresponding historical entity matrix ${\mathrm{M}}_{\mathrm{s},\mathrm{r},\mathrm{t}}\mathrm{ϵ}{\mathbb{R}}^{\left|\mathcal{E}\right|\times \left|\mathcal{R}\right|\times \left|\mathcal{E}\right|}$. If query (s, r, ?, t) has appeared before t, the value of the corresponding position in the matrix is 1, or else it is 0. We disregard the frequency of quadruples occurrence here, focusing solely on the presence or absence of historical events. This is because events that have repeated numerous times in the past may not necessarily recure in the future. For example, Michael Jordan led the Chicago Bulls to win 6 championships, and when Jordan withdrew from the Chicago Bulls, their chances of winning the NBA championship decreased.

Contrastive learning

The CENET model as proposed in Xu et al. (2023b) uses statistical frequency of historical events and contrastive learning to achieve knowledge graph reasoning. Through contrastive learning, it distinguishes whether the entity to be predicted by the query is a historical entity or a unhistorical entity. This work draws inspiration from the CENET model. Firstly, there are two types of queries. The first type is the entity to be predicted by the query is a historical entity while the second type is the query is predicting non-historical entities. By minimizing the loss function of contrastive learning to distinguish different types of queries, the same type of query representation is more similar, while different types of query representations are more different. Therefore, minimizing the loss function of contrastive learning can avoid interfering with the prediction results of queries due to irrelevant historical events. Note that the proposed TRCL model differs from the CENET model whereby the module only considers the representation of entities and relations, without considering the frequency of historical events.

The perspective in distinguishing queries is whether the lost entities belong to historical or non-historical entities for each query. This work uses contrastive learning to separate the representations of each query as much as possible. Assume that the embedding representation of q is given:

(7) $${\mathit{v}}_q=MLP\left(\mathrm{h}\oplus \mathrm{r}\right).$$

The queried information is encoded by $\mathrm{M}\mathrm{L}\mathrm{P}$, where $\mathrm{h}$ denotes the embedding of entities and r denotes the embedding of relations. The size of the training batches used in this article is divided by timestamps, with each timestamp corresponding to quadruples being a batch. The embedding of entities and relationships stems from the timestamp t corresponding to the current query, so the embedding representation of the query does not need to include time information.

In addition, let $I_q$ represent whether the missing object in query q has appeared in history. If it has appeared, $I_q$ is 1, otherwise $I_q$ is 0. In detail, assuming the predicted result of query q is o, if o has appeared before timestamp t, then $I_q=1$; otherwise, $I_q=0$.

(8) $$\mathrm{Q}\left(\mathrm{q}\right)=\underset{\mathrm{m}\mathrm{ϵ}\mathrm{M}\mathrm{\backslash }\mathrm{q}}{\cup }\{\mathrm{m}|{\mathrm{I}}_{\mathrm{m}}={\mathrm{I}}_{\mathrm{q}}\}$$

At last, the loss function for contrastive learning is as follows:

(9) $${\mathcal{L}}_{cl}=\sum _{q\in M}{\displaystyle \frac{-1}{\left|Q\left(q\right)\right|}\sum _{k\in Q\left(q\right)}log{\displaystyle \frac{exp\left(v_q\cdot v_k/\tau \right)}{{\sum }_{aϵM\mathrm{\backslash }\left\{q\right\}}\left(v_q\cdot v_a/\tau \right)}}}$$where $\mathrm{M}$ denotes the size of the batch. $\mathrm{\tau }$ denotes the temperature parameter. Referring to the CENET model, we set it to 1. $\mathrm{Q}\left(\mathrm{q}\right)$ represents the query m corresponding to the value of ${\mathrm{I}}_{\mathrm{q}}$ in the set $\mathrm{M}$, except for query q. The aim of ${\mathcal{L}}_{\mathrm{c}\mathrm{l}}$ is to make the query representations of the same variety closer.

Time decoder

Training objective

Because different queries can be duplicate facts or newly occurring facts, we have set a hyper-parameter to equilibrate the values of ${\mathrm{p}}^h$ and ${\mathrm{p}}^{\mathrm{n}h}$. The formula is as described below:

(16) $$\mathrm{p}=\mathrm{\alpha }\times {\mathrm{p}}^{\mathrm{h}}+\left(1-\mathrm{\alpha }\right)\times {\mathrm{p}}^{\mathrm{n}\mathrm{h}}$$where hyper-parameters $\mathrm{\alpha }\in \left[0,1\right]$.

The loss function ${\mathcal{L}}_{\mathrm{t}\mathrm{d}}$ based on time decoder for entity prediction is formalized as follows: where $\mathrm{p}(\mathrm{o}|\mathrm{s},\mathrm{r},\mathrm{t})$ is the entity prediction probability calculated by Eq. (16), ${\mathrm{y}}_{\mathrm{t}}^{\mathrm{e}}$ is the label vector, which is 1 if the corresponding fact exists, or else it is 0.

The ultimate loss function $\mathcal{L}$ is as described below:

(17) $$\mathcal{L}={\mathcal{L}}_{td}+{\mathcal{L}}_{cl}$$

Note that the loss function ${\mathcal{L}}_{\mathrm{c}\mathrm{l}}$ of contrastive learning and the loss function ${\mathcal{L}}_{\mathrm{t}\mathrm{d}}$ of entity prediction are trained simultaneously.

Experimental setup

This article uses four datasets to appraise the TRCL models on entity forecast task, including ICEWS14, ICEWS18, ICEWS05-15 (Lautenschlager, Shellman & Ward, 2015) and GDELT (Leetaru & Schrodt, 2013). These four datasets are widely used for TKG extrapolation. ICEWS14, ICEWS18, and ICEWS05-15 are three subsets of Integrated Crisis Early Warning System (ICEWS). ICEWS contains a large number of political events with timestamps. GDELT is a sub-class of the Global Database of Events, Language datasets, which is also an event set containing temporal information.

Similar to the partitioning methods used in literature REGCN (Li et al., 2021b) and Re-Net (Jin et al., 2019), we divided all datasets into training, validation, and testing sets in a rate of 0.8, 01, and 0.1. The statistical information of the four datasets is shown in Table 1.

This work applied two evaluation metrics that are extensively used in knowledge graph reasoning, namely mean reciprocal rank (MRR) and H@k (k = 1, 3, 10). MRR represents the average reciprocal values of the ranks of the true entity for all queries, and Hits@k denotes the scale of times that the true entity occur in the top k of the ranked candidates.

Several researches (Han et al., 2021; Sun et al., 2021) points out that the traditional static filtering setting previously used is not appropriate for extrapolation on temporal knowledge graph reasoning, as only simultaneous facts need to be filtered. Therefore, this work presents the results of the experiment using the recently widely used time aware filtering setting, which only separates out quadruples that occur during query time.

For all datasets, the embedding size d is selected as 200. The amount of one-dimensional convolution-based GCN layers is selected as 2 and the dropout rate for each layer is selected as 0.2. The parameters of TRCL are optimized by using adam (Kingma & Ba, 2014) during training, and the learning rate is selected as 0.001. The batch size is selected as the amount of quadruples in each timestamp. During training, the optimal local historical KG snapshot sequence lengths for Integrated Crisis Early Warning System (ICEWS) 14, ICEWS05-15, ICEWS18, and Global Database of Events, Language, and Tone (GDELT) are selected as 12, 19, 17 and 10, respectively. During testing, the optimal local historical KG snapshot sequence lengths for ICEWS14, ICEWS05-15, ICEWS18 and GDELT are selected as 13, 21, 20 and 11, respectively. Like RE-GCN and TiRGN, this article added static KG information to datasets ICEWS14, ICEWS05-15 and ICEWS18. For time-guided decoders, the amount of channels is selected as 50 and the kernel size is selected as 4 × 3. This article attempted numerous α values from 0 to 1 and ultimately determined α = 0.3 as the history weight for all the datasets.

Results

The proposed TRCL is compared to eight up-to-date TKG reasoning methods, all of which are typical extrapolated temporal knowledge graph reasoning. These methods include RGCRN (Seo et al., 2018), RE-NET (Jin et al., 2019), CyGNet (Zhu et al., 2021), xERTE (Han et al., 2020), TITer (Sun et al., 2021), RE-GCN (Li et al., 2021b), CEN (Li et al., 2022a), TiRGN (Li, Sun & Zhao, 2022) and CENET (Xu et al., 2023b). The experimental results of the TRCL model and nine baselines on four benchmark datasets are shown in Tables 2 and 3.

Discussions

According to experimental results, it can be concluded that the proposed TRCL model consistently outperforms RGCRN, RE-NET, RE-GCN, CEN, CyGNet, xERTE, and TITer models. Specifically, although the RGCRN, RE-NET, xERTE, RE-GCN, and CEN models consider adjacent timestamps, the RGCN and GRU models with one-dimensional convolution in the TRCL enhances its ability to recognize structural features and historical correlations more effectively. Therefore, the TRCL model outperforms the RGCRN, RE-NET, RE-GCN, and CEN models. Similar to CyGNet, TRCL also considers repeated historical facts. TRCL’s superior performance over CyGNet stems from its comprehensive consideration of the dependency relationships among historical facts and temporal periodicity of facts. Furthermore, the TRCL model outperforms the xERTE and TITer models due to their reliance on subgraph based search and path based search to predict target entities respectively. These methods often face limitations in utilizing long-term information, restricting their ability to recognize complex temporal dependencies. In contrast, TRCL’s approach takes into consideration a more nuanced understanding of temporal relationships, resulting in enhanced predictive accuracy.

On the other hand, when compared with the TiRGN model, except for the H@1 metric in dataset ICEWS05-15, the experimental results of the proposed TRCL model attained the best results, indicating that incorporating contrastive learning is profitable for enhancing the performance of our model. Through in-depth analysis of the experimental results of models TRCL and TiRGN, we found that the experimental effect is the best in ICEWS14, the experimental results in ICEWS18 and GDELT were second in effectiveness, the experimental effect is the worst in ICEWS05-15. Through analyzing the dataset, it was found that the dataset of ICEWS14 is the simplest. When comparing with ICEWS14, although the number of timestamps is the same in ICEWS18, this dataset has more entities and data. The experimental effect of ICEWS18 is also not as good as ICEWS14, indicating that contrastive learning is not effective in dealing with complex datasets. The ICEWS05-15 dataset has the highest number of entities and timestamps, but the amount of data is not the largest. In contrast, GDELT has a relatively large number of timestamps but not many entities, and there is extensive training data in GDELT, which enables the model to receive sufficient training. The experimental results of GDELT are much better than those in ICEWS05-15 dataset, indicating that comparative learning with many experimental data can improve classification performance. Moreover, the proposed TRCL model outperforms the CENET model. Specifically, the CENET model takes into account the frequency of contrastive learning and historical facts, however does not incorporate the model of factual evolution. Therefore, the proposed TRCL model is much higher than the CENET model.

Ablation study

To investigate the effects of repetitive historical information, recurrent encoder, and contrastive learning on the model TRCL, this work compared different variants of TRCL according to MRR and H@3 metrics: the variant TRCL-h removes repetitive historical information based on the TRCL model; the variant TRCL-nh removed the current encoder from the TRCL model; the variant TRCL-cl has removed contrastive learning from the TRCL model. We have shown the results of the variant model on datasets ICEWS14 and ICEWS18 in Table 4. The results illustrate that the TRCL model surpasses TRCL-h, TRCL-nh, and TRCL-cl in all metrics, which proves that the model can efficiently improve the capability of entity prediction tasks by integrating repetitive historical information, recurrent encoder, and contrastive learning. Specifically, as shown in Table 4.

From the perspective of MRR indicators, TRCL improved by approximately 2.3% on dataset ICEWS14 and 1.2% on dataset ICEWS18 compared to TRCL-h; From the H@3 indicator, TRCL improved by approximately 2.5% on dataset ICEWS14 and 1.6% on dataset ICEWS18 compared to TRCL-h. This indicates that incorporating repetitive historical information into TRCL can improve entity prediction tasks; From the perspective of MRR indicators, TRCL improved by approximately 8.3% on dataset ICEWS14 and 6.8% on dataset ICEWS18 compared to TRCL-nh; From the H @ 3 metric, TRCL improved by approximately 9.1% on dataset ICEWS14 and 7.4% on dataset ICEWS18 compared to TRCL-nh. This indicates that incorporating the current encoder into the TRCL model can effectively identify sequence information of facts.

Meanwhile, from the perspective of MRR indicators, the proposed TRCL model has improved by approximately 0.7% on dataset ICEWS14 and by approximately 0.4% on dataset ICEWS18 compared to TRCL-cl; From the H@3 metric, TRCL improved by approximately 0.4% on dataset ICEWS14 and approximately 0.4% on dataset ICEWS18 compared to TRCL-cl. This indicates that incorporating contrastive learning into TRCL can avoid the interference of repeated historical information on entity prediction tasks.

Sensitivity analysis

To investigate the influence of repeated historical facts on entity forecast tasks, this work conducted sensitivity analysis on hyper-parameter α in formula 16 on datasets ICEWS14 and ICEWS18. The value of a ranges from 0 to 1. The results are shown in Figs. 3 and 4. It can be seen that repeated historical facts have both positive and negative effects on entity prediction tasks. Specifically, on datasets ICEWS14 and ICEWS18, whether it is the MRR metric or the H@3 metric, the performance of entity prediction tasks is optimal when α = 0.3. This indicates that repetitive historical facts contribute to entity prediction tasks, but when this work overly focus on repetitive historical facts, it actually reduces the effectiveness of entity prediction. Our conclusion is also realistic, as historical events that occurred in the past may not necessarily occur in the future.

To investigate the influence of learning rate on entity forecast tasks, this work conducted sensitivity analysis on the learning rate on datasets ICEWS14 and ICEWS18. The values of learning rate are {0.1, 0.01, 0.05, 0.001, 0.005, 0.0001}. The results are shown in Figs. 5 and 6. It can be seen that on datasets ICEWS14 and ICEWS18, the performance of entity prediction tasks is optimal when the learning rate is 0.001, for both the MRR metric and the H@3 metric. This indicates that suitable learning rate contributed to good performance in entity prediction tasks.