Enabling personalized smart tourism with location-based social networks

Personalized smart tourism

With the rapid advancements in mobile internet and intelligent technologies, including artificial intelligence, big data analysis, machine learning, and recommendation algorithms, travel behaviors have undergone significant transformations. There is an increasing trend among users to utilize smart phones and mobile applications for accessing travel information, planning itineraries, and booking services. This intelligent travel approach serves as the technological foundation for personalized tourism.

Traditional travel planning processes typically require travelers to invest a substantial amount of time and effort in searching for information, organizing itineraries, and dealing with challenges related to information overload and decision-making (Wang & Wang, 2023; Xia et al., 2018). However, traditional travel recommendation systems often rely on generic strategies or static travel guides, lacking the ability to offer personalized travel recommendations tailored to users’ interests, preferences, and time constraints.

Personalized smart tourism encompasses the use of advanced technologies and intelligent algorithms (e.g., next POI recommendation, trip recommendation) to aid users in efficiently filtering and presenting the most pertinent and valuable information, thereby alleviating the cognitive burden associated with information retrieval and filtering (Kong et al., 2019) and provide valuable market insights and user insights to tourism enterprises (Chen et al., 2020).

POI recommendation

In contrast to conventional POI recommendation tasks, the POI sequence recommendation problem entails crafting a sequence of POIs adhering to particular spatiotemporal restrictions. Algorithms for POI sequence recommendation fall into two main categories: statistical-based and deep learning-based methods. Statistical-based approaches often draw inspiration from the orienteering problem (OP) and leverage heuristic techniques to optimize the accumulated scores within predefined constraints. These methods integrate specific query constraints and historical data, such as POI popularity and user preferences, to generate trajectory sequences. For example, Taylor, Lim & Chan (2018) use Integer Linear Programming (ILP) to recommend a sequence of places, considering factors like starting and ending points, time intervals, duration at each point, and popularity. Wei, Zheng & Peng (2012) introduce a collective knowledge-based framework for inferring routes, which extracts spatiotemporal characteristics from uncertain trajectories within predefined location and time constraints. They build a routable graph through a mutually reinforcing approach and employ a routing algorithm to produce top-k popular routes. Zheng & Xie (2011) initially capture the historical geographic locations of multiple users using a Tree-based Hierarchical Graph (TBHG) modeling technique. Yet, these methods either depend on local transition distributions, overlook long-term dependencies among POIs, or focus solely on time constraints. Consequently, relying solely on statistical approaches fails to deliver personalized recommendations or grasp the authentic check-in patterns of users.

Recently, numerous studies have turned to deep learning techniques to handle the intricate relationships and features present in data, tackling tasks related to POI sequences by leveraging sequence or location embeddings. Notably, recurrent neural networks (RNNs), long short-term memory (LSTM), and gated recurrent units (GRU) have been utilized to capture semantic connections between user mobility and POI sequences. Zhou, Mascolo & Zhao (2019) introduce a holistic deep learning framework that seamlessly merges community and user preferences. They develop a topic memory-enhanced network, employing neural attention mechanisms and nonlinear methodologies, to amplify both the interpretability and recommendation efficacy of POIs. Feng et al. (2018) introduce the DeepMove model, utilizing RNN as its backbone, to account for various factors impacting user mobility via a multimodal embedding approach. By employing historical attention mechanisms, they capture diverse periodicities and transition patterns within user flows, facilitating accurate predictions of user mobility (Feng et al., 2018). These sequence modeling approaches using RNNs typically capture linear transition patterns between check-in POIs but overlook the sparsity and high dimensionality of user sequences. Zhou et al. (2018) present a user trajectory autoencoder model grounded in semi-supervised learning. Their model relies on robust assumptions regarding the distribution of user sequences and effectively encodes semantic information within and across check-in sequences to capture human mobility patterns. Gao et al. (2021) introduce DeepTrip, an adversarial neural network model employing a generator and discriminator to produce query and travel representations, enhancing the recommendation of optimal routes. The model delves deeply into contextual POI information and employs an encoder–decoder structure to capture user mobility. Kong et al. (2024) extracted temporal dependence through time series decomposition and autocorrelation mechanisms, and extracted spatial dependence through learnable adaptive graph convolution operations. However, these deep learning techniques neglect user diversity, excessively depend on historical data, and struggle to accurately model the intricate and uncertain nature of user check-in behavior. In order to emphasise more clearly the place and importance of this study in the literature, we have added a literature table containing various important columns such as model name, dataset, evaluation, etc., as shown in Table 1.

In this study, we will employ self-supervised contrastive learning and incorporate user travel patterns to simulate users’ real travel behavior, thereby enhancing the POI sequence recommendation problem.

Exploring POIs in LBSNs

POI context information modeling

Location-based recommendation methods not only need to consider user preferences but also take into account the contextual information related to POIs in LBSNs, including spatial information, time information, semantic information, and sequence information, as shown in Fig. 3. The modeling process is as follows:

1. Spatial information. Our goal is to investigate users’ underlying geographic preferences, particularly their inclination towards selecting nearby check-in locations when choosing their next destination. To achieve this, We set a geographic threshold of 2 km and established the spatial connections among POIs within this threshold. We construct a POI-geographical graph $G_{vv}=\left(\mathcal{V}\cup \mathcal{V},{\mathcal{E}}_{vv}\right)$, where $\mathcal{V}$ denotes a set of POI, ${\mathcal{E}}_{vv}$ is the set of edges between POIs.

2. Time information. Users may have a preference for visiting certain POI during specific time periods, such as visiting museums in the morning and going shopping in the afternoon. In order to model the periodic patterns in time, we define a POI-time graph $G_{vt}=\left(\mathcal{V}\cup \mathcal{T},{\mathcal{E}}_{vt}\right)$, where $\mathcal{T}$ denotes the set of time-stamp, ${\mathcal{E}}_{vt}$ is the set of edges between POI and timestamp.

3. Semantic information. Our aim is to explore the relationships between POI, specifically by modeling an POI-category graph $G_{vc}=\left(\mathcal{V}\cup \mathcal{C},{\mathcal{E}}_{vc}\right)$, where $\mathcal{C}$ denotes the set of category, ${\mathcal{E}}_{vc}$ is the set of edges between POI and category.

4. Sequence information. Human mobility patterns exhibit strong sequential patterns, where the transition from one checked-in POI to another follows a non-uniform distribution. In order to leverage sequential effect, we employ text analysis methods to construct relationships between POI. Specifically, each POI is treated as a word, each sequence of checked-in POI is treated as a sentence, and all sequences of checked-in POI are treated as a corpus to be represented.

Alleviate data sparsity by contrastive learning

Existing supervised models for trip recommendation overly emphasize performance and overlook the potential correlation between contextual data and POI sequence data, resulting in inefficient data representation. To address the above issues, this study draws inspiration from self-supervised learning to enhance POI sequence recommendation. The ability of supervised learning to automatically learn from large amounts of labeled data has led to its wide application in fields such as natural language processing and computer vision. Models represented by SimCLR and MoCo have achieved remarkable results on the ImageNet dataset. The SimCLR model generates positive samples by rotating the image, cropping the image, adding Gaussian noise, and coloring (Chen et al., 2020). The MoCo model encodes negative samples using a momentum encoder to increase the number of negative samples, and also employs a sliding average strategy to ensure timeliness (He et al., 2020). In our method, a contrastive learning module is designed to pretrain the model and explore different data augmentation strategies to uncover the intrinsic correlations within the data, thereby enriching the self-supervised signals for enhancing data representation and alleviate data sparsity. The data augmentation strategies can be denoted as follows.

1. Token shuffling. The purpose of token shuffling is to randomly shuffle the order of token in an input sequence to increase the robustness and generalization performance of the model. When mapping token shuffling strategy to real travel behavior, it can be modeled as the scenario where even if the recommendation method suggests a sequence of POI to the user, the user may change the entire trip due to immediate preferences. For example, the method suggests a sequence of POIs<A, B, C, D, E, F > to a user, but the user may temporarily change it to <A, C, E, B, D, F > due to various factors. This is a result of the diversity of user preferences. To simulate this disruptive behavior, we randomly shuffle the order of POI in the sequence, create a new trip route, and add it to the training set. Define the sequence of POI [v_i, …, v_i+r+1] of S to be $\left[v_i^m,\dots ,v_{i+r+1}^m\right]$, and $S^R=R\left(S\right)=\left[v_1,v_2,\dots ,v_i^m,v_{i+r-1}^m,\dots ,v_n\right]$, where r = ⌈ωn⌉ is the length of the sub-interest sequence, 0 ≤ ω ≤ 1.

2. Random deletion. By applying the random deletion strategy to trip recommendation, we simulate the behavior of users temporarily not wanting to visit certain POI, thereby creating a new itinerary view. For example, users may change their trip plans for various reasons (e.g., personal reasons, weather conditions) and decide to skip or cancel visiting the next POI and proceed to the following one. Define the sequence $S^M=M\left(S\right)=\left[v_1^m,v_2^m,\dots ,v_n^m\right]$, where $v_1^m$ represents that if v₁ is selected, it will be erased; otherwise, $v_1^m=v_1$. Define l = ⌈μn⌉ as the set of points to be erased, where l is controlled by the hyperparameter μ, with 0 ≤ μ ≤ 1.

3. Random insertion. In the check-in dataset, the user’s POI check-in sequences are often sparse, making it difficult to fully capture the user’s preferences and the relevance of POIs when training with these sequences. To address this issue, the approach of inserting POI can be used to construct expanded check-in sequences. Specifically, we randomly select k different POI indices {idx₁, idx₂, …, idx_k} from the POI sequence, where k = ⌈αn⌉, idx_i ∈ [1, 2, …, n], α ∈ [0, 1] is the substitution rate. The substituted sequence is $S^s=S\left(S\right)=\left[v_1,v_2,\dots ,{\bar{v}}_{id{x}_i},v_{\left|S\right|}\right],$ where ${\bar{v}}_{id{x}_i}$ is the substituted interest point.

4. Synonyms substitution. The purpose is to recommend similar and substitutable POI to users in order to discover their additional real interests. Replacing elements in the sequence with highly correlated POIs reduces the information loss of the original sequence, thereby generating high-quality positive pairs. In trip recommendation, recommending similar and substitutable items can help users discover more of their actual interests. Meanwhile, users may have a preference for visiting similar POI within a geographic threshold. Specifically, the ratio of inserted interest points is controlled by β ∈ [0, 1]. Randomly select k different POI indices {idx₁, idx₂, …, idx_k}, where k = ⌈βn⌉, idx_i ∈ [1, 2, …, n]. The sequence after insertion is $S^I=I\left(S\right)=\left[v_1,v_2,\dots ,{\bar{v}}_{id{x}_i},v_{id{x}_i},\dots ,v_n\right]$, where ${\bar{v}}_{id{x}_i}$ is the POI related to v_idxi. The length of the expanded sequence S^I is k + n.

During the contrastive learning phase, by implementing the objective of having positive examples of POI close to the sequence and negative examples far from the sequence (Oord, Li & Vinyals, 2018), the correlation between POI and sequences is used as an additional signal to improve trip recommendation effectiveness.

For a given set of interest point sequences S, for each interest point sequence S_i, two augmentation strategies are randomly selected from the four strategies mentioned above. This results in an augmented sequence of 2N elements S₁, S₂, …, S_2i−1, S_2i, …, S_2N−1, S_2N, where i 1, 2, …, N. The pairs (S_2i−1, S_2i) are treated as positive pairs, while the remaining 2(N-1) augmented views serve as negative pairs for each positive pair. After encoding by the model’s decoder, each positive pair is represented as (h_2i−1, h_2i). In this paper, we adopt Noise Contrastive Estimation (NCE) as the loss function. The final pre-training loss function is as follows: (6)${\displaystyle \mathcal{L}=log\frac{exp\left(sim\left({\tilde{h}}_{2i-1},{\tilde{h}}_{2i}\right)\right)}{exp\left(sim\left({\tilde{h}}_{2i-1},{\tilde{h}}_{2i}\right)\right)+\sum _{k=1,k\ne 2i-1}^{2N}exp\left(sim\left({\tilde{h}}_{2i-1},{\tilde{h}}_k\right)\right)}.}$

The overall process of framwork

First, to convert the user’s query conditions into vector embeddings, the contextual representations of the POI obtained from Section ‘Alleviate data sparsity by contrastive learning’ are used to transform the start location l_s and the destination location l_d. In addition, the timestamps are divided into 24 h, and the start time t_s and end time t_d are one-hot encoded. After obtaining the embedding representations, we uses concatenation to obtain the query condition q: (7)${\displaystyle q=LeakyRelu\left(\left[s\left(l_s\right)\left|t\left(t_s\right)\right|s\left(l_d\right)|t\left(t_d\right)\right]W_q+b_q\right).}$

Second, we use an attention mechanism to fuse the query vector q and the historical sequence vector S, and the fused vector contains both historical information and query information. Through the attention mechanism, the model automatically learns which historical visited POIs are most relevant to the current query and then takes the weighted average of the representations of these locations to obtain a new vector Z.

Third, We input the fused vector Z into a encoder–decoder architecture and train the neural network using the parameters learned from pre-training, ultimately obtaining the recommendation results (Vaswani et al., 2017). The final loss function is defined as follows: (8)${\displaystyle \mathcal{L}=\parallel \widehat{Src}-Target\parallel .}$

In the above, $\widehat{Src}$ represents the output of the decoder, and Target represents the real POI sequence.

Experiment settings

Performance comparison

The performance of our proposed model and the eight baselines on two datasets evaluated by F₁ and pairs-F₁ is shown in Table 3. Our methods achieves the best results in both F₁ score and pairs-F₁ score, demonstrating the highest accuracy compared to other methods. The Markov-based methods focus more on transitions between POIs but overlook other contextual information, resulting in the poorest performance compared to other methods. On the other hand, the rank-based methods, Markov-Rank and POIRank, capture user mobility by considering both co-occurrence patterns and feature information of POIs, outperforming the Markov-based method. However, due to their reliance on statistical modeling or machine learning methods, these approaches fail to fully exploit users’ complex mobility patterns and their short and long-term preferences, leading to inferior performance compared to deep learning models such as DeepTrip and NASR. Although DeepTrip combines latent variables and utilizes adversarial generative neural network structures to capture users’ visiting intentions and mobility patterns, its performance is affected by data sparsity and ranks second. Our model exhibits the best performance. This is attributed to its effective fusion of POI contextual features, the design of intuitive augmentation strategies to simulate users’ real check-in behavior, and the provision of self-supervised signals for adequately modeling users’ complex demands.

Next, we analyze the results of recommendation visualization using Osaka as the validation city for trip recommendation, as shown in Fig. 4. Users provide the starting and ending locations of POIs, along with the corresponding time information. The figure displays the recommendation results of three baseline methods: Markov, Markov Ranking, DeepTrip, as well as our proposed method. From the analysis of the figure, it can be observed that the Markov-based methods can only form a short sequence of POIs. This is because this method focus on capturing local features and transition relationships, only using the last POI in the sequence to recommend the next one. The Markov algorithm will result in redundant POIs, as shown in the figure. By introducing POI ranking information, the Markov-Rank methods can effectively reduce the occurrence of redundant POIs but perform poorly in terms of the global sequence distribution. DeepTrip can capture global sequence patterns and outperforms the previous two methods, but it falls short in dealing with the issue of data sparsity and insufficient modeling of sequence position information, resulting in inferior recommendation performance compared to our method.

Data augmentation analysis

In this section, we evaluate the effect of various trip augmentation strategies for recommendation performance. We consider five options for each transformation, including aforementioned four strategies and None, resulting in 5 × 5 combinations. Especially, None means we do nothing and diagonal implies that we employ the same augmentation strategy for a sequences.

The testing results with different couples of augmentation strategy can be found in Fig. 5. We can make the following observations. First, Random Insertion is the most effective strategies, significantly outperforming other augmentation strategies. By solely utilizing the random insertion strategy, we obtained the optimal outcome. Additionally, when integrating random insertion with synonyms subtitution, we achieved the second highest performance. The reason for this phenomenon may be attributed to the fact that in recommendation model, which recommend a series of consecutive POIs, the sequence length is typically short. Therefore, both the random insert strategy and the synonyms subtitution strategy can effectively perturb the correlation between POIs, resulting in augmented sequences with higher confidence, especially with the random insertion strategy. As a result, whether employing only the random insert strategy or a combination of random insertion and synonyms subtitution, the model achieves remarkably high performance. Random deletion performs slightly worse than random insertion but better than other strategies, possibly because it alters the structure of the sequence and generates hard examples.

Impact of different infactors

In this part, we first verified the impact of individual factors among various factors in exploring POI context representation on the overall performance. Additionally, we examined the influence of the contrastive learning module on the model’s effectiveness. Hence, we use W/O to represent the removal of the corresponding factors, and we design several variants of proposed method to verify the impact of each information, as shown in Fig. 6.

After conducting ablation experiments on Osaka datasets, the method that incorporated the context feature module achieved the best performance. This indicates the significance of incorporating POI context information into the model. Compared to relying solely on a single sequence representation or a few representations, a more comprehensive set of information influences user decision-making. In the Osaka dataset, the sampling methods that removed the POI distance representation and POI category representation achieved the first and second worst performances, respectively. Removing the remaining two representations yielded slightly lower performance. This suggests a strong correlation between user check-in patterns in Osaka and time factors, indicating that users may be influenced more by their time periodic preferences when making decisions. This shows that there is a strong correlation between user check-in patterns in Osaka and geographical factors, indicating that users may be more affected by the distance of the check-in point and the POI itself when making a decision.

Additionally, through the exploration of the contrastive learning module on both datasets, we found that this module effectively improved the overall performance of the algorithm in terms of the F₁ score and pairs-F₁ score. This demonstrates that employing data augmentation strategies can enhance data representation, adequately modeling complex travel patterns and dynamic preferences of users. It indicates that the sparsity of sequence data and the heterogeneity of user needs are critical factors influencing sequence recommendation.

Parameters analysis

This section investigates the sensitivity of some key hyperparameters, including (1) the number of layers in the encoder and decoder, n_layer, (2) the number of heads in the multi-head attention mechanism, n_head, and (3) the model’s embedding dimension, d_model. We conduct extensive experiments to analyze the importance of these parameters. The chosen parameters for this study are: n_layer = 4, n_head = 8, d_model = 128. As shown in Fig. 7, the model’s representational dimension and the number of encoder–decoder layers cause more drastic changes in model performance, with the model dimension being the most significant. The possible reason is that a higher number of parameters means the model can learn and represent the complex relationships within the data more fully. When adjusted, the model has a greater capacity to adapt to a wider range of data characteristics. Model performance is most sensitive to the model’s dimension. When adjusting the number of attention heads, the model performance does not change significantly, which may be because the dataset does not have particularly complex dependencies that would lead to a notable performance improvement.