Enhancing sentiment analysis of online comments: a novel approach integrating topic modeling and deep learning

Text sentiment analysis

Text sentiment analysis, a pivotal subfield of natural language processing (NLP), aims to automatically identify and classify the subjective polarity (positive, negative, or neutral) expressed in textual data through computational techniques (Deng et al., 2021). This technology has become increasingly vital for understanding public opinion and tracking the evolution of societal sentiment. The field has witnessed a methodological evolution, progressing from traditional lexicon-based approaches to machine learning and, more recently, deep learning techniques (Chen et al., 2023b).

In text sentiment analysis, the LDA model has been widely recognized for its robust feature extraction capabilities. It can identify latent topics within textual data, facilitating the optimization of information resources (Wang & Wu, 2023). Simultaneously, the LSTM model, a prominent deep learning technique, excels at processing sequential data. It can directly and accurately predict sentiment labels in text, with applications spanning stock index prediction (Xu & Tian, 2021) and enhancing intelligent library services (Zhao et al., 2023). Both LDA and LSTM models offer unique strengths and when combined, have the potential to advance the field of text sentiment analysis.

Enhancing the comprehensiveness and accuracy of text sentiment analysis is an active area of research, with a particular focus on the fusion and complementarity of multiple models (Chen & Guo, 2024; Zhu et al., 2022). The emergence of numerous datasets and the application of multimodal methods have provided ample impetus for this trend (Zhu et al., 2023). For example, He et al. (2023) successfully integrated Bidirectional Encoder Representations from Transformers (BERT) with the LDA model to improve sentiment classification. Deng et al. (2021) combined convolutional neural networks, bidirectional LSTMs, and attention mechanisms for enhanced sentiment analysis. Liu et al. (2022) incorporated LDA into convulational neural network (CNN) to improve short-text fault information classification. Additionally, models such as Robustly Optimized BERT Approach Bidirectional Long-short Term Memory Conditional Random Field (RoBERTa BiLSTM CRF) (Xu et al., 2023) have gained significant attention for their ability to effectively capture sentiment information in text, thereby improving classification accuracy and contributing to new directions in fine-grained sentiment analysis research.

LDA theme model

The LDA model is a foundational technique in the field of text mining. By establishing a three-layer Bayesian probabilistic framework linking vocabulary, topics, and documents, LDA uncovers the latent thematic structure inherent within text data. The model operates under the assumption that the vocabulary within a document is drawn randomly from latent topics, allowing it to capture the underlying complexity of the text while ensuring topic coherence (Xue et al., 2024). As depicted in Fig. 1, the LDA process involves several key steps. Initially, a topic distribution θ_i is generated for each document through sampling from a Dirichlet distribution α, thus laying the groundwork for the document’s thematic content. Subsequently, each word’s topic affiliation is determined by sampling from a multinomial distribution over topics. Next, a word distribution φ_Zi,j is generated for each topic through another Dirichlet distribution β. Finally, the actual words comprising the document are generated by sampling from a multinomial distribution over words, effectively revealing the underlying topic structure of the text (Hu, Han & Wang, 2024).

The LDA model has found widespread application in the realm of text sentiment analysis. He, Zhou & Zhao (2022) employed LDA to conduct an in-depth analysis of user comments, extracting sentiment-laden topics that influence user experiences and offering recommendations for product and service enhancements. In the context of social media sentiment analysis, Wang, Sun & Wang (2022) leveraged LDA to precisely identify topics discussed by the public and uncover their corresponding sentiment orientations. Moreover, LDA has been utilized to analyze user comments from diverse settings such as museums and theme parks, effectively identifying areas of concern for users and providing actionable insights for design improvements to management (Fudholi et al., 2023).

As the complexity of text sentiment analysis increases, the limitations of using LDA models in isolation have become apparent. Consequently, the development of LDA fusion models, which combine LDA with other machine learning or deep learning models, has emerged as a crucial strategy for enhancing performance. This approach has demonstrated significant improvements in topic classification accuracy and sentiment analysis. For instance, Watanabe & Baturo (2024) proposed a model integrating seed words with sequential LDA modeling, resulting in increased precision in topic matching and enhanced stability of sentence-level topic transitions. Wu & Shen (2024) improved the accuracy and foresight of LDA in detecting emerging topics by optimizing evaluation methods and constructing a novel identification framework. Additionally, Wu, Du & Lin (2023) integrated the Neural Basis Expansion Analysis (N-BEATS) network model with LDA, improving the accuracy of technological theme prediction. These fusion models highlight the ongoing relevance and innovative potential of LDA-based approaches in text sentiment analysis.

Normalized flow’s RealNVP model

Normalized Flows (NFs) (Dias et al., 2020), exemplified by the RealNVP model, have emerged as a powerful class of generative models capable of transforming simple distributions into complex data distributions through a series of invertible transformations. The core strength of RealNVP lies in its coupling layers, which partition and affine-transform input data, enabling efficient information mixing and the generation of intricate distributions (Papamakarios et al., 2021). This capacity for complex distribution modeling has inspired the exploration of leveraging a shuffled RealNVP variant to enhance feature representations in the context of text analysis, distinct from the broader application of NFs.

The potential of NFs in capturing nuanced emotional expressions within textual data has been demonstrated in sentiment analysis. Tran et al. (2019) pioneered the application of NFs to discrete data through their discrete flow model, facilitating effective text sentiment analysis and sequence dependency modeling. Building on this, Li, Wu & Wang (2020) introduced the Flow-Emotion model, leveraging RealNVP to learn sentiment embedding spaces and incorporating attention mechanisms for context-aware modeling, further underscoring the efficacy of NFs in sentiment analysis. While these works highlight the power of NFs in capturing textual nuances, they primarily focus on sentiment classification and do not explore the integration of NFs with topic modeling techniques like LDA, a gap addressed in this research.

Beyond sentiment analysis, RealNVP has found extensive application in modeling temporal data. Papamakarios, Pavlakou & Murray (2017) combined autoregressive structures and masking with RealNVP in their Masked Autoregressive Flow (MAF) model, establishing a foundation for efficient sequential modeling and the application of NFs in time series analysis. Kim et al. (2022) extended RealNVP to the financial domain, enabling fast pricing of path-dependent exotic options by simulating stochastic volatility models. Further highlighting RealNVP’s aptitude for handling complex time series, Kobyzev, Prince & Brubaker (2021) and Zhao et al. (2024) achieved significant results in anomaly detection with their Conditional Flow for Anomaly Detection (CFlow-AD) and Deep Denoising Autoencoder Normalizing Flow (DDANF) models, respectively, by integrating RealNVP with conditional normalized flows and denoising autoencoders. These applications showcase the versatility of RealNVP in diverse domains; however, its potential for enhancing feature representations derived from topic models like LDA for downstream tasks like LSTM-based text analysis remains largely unexplored, a direction pursued in this study.

LSTM model

The LSTM model, an advanced recurrent neural network architecture, excels in processing sequential data due to its unique design. As depicted in Fig. 2, the LSTM model incorporates memory cells and three logical gates: the input gate, forget gate, and output gate. This architecture enables the LSTM to effectively capture and process long-term dependencies, making it particularly suitable for tasks requiring contextual understanding (Yu et al., 2023). By seamlessly connecting long-term memory with current input, LSTM addresses the challenges of long-term dependencies and gradient vanishing that often plague traditional recurrent neural networks.

The LSTM model’s exceptional performance in text sentiment analysis is attributed to its ability to capture sentence structure and semantics, making it a valuable tool for sentiment classification tasks. By integrating deep learning with time series analysis, LSTM leverages contextual information effectively, thereby enhancing the model’s scalability and performance (Dellal-Hedjazi & Alimazighi, 2022; Liu, Wang & Li, 2023). For instance, a text sentiment polarity classification method based on an improved residual neural network (ResNet) and LSTM has demonstrated increased classification accuracy while maintaining operational efficiency (Liu, Yang & Yu, 2023). Furthermore, Xuan & Deng (2023) enhanced LSTM’s ability to capture textual sentiment by incorporating a deep attention mechanism and topic embedding strategy.

In scenarios involving complex, long texts, a single LSTM model may fail to retain crucial information during processing. Consequently, multi-model fusion has emerged as a key strategy for enhancing performance. Khan et al. (2022a) proposed a deep learning model integrating CNN with LSTM, achieving notable success in sentiment analysis of social media texts. This hybrid model first extracts local features from the text using CNN and subsequently processes temporal dependencies using LSTM, effectively combining the strengths of both architectures: CNN for feature extraction and LSTMs for temporal modeling.

Numerous studies have explored the fusion of LDA and LSTM models. For instance, Zeng, Yang & Zhou (2022) employed a combination of LDA and the Attention-BiLSTM model to reveal the dynamics of online public opinion. Li et al. (2022a) integrated LDA, BiLSTM, and the Self-attention model, achieving enhanced accuracy in short text classification. Hu et al. (2022) proposed a CNN-BiLSTM-MHSA-based electroencephalography (EEG) emotion recognition method, utilizing CNN to extract local features, BiLSTM to establish a temporal emotion change model, and MHSA to enhance the focus on key features, significantly improving emotion recognition rates on the DEAP dataset. These studies underscore the potential of combining models for improved text classification and sentiment analysis.

Word vectorization

Word vectorization, a fundamental technique in NLP, converted words into fixed-dimensional vectors, encapsulating semantic relationships and reducing feature dimensionality (Hu, Han & Wang, 2024). This study employed the Skip-gram method (Xia et al., 2020) for word vectorization, the principle of which is illustrated in Fig. 5. The core of Skip-gram lies in predicting a word’s context words to capture their inherent connections. Utilizing the preprocessed document set S_t as input, word vectors were generated using Skip-gram. The similarity between word vectors was quantified by calculating their inner product, thus inferring the correlation between words. Specifically, the similarity between any two-word vectors, W_i and W_j, was given by: (1)${\displaystyle \text{Similarity}\left(W_i,W_j\right)=W_i\cdot W_j=\sum _{k=1}^n{W}_{ik}\cdot W_{jk}.}$ Where W_i and W_j represented the vector forms of the two words, n was the dimensionality of the word vector, and W_ik and W_jk represented the k-th component of W_i and W_j, respectively.

Furthermore, this study employed the softmax function to compute the probability distribution of each word in the vocabulary being predicted given a central word. The softmax function is defined as follows: (2)${\displaystyle P\left(W_j|W_i\right)=\frac{exp\left(W_i\cdot W_j\right)}{\sum _{k=1}^{m}exp\left(W_i\cdot W_k\right)}.}$ Where P(W_j—W_i) represented the probability of word W_j being predicted given the central word W_i, and m represented the total number of distinct words in the vocabulary.

The probability distribution of each word in the vocabulary being predicted, given a central word, was obtained by calculating the inner product between their corresponding word vectors and normalizing the results using the softmax function. Through this word to vector (Word2Vec) model processing, text data underwent vectorized transformation, converting high-dimensional textual information into a lower-dimensional set of word vectors, D_t. The Skip-gram word embedding method was chosen for its ability to capture nuanced semantic relationships between words, thereby improving the model’s accuracy (Shobana & Murali, 2021).

As Fig. 5 illustrates, the Skip-gram model acquires distributed representations of words by predicting context words from a center word. This process maps words into a low-dimensional dense vector space, where each word vector encodes semantic information. Semantically similar words are positioned close to each other in this vector space, effectively capturing their relationships. Upon completion of Skip-gram training, each word is associated with a low-dimensional dense vector, and the collection of these vectors constitutes the word vector set D_t (illustrated in Fig. 4). This dimensionality reduction of the text data furnishes structured semantic input for downstream models.

LDA topic modelling stages

Training the LDA model

To investigate the underlying structure of the text data, this study employed the LDA model to identify latent topics within the documents. To optimize model performance, data were read from the document-term matrix file R_t and the LDA model was trained with adjusted parameters.

To identify optimal model parameters, this study employed perplexity and coherence as evaluation metrics (Ran & Li, 2023). Perplexity, a measure of how well a probability model predicts a sample, is a key indicator of model fit. It was calculated as follows: (7)${\displaystyle \text{Perplexity}\left(D\right)=exp\left\{-\frac{\sum _{d=1}^M\mathit{log}p\left(w_d\right)}{\sum _{d=1}^M{N}_d}\right\}.}$ Where D represented the test set in the corpus, containing M reviews. N_d denoted the number of words in review d, w_d represented the words in review d, and p(w)_d was the probability of generating the word w_d according to the model. The lowest point, or a point close to the lowest point, on the perplexity curve was typically considered the optimal model configuration, as it indicated improved generalization ability.

In addition to perplexity, this study utilized a coherence curve to assess the semantic coherence among words within the model. The peak of the coherence curve typically signified the optimal number of topics, where word relationships were strongest and most meaningful.

By comprehensively considering the analysis results of perplexity and coherence curves, the optimal number of topics for the LDA model was determined. After establishing the optimal configuration, the trained LDA model file L_t and its evaluation results were produced. Utilizing perplexity and coherence curves not only enhanced the accuracy of LDA model selection but also ensured the model’s effectiveness in capturing text associations and temporal dynamics. Moreover, these metrics served as crucial indicators for assessing the performance of subsequent models in the analysis pipeline.

Topic feature extraction

In this study, the trained LDA model L_t and the processed word vector dataset D_t served as input for topic feature extraction. To ensure compatibility with the LDA model’s input requirements, appropriate transformations were applied to the original data. Subsequently, the Gibbs sampling (Zhao et al., 2021) algorithm was employed for iterative approximation of the model’s posterior distribution. In each iteration, new parameter values were sampled based on the current estimates. Core statistics, such as the mean and standard deviation of model parameters, were then computed from the numerous samples obtained.

In the topic feature extraction process, the number of topics, k, was initialized based on the analysis of coherence and perplexity curves. To determine the topic distribution for each document, the probability of observing the remaining words in a document, given the removal of a specific word, was calculated as follows: (8)${\displaystyle P\left(Zi=k|w,z-i,\alpha ,\beta \right)\propto \frac{nk,-i+\alpha }{n.,-i+K\alpha }\times \frac{nw,k,-i+\beta }{nk,-i+V\beta }.}$ Where nk, -i represented the number of words assigned to topic k in the current document after excluding the i-th word; n., -i denoted the total number of topics assigned to all words in the current document after removing the i-th word; nw, k, -i indicated the number of times word w was assigned to topic k after excluding the i-th word, and V was the total number of unique words in the corpus. Through repeated iterations of Gibbs sampling until vocabulary convergence, the final output was a dataset file E_t containing feature vectors of the topic distribution for each document.

The iterative Gibbs sampling and associated probability calculations facilitated the accurate extraction of topic features, revealing the underlying semantic structure of the text.

Feature enhancement and fusion

Feature enhancement with shuffle

This study introduces the RealNVP model to model and enhance the topic features generated by LDA. RealNVP’s concise and reversible structure contributes to improved efficiency in modeling text topic features for sentiment analysis (Dinh, Sohl-Dickstein & Bengio, 2016). As illustrated in Fig. 6, the RealNVP model incorporates the Shuffle operation within its coupling layer to facilitate data mixing and bolster the model’s generalization capabilities. The Shuffle operation randomly rearranges the concatenated vectors h ₁ and h ₂ from the flow output, where s and t are vector functions parameterized by neural networks (Su, 2018).

The reversibility inherent to the RealNVP model ensures lossless information transmission during transformation, providing high-quality feature representations to the BiLSTM model. While preserving model reversibility, the Shuffle operation introduces data perturbation, thereby augmenting the model’s adaptability and robustness for sentiment classification tasks. Experimental design and hyperparameter tuning mitigate potential instability arising from this randomness, ensuring stable model output. Furthermore, aligning the RealNVP model’s hidden layer dimension with that of the LDA topic features guarantees effective data processing.

When applied to LDA’s topic features, the RealNVP model enhances both expressive power and robustness, offering an effective feature enhancement strategy for sentiment analysis. The model’s transformation is defined as:

(9)${\displaystyle y_{1:d}=x_{1:d}.}$ (10)${\displaystyle y_{d+1:D}=x_{d+1:D}\cdot exp\left(s\left(x_{1:d}\right)\right)+t\left(x_{1:d}\right).}$ Where s and t, parameterized by neural networks, implement nonlinear data transformations. The corresponding inverse transformation is:

(11)${\displaystyle x_{1:d}=y_{1:d}.}$ (12)${\displaystyle x_{d+1:D}=\left(y_{d+1:D}-t\left(x_{1:d}\right)\right)\cdot exp\left(-s\left(x_{1:d}\right)\right).}$

This transformation preserves data volume while achieving effective data mixing, furnishing the BiLSTM model with rich and uniform feature representations. By processing the LDA-processed text dataset E_t with the RealNVP model’s Shuffle coupling layer transformation, the data is encouraged to fully integrate contextual information, yielding an enhanced topic feature dataset F_t.

Feature fusion

Direct concatenation was selected as the feature fusion strategy due to its simplicity, ability to maintain information integrity, and proven success in multi-modal tasks (Xia, Li & Liu, 2023). By directly concatenating the original word vector sequence D_t with the enhanced topic feature vector F_t, both the fine-grained semantics of the text and the high-level topic information are preserved, avoiding the complexity and potential information loss associated with other fusion methods.

While LDA topic models excel at extracting the topic distribution of a text, they may overlook crucial semantic details such as word order and subtle nuances in meaning (Liu & Wei, 2020). Word vector sequences, through their dense vector representations, effectively encode this information. Therefore, incorporating the original word vector sequence in the feature fusion stage compensates for the limitations of the topic model, providing the BiLSTM model with a richer and more comprehensive text representation, ultimately leading to improved sentiment classification accuracy.

By concatenating the original word vector sequence D_t and the enhanced topic feature F_t, effective fusion of multi-modal features is achieved. This strategy not only enhances the model’s expressive power, enabling it to capture a broader range of textual information, but also strengthens its understanding of textual sentiment, culminating in a dataset N_t well-suited for BiLSTM input.

LSTM sentiment analysis model

Constructing a double-layer improved LSTM and MHSA model

To delve into the temporal and contextual information within the text data, a double-layer improved BiLSTM model was constructed, incorporating a MHSA to enhance the model’s ability to capture complex emotional dynamics and semantic relationships. As shown in Fig. 4, the network architecture comprised an Embedding layer, two stacked improved BiLSTM layers (with a Dropout layer following each to mitigate overfitting), an MHSA layer, and a fully connected Dense layer. This design progressively extracted and processed text features, enabling the model to effectively capture the intricate nature and temporal dynamics of textual data.

In contrast to traditional sentiment analysis approaches that employ standard LSTM networks, this study utilized a modified LSTM variant featuring a coupled forget and input gate mechanism. The internal architecture of this modification is illustrated in Fig. 7.

In the standard LSTM architecture (Fig. 2), the forget gate’s calculation is governed by the following equation: (13)${\displaystyle F_t=\sigma \left(W_f\cdot \left[H_{t-1},X_t\right]+b_f\right).}$

In this equation, the sigmoid activation function ( σ) compresses the input value into the range [0, 1], dictating the proportion of information from the previous cell state to be discarded. W_f, the forget gate’s weight matrix, maps the input to the forget ratio, while b_f, the forget gate’s bias vector, enables it to maintain a baseline activation level independent of input. The input gate’s calculation proceeds as follows:

(14)${\displaystyle I_t=\sigma \left(W_i\cdot \left[H_{t-1},X_t\right]+b_i\right).}$ (15)${\displaystyle {\hat{C}}_t=tanh\left(W_c\cdot \left[H_{t-1},X_t\right]+b_c\right).}$

In this expression, Ĉ_t denotes the candidate cell state, representing the new information potentially added to the cell state in the current time step. However, the conventional LSTM architecture suffers from several drawbacks. First, due to the independent calculation of the forget and input gates, each gating mechanism requires separate weight matrices and bias vectors. This increases the model’s parameter count, raising the risk of overfitting, particularly with limited training data. Second, each gating mechanism necessitates independent matrix operations and activation function calculations, increasing the model’s computational complexity. This can become a performance bottleneck for large-scale data processing or real-time applications. Third, the separate computations of the forget and input gates may lead to redundant or conflicting decision outcomes. For example, the information retained by the forget gate might be similar to the new information added by the input gate, resulting in information redundancy. To mitigate these shortcomings, this study employed a coupled forget and input gate design, as depicted in Fig. 7. The calculation is defined as follows: (16)${\displaystyle C_t=F_t\mathrm{\ast }{C}_{t-1}+\left(1-F_t\right)\mathrm{\ast }{\hat{C}}_t.}$

In this design, 1 − F_t can be interpreted as an implicit representation of the input gate. This coupling strategy allows the forget and input gates to not only share parameters but also operate synergistically in a complementary fashion. Consequently, this design not only streamlines the network’s structural complexity but also improves the model’s training efficiency and response speed by reducing redundant computations and the number of parameters, while maintaining accuracy.

The double-layer improved BiLSTM architecture enhanced the model’s capacity to process complex text data, particularly long sequences with deep-level dependencies. The first BiLSTM layer extracted initial temporal features while preserving the output at each time step to ensure comprehensive temporal information transfer. The second BiLSTM layer further refined these features, focusing on capturing long-term dependencies within the text. During model construction, careful consideration was given to the dimensionality of each layer. The output dimension of the first BiLSTM layer was set to be sufficiently large to capture the rich features present in the text, while the second layer’s output dimension was tailored for core feature extraction. The model’s input dimension was aligned with the output dimension of the preceding LDA model to facilitate effective information flow.

Subsequently, MHSA was employed to comprehensively capture the intricate interactions within the text features. MHSA’s multiple attention heads enable parallel processing of different parts of the input sequence, effectively capturing both global and local contextual information (Xiao et al., 2020). This multi-level attention mechanism not only enhances the model’s sensitivity to global information but also facilitates a deeper understanding of semantic associations and contextual dependencies within the text, crucial for tasks like sentiment analysis. Compared to other self-attention mechanisms, MHSA’s multi-head structure allows it to capture multi-scale features in parallel, improving the model’s expressive power while mitigating the risk of overfitting through attention dispersion (Xiao et al., 2020). The multi-head mechanism empowers MHSA to simultaneously focus on various aspects and granularities of the input sequence, striking a balance between global semantics and local details. This multi-granularity feature extraction capability is essential for in-depth understanding of textual semantics, particularly for sentiment analysis. Additionally, the fusion of multi-head outputs further enhances the model’s expressive power, enabling it to learn more complex and abstract text feature representations. As depicted in Fig. 4, MHSA takes the output sequence of the second BiLSTM layer as input and generates Query, Key, and Value vectors through linear transformations (Tan et al., 2023), which are then used to calculate attention weights. ${\displaystyle Q=X{W}^Q,K=X{W}^K,V=X{W}^V}$ where W^Q, W^K, and W^V represent the parameters learned by the model. For each attention head, the model computes attention scores and normalizes them using the softmax function to derive the attention weights. (17)${\displaystyle {\text{head}}_h=\text{Attention}\left(Q{W}^{Qh},K{W}^{Kh},V{W}^{Vh}\right)=softmax\left(\frac{Q{W}^{Qh}{\left(K{W}^{Kh}\right)}^T}{\sqrt{d_k}}\right)V{W}^{Vh}.}$

Finally, the outputs from all attention heads are concatenated and then transformed via a linear layer to produce the final output of the MHSA layer. (18)${\displaystyle MultiHead\left(Q,K,V\right)=Concat\left(hea{d}_1,hea{d}_2,\dots ,hea{d}_h\right)W^o.}$ Where W^o is a trainable weight matrix used for dimension adjustment.

At the output stage, a fully connected Dense layer integrated the features processed by the MHSA and employed a softmax activation function for three-class sentiment classification. This ensured that the output results were both probability distributions and mutually exclusive across categories. To mitigate overfitting and enhance the model’s generalization ability, Dropout layers (Chen et al., 2023a) were incorporated between the BiLSTM layers and between the MHSA layer and the Dense layer. Dropout layers randomly deactivated a proportion of neuron connections during training, reducing complex co-adaptations and promoting the development of more robust features.

Regarding loss function selection, this study employed the sparse_categorical_crossentropy loss function, which was well-suited for multi-class classification problems with softmax activation. This function quantified the dissimilarity between the predicted probability distribution and the true probability distribution. The mathematical formula for categorical cross-entropy loss is: (19)${\displaystyle \text{loss}=-\frac{1}{m}\sum _{j=1}^m\sum _{j=1}^k{y}_{ij}log\left({\check{y}}_{ij}\right).}$ Where m represented the number of samples, and k represented the number of categories.

The Adam optimizer was selected for model optimization in this study. Adam utilizes adaptive learning rates and incorporates bias correction, resulting in a stable learning rate range across iterations. This enhanced the stability and convergence speed of model training.

The double-layer improved BiLSTM architecture (L_t) increased the model’s capacity to represent and analyze complex text data effectively.

Model training and validation

Dataset E_t and BiLSTM model L_t were utilized for model evaluation and optimization. The data were divided into training set X_t, and a test set C_t to assess model performance and ensure generalizability.

To optimize the model’s ability to process text and capture temporal dynamics, the model structure, parameters, and learning rate were iteratively refined. This process aimed to address the limitations of the LDA model and leverage the strengths of the fusion approach. Through analysis of performance metrics during training and referencing the training set performance report, optimal model parameters were identified and saved as a Q_t file.

Model evaluation and validation

The model was evaluated by loading the training dataset X_t and the previously optimized model weights Q_t. Inferences were performed on the optimized model using the test dataset C_t, generating updated coherence and perplexity curves for visual assessment of model performance.

Key performance indicators (KPIs), including accuracy and F1 score (Santhiran, Varathan & Chiam, 2024), were calculated for both the original and improved models. Comparison of these KPIs demonstrates the enhanced performance of the improved model across multiple dimensions.

Experiments

The research was conducted utilizing a Windows 11 operating system environment. Hardware specifications included an Intel(R) Core (TM) i5-8265U CPU with a processing speed of 1.60 GHz (up to 1.80 GHz). The programming language platform employed was Python version 3.6.8. Software tools for development and analysis encompassed PyCharm Community Edition 2023.1.3 and Jupyter Notebook.

Datasets

In this study, an in-depth analysis of online comment data from visitors was conducted. A total of 7,991 core comments were collected from Dianping.com (Zhang & Long, 2024) through the utilization of web crawling technology (Zhang & Wu, 2022). Additionally, publicly available datasets (comprising 8,186 Chinese product reviews (IDataScience, 2022) and 50,000 English movie and TV show reviews (Ni, Li & McAuley, 2019)) were used to validate the model’s cross-lingual generalization capabilities. Given the correlation between star ratings and user sentiment, comments with 4 or 5 stars were classified as positive evaluations, those with 2 to 3.5 stars as neutral evaluations, and those with 1.5 stars or below as negative evaluations. The detailed classification is presented in Table 1.

In the data processing stage, regular expressions were first utilized to remove special symbols and extraneous punctuation from the text. Subsequently, the jieba library was employed for word segmentation, and stop words were eliminated. Following preprocessing, the dataset was randomly partitioned into a training set and a testing set at a ratio of 80%/20%. These sets were then used for model training and performance evaluation, respectively.

Hyperparameter setting

The number of topics in the LDA model significantly influenced its performance. By plotting the coherence curve (Fig. 8A) and the perplexity curve (Fig. 8B) for each dataset, the optimal number of topics was determined. For the Shanghai Astronomy Museum dataset, coherence peaked at four topics, with perplexity (after inversion) also being relatively high, indicating the model’s optimal generalization ability. For the mobile phone products dataset, coherence was highest at six topics, with good perplexity (after inversion). For the Amazon movie and TV review dataset, both coherence and perplexity (after inversion) reached their highest points at seven topics. The traditional LDA model exhibited some overfitting when processing the Chinese datasets, with perplexity increasing as the number of topics rose. However, the exponential upward trend of the perplexity curve also hinted at the limitations of the LDA model in handling complex contexts and temporal sequences. Therefore, next studies may consider integrating other models to enhance overall performance and address the overfitting issue.

The number of topics in the LDA model was optimized by evaluating coherence and perplexity on the respective datasets. The RealNVP model utilized 4 flows to balance model complexity and expressiveness, and was trained using maximum likelihood estimation and the Adam optimizer. The double-layered BiLSTM model employed 128-dimensional word vectors, with 128 and 64 units in its respective layers, to achieve progressive feature abstraction. The model incorporated four attention heads to capture multifaceted semantic information, with the attention dimension aligned with the BiLSTM output dimension. During training, a dropout ratio of 0.5 and a batch size of 32 were employed. Categorical cross-entropy was used as the loss function, and the Adam optimizer was utilized for optimization. Other parameter settings were as follows: a random seed of 42, a document processing batch size of 100, and 15 corpus iterations.

Evaluation metrics

For model evaluation, this study employed multiple metrics to comprehensively assess model performance. The effectiveness of model improvements was demonstrated by comparing coherence and perplexity curves.

To further quantify model performance, accuracy (ACC) (Bello, Ng & Leung, 2023) was selected as the primary evaluation metric. Accuracy measured the proportion of correctly classified instances out of the total number of instances. It was calculated as follows: (20)${\displaystyle \text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}.}$ Where TP and TN represented true positives and true negatives, respectively, indicating correctly predicted positive and negative instances. FP and FN, on the other hand, represented false positives and false negatives, which were incorrectly predicted positive and negative instances.

In addition to accuracy, the F1 score (Hsieh & Zeng, 2022) was included as a supplementary evaluation metric to account for both precision and recall. The F1 score was the harmonic mean of precision and recall, providing a balanced measure of a model’s performance, particularly in cases of imbalanced class distributions (Powers, 2011). It was calculated as follows: (21)${\displaystyle F1=\frac{2\ast TP}{2\ast TP+FP+FN}.}$

By utilizing multiple evaluation metrics, including accuracy and F1 score, this study conducted a comprehensive analysis of model performance from various perspectives.

Comparison of model performance results

This study compared and analyzed the performance of several models for sentiment analysis: the traditional LDA model, the LSTM model, the LDA-CNN model (Liu et al., 2022), the CNN-LSTM model (Khan et al., 2022a), and the proposed LDA-RealNVP-DBiLSTM-MHSA model. The LDA-CNN model enhanced short text classification by integrating global topic information with local contextual features. The CNN-LSTM model has demonstrated strong performance in sentiment analysis of English and Roman Urdu social media texts. This study addressed the challenges of text relevance and temporality in sentiment analysis by optimizing topic selection and model structure, incorporating RealNVP with Shuffle operation to enhance LDA topic features, utilizing a double-layered improved BiLSTM to model text temporality, and integrating MHSA to capture multi-scale semantic relationships.

To objectively evaluate model performance, all models were trained and tested on the same dataset (80% training set, 20% testing set). Comparing coherence and perplexity across models (Fig. 9) revealed consistent improvements in coherence for the optimized LDA-RealNVP-DBiLSTM-MHSA model across all datasets, with a notable improvement of approximately 10% on the Shanghai Astronomy Museum dataset. Additionally, the model demonstrated good performance in terms of perplexity. The incorporation of strategies such as RealNVP with Shuffle operation, the double-layer improved BiLSTM, and MHSA effectively enhanced the overall performance of the model in handling text. This mitigated the limitations of the traditional LDA model in modeling complex contextual temporality and also alleviated the issue of overfitting.

The results of the multi-model performance comparison were presented in Table 2. The optimized LDA-RealNVP-DBiLSTM-MHSA model outperformed the LSTM, LDA-CNN, and CNN-LSTM models in terms of both ACC and F1 score. Notably, while the CNN-LSTM model has demonstrated strong performance in processing English and Roman Urdu data, its performance was comparatively lower on the Chinese dataset used in this study, as reflected in its ACC and F1 scores. LSTM outperforms LDA-CNN in online comment sentiment analysis primarily because LSTM can effectively capture long-range dependencies and contextual information in sequential data, thereby better understanding complex sentence structures and emotional expressions.

Comparison of experimental results

To process the comment data more effectively, the study employed Skip-gram and the LDA model to reduce the dimensionality of high-dimensional word vectors. RealNVP was then utilized for feature enhancement, and the resulting vectors were input into a double-layer improved BiLSTM and MHSA model for refined processing. The experimental results, as shown in Table 3, demonstrate the exceptional performance of the proposed LDA-RealNVP-DBiLSTM-MHSA model in processing online review data. It significantly outperformed other models in both accuracy and F1 evaluation metrics. This outcome confirms the effectiveness and practicality of the LDA-RealNVP-DBiLSTM-MHSA model, particularly in handling complex text relevance and temporality.

To further illustrate the advantages of the LDA-RealNVP-DBiLSTM-MHSA model, Fig. 10 presents comparisons of ACC and F1 values for various models across different datasets. The results consistently confirmed the superiority of the proposed model over the baseline models in terms of all evaluation metrics. On the English review dataset, as shown in Fig. 10B, although the F1 value of the proposed model was close to that of the LDA-CNN model, it still achieved the highest score. This can be attributed to the incorporation of RealNVP with Shuffle operation after LDA in the proposed model, which enabled it to better capture text relevance. The experimental results fully substantiate the superiority of the LDA-RealNVP-DBiLSTM-MHSA model in processing both Chinese and English online review data.