Integrating multi-criteria decision-making with hybrid deep learning for sentiment analysis in recommender systems

Multi-criteria problem formulation

Systems that employ both overall user-item evaluations and ratings on specific criteria to provide suggestions are called multi-criteria recommender systems (MCRSs). This section employs a deep neural network to predict a user’s ratings across many categories for a given item. The multi-criteria recommender systems are formulated as, (1)${\displaystyle U_{ij}\times I_{ij}\times C_{ij}\to R_0\times R_1\to .....R_{n-1}}$

where U_i,j is the j users of the dataset i = T, M, A. This T, M, A corresponds to TripAdvisor reviews, tmdb 5000 movies, and an Amazon user. I_i,j represents the j items from i = T, M, A.C_i,j represents the j criteria from i = T, M, A. The R₀ ×R₁ ×.............. R_n−1 denotes the overall rating given by each user of the particular dataset i at the number of criteria n. In the formulation, the total rating data is viewed as a distinct form of criteria rating.

A multi-criteria predictive model is constructed by fitting observed user-item-criterion rating data and then utilized to forecast the overall ratings and several criterion-specific ratings that a user would give to strange things. Since the user-item criterion rating data is three-dimensional, in this research third-order tensor, a generalization of the matrix, is naturally used to represent it. In the event of a movie, a third-order user-item-criterion rating tensor would be used as shown in Fig. 1, in which each user would assign a rating between 1, 2, 3, 4, 5 to each criterion of the item, with a question mark denoting an undetermined rating. Table 2 shows the criteria used in each of the datasets. The tensor data was then used to construct multiple factorization algorithms for obtaining the multi-criteria suggestions.

As the primary focus is on collaborative filtering, item ID or user ID is the only feature utilized in this work. Nevertheless, this model’s feature representation is versatile enough to incorporate user and item content features, which can address the cold start problem. This research utilized the feature representation approach suggested by Sánchez-Moreno et al. (2020) in their NCF framework since the ID is a definite feature without a logical order. The ID was initially transformed into an embedding vector, a dense continuous-valued, low-dimensional vector. The embedding vectors are modified from their original random values during model training to achieve the best possible loss minimization. In the input layer, where the rating criteria D is applied, the input vector I is represented as, (2)${\displaystyle I=concatenate\left(A_u,A_i\right)}$

where A_u, A_i are the vectors of the user and the items. The Rectified Linear Units (ReLU) is then selected as the activation function, followed by a series of hidden layers. (3)${\displaystyle ReLu\left(x\right)=max\left(x,0\right)}$

In this stage, calculation is performed on the MC item based CF similarity between a target item i and a neighbor item j by calculating the partial similarities between each rating criteria c, and using an aggregation function to acquire the overall similarity value. Compared to the conventional item-based CF similarity techniques, using Euclidean distance as the similarity measure proved to be an excellent choice for item-item similarity computation. Below is a breakdown of how we use the Euclidean Distance similarity measure to compute MC item-based CF similarity values between the target item i and the item neighbor j based on each criterion. (4)${\displaystyle d_i^c,j=\sum _{u=1}^{m}abs{\left(S_{u,i}^c-S_{u,j}^c\right)}^2}$

where $s_{\left(u,i\right)}^c$ and v represent user u’s ratings on items i and j relative to criterion c. The number of people who shared ratings for items i and j is denoted by m. A higher similarity rating indicates a closer relationship between two things. Therefore, to translate the distance into a similarity value based on each criterion, the following metric is required (5)${\displaystyle S{i}_{i,j}=\frac{1}{1+d_{i,j}^c}.}$

To calculate the overall similarity value between a target item i and a neighbor item j, the least similarity is utilized as an aggregation strategy for partial similarities. (6)${\displaystyle S{i}_{i,j}=\frac{1}{1+d_{i,j}^c}.}$

Partial similarity value according to criterion c, denoted by Si_i,jc, for a set of criteria c, denoted by m.

Preprocessing

Before it can be used for sentiment analysis, the text training data must be cleaned up. Text cleaning is a stage in preprocessing that involves removing words or other components from the text that do not contain important information and, as a result, has the potential to impede the efficiency of sentiment analysis. To convert text data into numerical vectors, methods such as word embedding can be used after the text has been cleaned, segmented into individual words, and then lemmatized to return them to their original form. Many unsupervised methods for word representation rely on global word representation to capture the meaning of a single word embedding in the context of the entire detected corpus, using word frequency and co-occurrence counts as their primary measurements of success. The GloVe model generates a word vector space with meaningful substructure by training on worldwide co-occurrence amounts of words and adequately using statistics by reducing least-squares error. When using a vector distance to compare two words, this outline is sufficient in preserving their similarities (Tifrea, Bécigneul & Ganea, 2018; Mohammed, Jacksi & Zeebaree, 2020).

The co-occurrence matrix M is used to keep track of these data, with each entry representing the frequency with which word y appears next to word x. Because of this, (7)${\displaystyle P_{x,y}=P\left(x|y\right)=\frac{M_{x,y}}{M_x}}$

where (8)${\displaystyle M_x=\sum _z{M}_{xz}}$ in which we count z words in the count matrix. It is the likelihood that a word at index y appears next to a word at index x.

Starting with ratios of co-occurrence probability is a good place to discover how words are embedded in sentences. The function K is first defined as (9)${\displaystyle K\left(W_x,W_y,\widetilde{W_z}\right)=\frac{P\left(z|x\right)}{P\left(z|y\right)}.}$

This is based on a third independent context vector indexed by z and two-word vectors indexed by x and y. The information in the ratio is encoded in the letter M, and the vector representation of the resulting difference is as simple as subtracting two vectors. (10)${\displaystyle K\left(W_x{-W}_y,\sim W_z\right)=\frac{P\left(z|x\right)}{P\left(z|y\right)}.}$

The vector is on the left and the scalar is on the right. Calculating the product of two terms avoids this (11)${\displaystyle K\left({\left(W\right)}_{x-W\left(y\right)}^T\sim W_z\right)=\frac{P\left(z|x\right)}{P\left(z|y\right)}.}$

As long as context words and standard words are random in the word-word co-occurrence matrix, this may replace the probabilities ratio with, (12)${\displaystyle K\left({\left(W\right)}_{x-W}{\left(y\right)}^{T\widetilde{W_z}}\right)=k\left({W_x}^T.\widetilde{W_z}-{W_x}^T.\widetilde{W_z}\right)=\frac{K\left({W_x}^{T\widetilde{W_z}}\right)}{K\left({W_y}^{T\widetilde{W_z}}\right)},}$ (13)${\displaystyle K\left({W_x}^T\widetilde{W_z}\right)=P\left(z|x\right)=\frac{M_{x,z}}{M_x},}$

since k(x)= e^x is a solution! (14)${\displaystyle k\left(x\right)=e^{w_x^T.\widetilde{W_z}}=P\left(z|x\right)}$ (15)${\displaystyle {W_x}^T\widetilde{W_z}=log{P}_{z|x}=log{M}_{x,z}-log{M}_x}$ after adding some biases, (16)${\displaystyle {W_x}^T\widetilde{W_z}+b_x+\widetilde{b_z}=log{M}_{x,z}.}$

Then the loss function of our model is (17)${\displaystyle L=\sum _{x,y=1}^{N}k\left(M_{x,y}\right){\left({W_x}^T\widetilde{W_z}+b_x+\widetilde{b_z}-log{M}_{x,z}\right)}^2.}$

Some decision-making studies assess expert opinions using sentiment analysis methodologies. These few studies calculate the value of expert opinion using lexicon-based sentiment analysis techniques. The opinion value is calculated by locating the lexicon’s opinion terms in the assessments. The authors employ sentiment lexicons to interpret expert opinions. The lexical coverage of the sentiment lexicon limits these two ideas, and they do not evaluate experts’ evaluations semantically.

Deep learning model

Compared to the performance of a single model, hybrid models have the potential to improve the accuracy of sentiment analysis (Wiebe et al., 2004). The proposal uses two hybrid deep learning models, each with its unique combination of residual attention CNN (RACNN) and Bi-LSTM networks in the deep learning layers. This allows us to take advantage of both approaches’ strengths and compensate for individual weaknesses. The combination will enable us to make the most of RACNN and Bi-LSTM’s capabilities, with RACNN extracting features and Bi-LSTM storing historical information at state nodes. The RACNN and Bi-LSTM models are combined to form the first hybrid model, while the Bi-LSTM and RACNN models include the second hybrid model. We used an ordinal scale with four classes (disagreement, negative, neutral, and positive) to identify the ratings as either positive, negative, neutral, or neutral to train and validate the outcome of the sentiment analysis. The five categories represented here are highly damaging, pessimistic, pessimistic, optimistic, and optimistic.

The residual attention network is an end-to-end trainable convolutional neural network incorporating an attention mechanism into the state-of-the-art feed-forward network architecture. “Attention” is a method used in artificial neural networks to simulate the human ability to pay close mental attention. The impact boosts some aspects of the input data while reducing the prominence of others; this is done so that of may pay more attention to the less prominent but more crucial details. The convolutional neural network (CNN) is a max-pooling structure formed by a fully linked set of one or more feed-forward networks. The input data structure in two dimensions is considered, leading to improved outcomes in picture and voice applications; this method is, therefore, well-suited to two-dimensional audio data. To train, CNNs require fewer inputs than other models. Prediction probabilities may be generated using a softmax layer and a pooling layer in convolutional neural networks, where information is filtered using a window of varying sizes across the text. Due to severe overfitting in the model, which may be resolved during training, convolutional neural networks acquire greater accuracy but not dramatically. When fed an appropriately sized weight matrix, LSTM can do calculations that would be impossible on a regular computer. Figure 3 visually represents these model relationships and the procedure for establishing connections and the flow of data-processing operations. After finishing conducting and setting up these models, the outcome is printed off directly from the code.

A dimension is considered a variable if its value is ‘None’. In our model, the ’None’ dimension always refers to the batch size, which does not need to be specified in any particular way. The embedding layer called the embedding function, starts with random weights. This embedding layer will learn the embedding for every word in the training dataset. Next, the hybrid models integrate two well-known deep learning models, specifically RACNN and Bi-LSTM (Mitkov, 2022), to take advantage of the benefits of the two different network architectures during sentiment analysis. Finally, the output layer is equipped with a ReLU activation function. The process requires us to calculate the comparison between the active user c_a, and their neighboring user c_n, which is obtained using the cosine metric by Eq. (18). In our case, the neighbors of user c_a are users who have rated the same items as user c_a in a similar way or have scores on the same items that are similar.

Dataset

The proposed model is evaluated by applying it to three sets of multi-criteria ratings, movie, restaurant, and product recommendation.