Feature level fine grained sentiment analysis using boosted long short-term memory with improvised local search whale optimization

Research gap

• The domain dependence on sentiment terms is the major obstacle to opinion mining and sentiment analysis. A feature set may perform exceptionally well in one domain while exhibiting very poor performance in another.

• For sentiment analysis to be effective, opinion words must interact with implicit data. The implicit information determines how sentimental phrases actually work.

• People express their ideas in many ways. Every person has a unique opinion since everyone has a distinct manner of thinking and expressing themselves.

• Typographical flaws can occasionally make it difficult to gather opinions.

• Natural language overhead, such as ambiguity, co-reference, implicitness, inference, etc., made sentiment analysis tools more difficult to use.

• It might be difficult to classify sentences as positive or negative, very positive, very negative, and neutral in terms of opinion mining (OM) at the sentence level since every writer has a distinct writing style and because a sentence may include positive or negative, very positive, very negative, and neutral concepts.

Proposed methodology

In this article, ILW-LSTM, a deep learning approach, is suggested for the feature-level sentiment analysis. This method includes five main stages such as ‘data acquisition’, ‘data pre-processing’, ‘feature extraction’, ‘feature selection’, last ‘sentiment classification’. At first, the data is gathered from the taken dataset then the textual data is taken for sentiment analysis to determine the essential aspects. The textual data is subjected to pre-processing, which includes ‘white tokenization’, ‘lemmatization’, and ‘snowball stemming’. Tokenization is a technique for dividing text into smaller bits. Text in huge amounts is broken down into words or phrases. Depending on the issue, precise data criteria are defined to divide the text content into pertinent tokens. One of the most used data pre-processing techniques is lemmatization. The suggested method makes use of the long-term frequency-based modified inverse class frequency for textual feature extraction. Subsequently, the Levy flight-dependent mayfly optimization algorithm (LFMO) is employed to improve the classification accuracy. This method subjects to the training model and chooses the optimal set of features. The final step is to add the selected characteristics to the proposed ILW-LSTM for polarity classification. The output of the ILW-LSTM represents the classes as ‘positive’, ‘negative’, ‘very positive’, ‘very negative’, and ‘neutral’ (sentiments) of the entered data. Figure 1 displays the general stream diagram of the suggested feature-level sentiment analysis.

Data acquisition

A rating is created by combining information from the review about the sentence subjects and attitudes. Below are the pre-processing steps for the customer reviews that will be processed further.

Pre-processing

Pre-processing procedures are employed to remove undesirable datasets from the group. In this case, pre-processing is carried out as part of the sentiment analysis data preparation procedure. The three pre-processing operations are ‘white space tokenization’ ‘lemmatization’, and ‘snowball stemming’.

Feature extraction

The method of feature extraction is crucial to sentiment analysis. It is mainly employed to convert raw or original data into a low-dimensional feature. The characteristics from the text are taken out in this part for sentiment analysis. Our suggested method uses a log-term frequency-based modified inverse class frequency (LFMI) for further evaluation. Additionally, this method incorporates testing and training, with the extraction of textual elements occurring during training.

LFMO-based feature selection

The study implements feature selection using the optimization technique known as LFMO, which is described as,

The mayfly algorithm was inspired by the way that mayflies interact with one another, particularly during mating. Once the eggs hatch, then the mayflies are immediately regarded as adults. Despite how long they are alive, only the fittest mayflies tend to survive. Every mayfly in the search space has a position that correlates to a method for resolving the problem. The traditional mayfly method uses RANDOM functions to generate new variables that result in the local optimum. Here, Levy flying and the Mayfly algorithm were coupled by the researchers to increase the mayfly’s ability to seek and determine the best solution. According to the Levy flight notion, if a Levy flight-based technique is used for structural analysis, it provides quick convergence rate and does not require any derivative information. It also improves the local search avoidance and local trapping of the ideal solution. The following stages are necessary for the proposed mayfly optimization method to function:

Stage 1: There should be two sets of mayflies, one for the male population and the other for the female. Then, each and every mayfly is randomly positioned in the problem space as a candidate solution, indicated by the d-dimensional vector $Q_{Gx}=\left(Q_{G1},Q_{G2},\dots ,Q_{Gd}\right).$ Then, depending on the established goal function $F\left(C_T\left(Q_{Gx}\right)\right)$, the performance is evaluated (Nagarajan et al., 2022; Zervoudakis & Tsafarakis, 2020).

Stage 2: The initialization of a mayfly’s velocity $vel=\left(ve{l}_1,\dots ,ve{l}_d\right)$ occurs during a positional change. Hybrid interplay between individuals and social flying experiences determines its path. Every mayfly adapts to change its route to match its current personal optimal position $\left(Pbest\right)$. Additionally, it also changes dependent on the best position that any other mayfly in the swarm has obtained thus far $\left(Gbest\right)$.

Stage 3: With initialized velocities of $ve{l}_{mx}$, the population of male mayflies is designated as $Q_{Gmx}\left(x=1,2,\dots ,IG\right)$. The male population mayflies, which congregate in swarms, indicate that each mayfly’s position changes on its personal experience and that of its neighbors. $Q_{Gx}^T$ is taken to be mayfly x’s current location in search space at time step T, and it changes as velocity $ve{l}_x^{T+1}$ is added to the current position. This expression is written exactly as it is provided here.

(5) $$Q_{Gmx}^{T+1}=Q_{Gx}^T+ve{l}_x^{T+1}$$

A few meters above the water, with $Q_{Gxm}^{0}U\left(Q_{Gmmin},Q_{Gmmax}\right)$, male mayflies are thought to be present and engaged in a nuptial dance. Since they are always moving, it may be inferred that these mayflies don’t have remarkable speeds. The velocity x of a male mayfly is therefore determined as follows.

(6) $$ve{l}_{xy}^{T+1}=g\ast ve{l}_{xy}^T+m_1{e}^{-\beta r_p^2}\left(Pbes{t}_{xy}-Q_{Gmxy}^T\right)+m_2{e}^{-\beta r_g^2}\left(Gbes{t}_y-Q_{Gmxy}^T\right)$$

Here, the contributions of the social and cognitive components are scaled up using positive attraction constants $m_1$. $ve{l}_{xy}^T$ relates to the mayfly x’s velocity in dimension $y=1,\dots ,i$ at time step T. $Q_{Gmxy}^T$ specifies the mayfly’s $x^{th}$ position in dimension j at time step $T,m_1$. Likewise, $r_g$ stands for the Cartesian distance between $Q_{Gx}$ and $Gbest$, whereas $r_p$ represents the distance between $Q_{Gx}$ and $Pbes{t}_x$. Additionally, $Pbes{t}_x$ designates the mayfly in its finest location yet. The personal optimal position $Pbes{t}_{xy}$ at the following time step $T+1$ is calculated based on the minimization issues under consideration.

(7) $$Pbes{t}_x=\{\begin{array}{c}{Q}_{Gmx}^{T+1},iff\left(Q_{Gmx}^{T+1}\right)<F\left(Pbes{t}_x\right)\\ remainsthesame,otherwise\end{array}$$

The formula for the $Gbest$ position at step T time is given below.

(8) $$\begin{array}{cc}Gbest\in & \left\{Pbes{t}_1,Pbes{t}_2,\dots ,Pbes{t}_I,|F\left(Cbest\right)\right\}\hfill \\ & =min\left\{F\left(Cbes{t}_1\right),F\left(Cbes{t}_2\right),\dots ,F\left(Pbes{t}_{IG}\right)\right\}\hfill \end{array}$$

Here is the formula for calculating the Cartesian distance.

(9) $$\parallel Q_{Gmx}-X_x\parallel =\sqrt{\sum _{y=1}^i{\left(Q_{Gmxy}-X_{xy}\right)}^2}$$

For the algorithm to work as intended, the finest mayflies in the swarm must repeatedly execute the up-and-down nuptial dance. Therefore, it is necessary to continuously adjust the velocity of these best mayflies, which is computed as follows.

(10) $$ve{l}_{xy}^{T+1}=ve{l}_{xy}^T+d\times b$$

Here, d stands for the coefficient of nuptial dance, and b is the random rate between [1, 1].

Stage 4: The population of female mayflies is initialized $Q_{Gfx}\left(x=1,2,\dots ,IG\right)$ with velocities $ve{l}_{fx}$ in this stage. Female mayflies often do not swarm as males do. Instead, it usually flies in the direction of its male peers to mate. $Q_{Gfx}^T$ is taken to be the location of female mayfly x in search space at step T time, and its position changes as a result of adding velocity $ve{l}_x^{T+1}$ to the current position.

(11) $$Q_{Gfx}^{T+1}=Q_{Gfx}^T+ve{l}_x^{T+1}$$

$Q_{Gxm}^{0}U\left(Q_{Gmmin},Q_{Gmmax}\right)$ prevents one from randomizing the attraction process in this case. Thus, it is agreed that the model would be a deterministic process. Since there are issues with minimization, their velocities are calculated as given below.

(12) $$ve{l}_{xy}^{T+1}=\{\begin{array}{c}g\ast ve{l}_{xy}^T+m_2{e}^{-\beta r_{mf}^2}\left(Q_{Gmxy}^T-Q_{Gfxy}^T\right)ifF\left(Q_{Gfx}\right)>F\left(Q_{Gmx}\right)\\ g\ast ve{l}_{xy}^T+Fl\times b,ifF\left(Q_{Gfx}\right)\le F\left(Q_{Gmx}\right)\end{array}$$

Here, $b$ stands for the random value in the interval [1, 1], and $Fl$ stands for the random walk coefficient. Equation (15) is used to get this value.

Stage 5: A mayfly candidate solution’s velocity is determined in this stage using the Levy flight method. To determine the mayfly candidate’s speed, Eq. (13) is utilized.

(13) $$ve{l}_{xy}^{T+1}=\{\begin{array}{c}ve{l}_{max},ifve{l}_{xy}^{T+1}>ve{l}_{max}\\ -ve{l}_{max},ifve{l}_{xy}^{T+1}<-ve{l}_{max}\end{array}$$

In this step, the global finest component’s location is changed using the Levy flight method. Although the Levy flying approach has so far been applied for exploration, it is connected to a particular search. Here, $ve{l}_{max}$ is determined using the formula:

(14) $$ve{l}_{max}=levy\left(\lambda \right)\ast \left(Q_{Gmmax}-Q_{Gmmin}\right)$$were, $levy(\lambda )=0.01\frac{r_5{}^{\sigma }}{{\left|r_6\right|}^{1/\beta }}$. $levy\left(\lambda \right)$ specifies the step length and incorporates the infinite variance of the Levy distribution and mean values of $1<\lambda <3$. $\lambda$ represents the step length. The distribution factor is indicated by the symbol.

Stage 6: The gravitational constant, or $g$ value, can be thought of as a stable integer between $(0,1]$.

(15) $$g=g_{max}-{\displaystyle \frac{g_{max}-g_{min}}{iteratio{n}_{max}}\times Iteration}$$where $Iteration$ represents the algorithm’s current iteration, $iteratio{n}_{max}$ denotes the max number of iterations, and $g_{max},g_{min}$ signify the max and min values that may be considered for the gravity coefficient, respectively.

Stage 7: Mayflies mate and the young are inspected. The crossover operator described here discusses the mating behavior of the mayflies. Each parent is chosen from the male and female population using the same selection method, which is the attractiveness of females to men. One might choose the parents specifically based on fitness function or at random. In terms of the fitness function, the best female mates with the best male, the second-best female mates with the second-best male, etc. This crossing produces two offspring, for which the following formulation is provided.

(16) $$offspring1=L\times male+\left(1-L\right)\times female$$

(17) $$offspring2=L\times female+\left(1-L\right)\times male$$

Here, the male stands for the male parent, the female for the female parent, and L is a random value falling inside a certain range. The offspring’s starting velocity is set at zero. Finally, a new subset of articles with additional educational elements is the outcome of this step. Figure 2 depicts the flow chart for the LFMO-based feature selection algorithm.

Sentiment classification

The selected features from the feature selection process are inputted into the suggested ILW-LSTM classifier, to classify sentiment. A unique variety of RNNs called LSTM was created to address the vanishing and exploding gradients problem that recurrent neural networks encounter. Like other RNN types, LSTMs produce their output based on the data from the current time step and the output from the previous time step and then transmit their current output to the subsequent time step. A memory cell that can preserve its state for any length of time makes up each LSTM unit, as well as three non-linear gates: an input gate $\left(i{n}_t\right)$ a forget gate $\left(f{g}_t\right)$, and an output gate $\left(ou{t}_t\right)$. Information entering and leaving the memory cell is managed by these gates. Assume $tanh(.)$ as hyperbolic tangent function, $\odot$ as dot product, and $\sigma (.)$ as a sigmoid function applied to elements. At time t, $i{n}_{t}andhi{d}_t$ represent the input and hidden state vectors, respectively. Gate weight matrices are displayed by X and Y, while bias vectors are denoted by bias. By producing a number in the range [0, 1], the forget gate determines what data has to be forgotten.

(18) $$f{g}_t=\sigma \left(Y_{fg}hi{d}_{t-1}+X_{fg}i{n}_t+bia{s}_{fg}\right)$$

The following equations are used by the input gate to compute $i{n}_t$ and ${\tilde{c}}_t$, combine them, and decide what additional information should be stored.

(19) $$i{n}_t=\sigma \left(Y_{in}hi{d}_{t-1}+X_{in}i{n}_t+bia{s}_{in}\right)$$

(20) $${\tilde{c}}_t=tanh\left(Y_{c}hi{d}_{t-1}+X_{c}i{n}_t+bia{s}_c\right)$$

(21) $$c_t=f{g}_t\odot c_{t-1}+i{n}_t\odot {\tilde{c}}_t$$

The following equations determine which components of the cell state should be output by the output gate.

(22) $$ou{t}_t=\sigma \left(Y_{out}hi{d}_{t-1}+X_{out}i{n}_t+bia{s}_{out}\right)$$

(23) $$hi{d}_t=ou{t}_t\odot tanh\left(c_t\right)$$

The optimization of the LSTM using ILW is described as follows.

Optimization of LSTM based on ILW

The foundation for WOA is the humpback whales’ hunting method. The biggest member of the baleen whale family is the humpback whale. The unique hunting technique used by humpback whales is one of its uncommon features. These whales can recognize the victim and surround them. Equations (24) and (25) illustrate how the encircling movement is expressed

(24) $$W=\left|Q.D^{\ast }\left(i\right)-D\left(i\right)\right|$$

(25) $$D\left(i+1\right)=D^{\ast }\left(i\right)-P.W$$

Here, P and Q are the vectors of the coefficient, ‘.’ designates the element-by-element multiplication, and D is the coordinates that detail the best outcome that was attained. The coefficient vectors P and Q are calculated via the following equation.

(26) $$P=2b.randv-b$$

(27) $$Q=2.randv$$where b is exactly decreased across all iterations from 2 to 0, and $randv$ is a random vector between [0, 1]. There are two ways to mathematically express the humpback whale bubble-net approach; the decreasing encircling mechanism lowers the value of b in Eq. (26). The first assessment made by the spiral updating position strategy is the separation between the prey’s position at $\left(i^{\ast },j^{\ast }\right)$ and the whale’s position at $\left(i,j\right)$. The spiral equation connecting the position of the whale and its prey is then calculated to replicate the helix-shaped group of humpback whales, as shown in Eq. (28)

(28) $$D\left(i+1\right)=W^{\mathrm{\prime }}.e^{al}.\cos \left(2\pi l\right)+D^{\ast }\left(i\right)$$

(29) $$D\left(i+1\right)=\{\begin{array}{cc}{D}^{\ast }\left(t\right)-P.W\hfill & ifr<0.5\\ W^{\mathrm{\prime }}.e^{al}.\cos \left(2\pi l\right)+D^{\ast }\left(i\right)& ifr\ge 0.5\end{array}$$

Equation (29) illustrates the statistical formula for updating the answer depending on the shrinking encircling technique or the spiral method, where $r$ is a random value between [0, 1]. $D$ may be used to look for the vector of the prey. To dissuade searchers from the reference whale, Vector $D$ has a random rate in the range [1, 1]. Its numerical expression is given as follows

(30) $$W=\left|Q.D_{randam}-D\right|$$

(31) $$D\left(i+1\right)=D_{randam}-P.W$$where $D_{randam}$ is the random position vector chosen among the available solutions and the solution representation of the proposed algorithm is shown in Fig. 3.

Initially, features within the chromosome should be chosen. As an outcome, the length of the chromosome is 10, pertaining to ten aspects that are depicted in Fig. 3. Considering the chromosome, each feature is assigned a unique value, i.e., one when selected or 0 otherwise. Following that, original chromosomes are chosen at random to form the population. After that, several search agents were selected for mating in order to produce offspring chromosomes based on the fitness value associated with each solution (i.e., chromosome). Later, a fitness function is employed for calculating the fitness value. The lower the fitness value, the better the solution. The “search agents” are then selected from the current population based on the fitness function. The nature of the ILW-LSTM lies in the hypothesis that mating two best solutions could generate an optimal solution.

Dataset description

Prompt Cloud generated this dataset, which has been combined into a usable dataset. It includes Amazon-generated phone reviews. The majority of reviewers have awarded unlocked mobile phones four-star and three-star ratings. To identify patterns in reviews, ratings, and price as well as the relationships between them, PromptCloud examined 400 k customer reviews of unlocked mobiles available on Amazon.com. Below is a list of the fields.

• Title of the product

• Trade mark

• Cost

• Ranking

• Review text

• How many consumers thought the review was beneficial

The data was attained in December 2016 by the flatterers used to provide our data extraction services. The reviews are around 230 characters long on average. It also revealed that longer reviews are frequently more beneficial and that there is a favourable relationship between price and rating.

Hyper-parameters setting

The accuracy was optimized and the hyper-parameter was tuned using the improvised local search whale optimization approach. The hyper parameters measures in the suggested approach are shown in Table 1.

Experimental setup

To create deep learning models, a variety of tools and libraries are available. Keras is the preferred tool. Tensor Flow was used as Keras’ backend since it was GPU-compatible. The following computer specs were used for deep learning experiments: The Python program is used to execute the experiment on a computer with 12 GB of RAM and an Intel TM core (7M) i3-6100CPU running at 3.70 GHz. The model was built and trained using Python programming. The system’s performance is shown by comparing its evaluation metrics to those of other systems currently in use.

Evaluation metrics

Similar studies will compare the classification achievements from sentiment classification with the confusion matrix measurements to demonstrate the accuracy of the methodology. The confusion matrix is employed to determine measurement values for accuracy, precision, recall, and F1-score. One class must always be designated as positive and the other as negative when dealing with two-class categorization issues. Take into account the test set, which consists of both positive and negative samples. The goal of every classifier is to assign a class to each sample; however, certain classifications might not be acceptable. The number of ‘true positive’, ‘true negative’, ‘false positive’, and ‘false negative’ samples are counted to evaluate the performance of the classifier.

• True Positive: Number of examples when both the correct class label and the anticipated class label are positive.

• True Negative: The number of instances where the actual class label is right while the projected class label is false.

• False Positive: Number of cases where the actual class label is inaccurate while the projected class label is positive.

• False Negative: Number of cases where the actual class label is inaccurate while the projected class label is negative.

Our suggested ILW-LSTM model is put to the test using standard performance evaluation. The following metrics can be used to evaluate the model:

• Accuracy: It may be identified as the percentage of all accurately classified occurrences in the overall count of instances.

$$\mathit{A}\mathit{c}\mathit{c}\mathit{u}\mathit{r}\mathit{a}\mathit{c}\mathit{y}=\left({\displaystyle \frac{\left(\mathit{T}\mathit{p}+\mathit{T}\mathit{n}\right)}{\left(\mathit{T}\mathit{p}+\mathit{T}\mathit{n}+\mathit{F}\mathit{p}+\mathit{F}\mathit{n}\right)}}\right)$$

• Precision: It is characterized by the proportion of correctly classified positive occurrences to the total number of positively anticipated instances.

$$\mathit{P}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}=\left({\displaystyle \frac{\mathit{T}\mathit{p}}{\left(\mathit{T}\mathit{p}+\mathit{F}\mathit{p}\right)}}\right)$$

• Recall: It is described as the fraction of suitably categorized positive instances from the total count of positive instances.

$$\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}=\left({\displaystyle \frac{\mathit{T}\mathit{p}}{\left(\mathit{T}\mathit{p}+\mathit{F}\mathit{n}\right)}}\right)$$

• F1-score: It can be defined as the average harmonic between precision and recall.

$$\mathit{F}{1}_{\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}}=\left({\displaystyle \frac{2\left(\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}\times \mathit{p}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}\right)}{\mathit{R}\mathit{e}\mathit{c}\mathit{a}\mathit{l}\mathit{l}+\mathit{p}\mathit{r}\mathit{e}\mathit{c}\mathit{i}\mathit{s}\mathit{i}\mathit{o}\mathit{n}}}\right)$$

Experimental results

Accuracy, recall, precision and F1-score are calculated according to the obtained confusion matrix that is given in Fig. 4:

We noticed that the suggested hybrid model’s classification results can be distinct throughout the testing of the proposed models. In other words, the identical input was categorized as negative by the suggested model. The accuracy of the training and validation datasets compared to the training epochs is shown in Fig. 5. The values for the epoch are on the x-axis, while the values of accuracy are on the y-axis. The accuracy of the proposed model was found to be 97 percent. To assess the false positive and true positive of our methods, the authors like to draw the receiver operating characteristic (ROC) curves for our systems in the additional Fig. 6.

The ROC curve for the suggested method is displayed in Fig. 6. The ROC curves demonstrate how well our model performs at various 0–1 thresholds. Based on the ROC for the ILW-LSTM approach, this work discovered that the suggested technique has good effectiveness for feature-level sentiment analysis.

Metrics for class-based categorization for the suggested models are shown in Table 2. Aside from accuracy, measures like F1-score, recall, and precision were also considered for evaluating the effectiveness of the model because the dataset utilized in the research is imbalanced. The suggested models performed best in terms of class-based F1-score, precision, and recall score for the negative class. Figure 7 illustrates the precision, F1-score and Recall values for the classification outcome.

Comparative analysis

To demonstrate the efficacy of the proposed model’s performance, the findings from the suggested work are compared to those from several other current methodologies. These findings clearly proved the value of the ILW-LSTM technique and encouraged several research projects to create a sentiment categorization system based on deep learning algorithms. As a consequence, the findings of the suggested model have been compared with those of current methods in this section.

A comparison of the different methods using the suggested methodology is shown in Table 3 for clarity. Accuracy, precision, recall, and F1-score are the factors used in the comparison analysis. In this case, the suggested methodology values for accuracy, precision, recall, and F1-score are 97.61%, 97.24%, 99.35%, and 98.27%, respectively, when compared to those of CNN, LSTM, Bi-LSTM, CNN-LSTM, and ConvBi-LSTM. On PromptCloud datasets, the ILW-LSTM model performed far better than other deep learning models. Figure 8 shows a comparison of the accuracy, precision, F1-score and recall of several methodologies.