Computational linguistics based text emotion analysis using enhanced beetle antenna search with deep learning during COVID-19 pandemic

Data pre-processing

Primarily, the presented CLSA-EBASDL method performs data preprocessing to transform the information into a compatible format. Data analysis application requires data preprocessing to eliminate the redundant data for increasing the learning procedure of the classification model for improved performance. Redundant data describes any data without contribution or contributes minimum to forecasting the target class, but it raises the dimension of the feature vector and therefore presents superfluous computation difficulties. As a result, the accuracy of the classifier model is degraded if inappropriate or no preprocessing is performed. Therefore, data preprocessing or cleaning is carried out before encoding (Thanarajan et al., 2023). In this study, Python’s NLP tool is utilized for preprocessing the tweeter dataset. Firstly, the texts are translated into lowercase, followed by link removal, punctuation, and HTML tags. Next, lemmatization and stemming models are carried out to clean the text, and finally, stopwords are detached.

• URL links, numbers, tags, and punctuation removal: This process does not contribute to enhancing the performance of the classification since they provide no further meaning for the learning model and raise the difficulty of feature space, thus assisting to decrease the feature space.

• Convert to lowercase: This process minimizes the difficulty of the feature subset as ‘go’ and ‘Go’ are considered distinct features in the ML model, thereby converting to lowercase would be ‘go’. The model considers lower and uppercase words as dissimilar words that affect the training model and classification accuracy.

• Stopwords removal: Stop word is often utilized words that provide no beneficial information for analysis namely ‘the’, ‘is’, ‘a’, and ‘an’ are detached.

• Stemming and lemmatization: This process aims to reduce the derivationally correlated forms and inflectional forms of words into a base form. For instance, ‘walks, ‘walking’, and ‘walked’ are transformed into the root word ‘walk’.

Word embedding

In this study, the BERT process is applied to the word embedding process. Word embedding depends on the distributed hypothesis of word representation. The word embedding characterizes natural language words as a lower-dimension vector representation that computers might understand (Kumari, 2022). The semantic relevance of words is evaluated by the comparison between vectors. At this point, it is widely employed in NLP tasks like BERT, Word2Vec, Glove, and so on. There exist two approaches to employing pre-trained language representation for downstream tasks: fine-tuning and feature-based models. Even though BERT is popularly applied in a fine-tuning model for NLP tasks, it can be utilised as a feature-based model and utilize the encoder for text representation. Similar to BERT, the WordPiece tokenizer has been utilized for the sequence of input text. The experiment shows that the WordPiece tokenizer is more efficient than the natural tokenizer which represents the technique of word segmentation depending on comma, space, and other punctuations, as well as the CoreNLP toolkit, which is widely employed in this study. BERT could express the tokenized words as the corresponding word embeddings, and the sentences are inputted as BERT modules, and the sentence vector representations of all sentences are attained. (1)${\displaystyle r_m=BERT\left(s_m\right)s_m\in S,m\in \left[0,M\right].}$

In Eq. (1), M denotes the sentence count, m indicates the index of the sentence, s_m represents the text of the mth sentence, and S shows the group of sentences, r_m indicates the sentence vector. Then, sentence vectors or word embeddings are utilized as input to the extractor and abstractor.

Sentiment classification process

In this study, the ABiLSTM algorithm is used for sentiment classification purposes. It uses past datasets of sun irradiance from the target location as input features (Wang et al., 2019). The input can be signified by vector X^T, vector Y^T shows the corresponding input data, and vector Y^T+θ characterizes the sentiment classification.

(2)${\displaystyle X^T=\left(X_{1^{{}^{\prime }}}{X}_2,X_3\dots ..X_T\right)}$ (3)${\displaystyle Y^T=\left(Y_{1^{{}^{\prime }}}{Y}_2,Y_3\dots ..Y_T\right).}$

Now, T denotes the overall length of the time step and θ indicates the future time step. Since there exists no expressive dataset beforehand, the time window (w), and the input is set, X(t − w), X(t − w + 1), …, X(t − 1) are applied for calculating Y(t + θ) for all the tasks, in which Δ denotes the time frame in advance of prediction Y indicates the prediction of input data utilizing a deep neural network f on a formerly observed real-time dataset. (4)${\displaystyle Y\left(t+\Delta \right)=f\left(X\left(t-w\right),X\left(t-w+1\right),\dots ,X\left(t-1\right)\right).}$

Previous datasets at (t − w) time are signified as I_rr(t − w), and the input parameter is characterized in the following expression for sentiment classification. (5)${\displaystyle \left\{I_{rr}\left(t-w\right),I_{rr}\left(t-w+1\right),\dots ,I_{rr}\left(t-1\right)\right\}.}$

The features of autocorrelation and partial autocorrelation data were utilized for establishing the window size for the lag time sequence. The i- th hidden layer L, where the i value is fixed in model tuning, is characterized as L_i. In this study, the framework utilized for prediction is the attention-based BiLSTM-NN, which is composed: of encoder, attention, and softmax layers. Figure 2 demonstrates the infrastructure of BLSTM. The BLSTM acts as an encoding layer. The attention layer exploits the hidden output from these layers beforehand, creating a context vector to generate the scoring function. Subsequently, the classification of data performs decoding at the FC layer or transferred to the dense layer.

The recurrent neural network has been applied to model consecutive datasets in various problems. However due to problems with exploding or vanishing gradients, RNN is not able to learn long-term dependency. To overcome this problem, the LSTM network is recommended and constructed based on RNN. Cell memory state and three gating mechanisms encompass an LSTM’s architecture:

(6)${\displaystyle f_t=\sigma \left(W_f\left[h_{t-1^{{}^{\prime }}}{x}_t\right]+b_f\right)}$ (7)${\displaystyle i_t=\sigma \left(W_i\left[h_{t-1^{{}^{\prime }}}{x}_t\right]+b_i\right)}$ (8)${\displaystyle o_t=\sigma \left(W_o\left[h_{t-1^{{}^{\prime }}}{x}_t\right]+b_o\right)}$ (9)${\displaystyle C_t=f_t\mathrm{\ast }{C}_{t-1}+i_t\mathrm{\ast }tanh\left(W_c\left[h_{t-1^{{}^{\prime }}}{x}_t\right]+b_c\right)}$ (10)${\displaystyle h_t=o_t\mathrm{\ast }tanh\left(C_t\right).}$

The LSTM cell biases (b_j, b_f, b_o) and weighted matrices (W_j, W_f, W_o) correspondingly represent the parameters of the input, forget, and output gates, The symbol ∗ denotes the sigmoid function and the element-wise multiplication. The word embedding of LSTM cell input is characterized as x_t and the hidden state vector as h_t..

BiLSTM: The input is processed in sequential order using LSTM, which results in the effect of the previous input only and not the future one. The BiLSTM model was developed to make the model can also be influenced by future values. The LSTM processing chain is duplicated, which allows the input to manage reverse and forward time series, permitting the network to regard the future context of the network. The concluding output, h_t, of BiLSTM at the step t is given below: (11)${\displaystyle h_t=\left[f{h}_t+b{h}_t\right].}$

The attention module is used for considering the sensitive design variables. In real-time, the BiLSTM or LSTM will output a hidden h_t layer at every time step.

The h_t vector is intended into a single layer MLp that learns hidden representations u_t. Next, u_t and parameter context uw, a scalar significance value for h_t is calculated. Lastly, the attention-based models exploit a softmax function to compute the weighted mean of state h_t and it is shown below:

(12)${\displaystyle u_t=tanh\left(W_w{h}_t+b_w\right)}$ (13)${\displaystyle a_t=\frac{e^{\left(u_t^T{u}_w\right)}}{\sum _t^{}{e}^{\left(u_t^T{u}_w\right)}}}$ (14)${\displaystyle c=\sum _t^{}{a}_t{h}_{t^{\prime }}.}$

An FC softmax layer is applied as a classification. Vector c is utilized as the feature for irradiation prediction: (15)${\displaystyle \overrightarrow{y_i}=softmax\left(W_{c}C+b_c\right)}$

$\overrightarrow{y_i}$ shows the model prediction value, W_cC indicates the weighted matrix, and b_c denotes bias: (16)${\displaystyle L=-\sum _{}^{}{y}_{i}log\overrightarrow{y_i}.}$

In Eq. (16), y_i indicates the observed irritation and $\overrightarrow{y_i}$ denotes the model predictive irradiation. The BP model derives the loss function for the whole set of parameters, and the SGD model is utilized for updating the model parameters.

Hyperparameter tuning process

Lastly, the EBAS method is exploited for optimum hyperparameter adjustment of the ABiLSTM model. In BSA, we consider the search for food resources (viz., locations with a maximal intensity of food smell) as an optimization issue for mathematical modelling of the behaviour of a beetle (Al Banna et al., 2021). The maps of odor concentration in the atmosphere correspond to the value of the objective function. The aim is to search for the maximum odor concentration viz., the food resource. Consider g(x) refers to the function demonstrating the odor concentration at point x. The maximal odor concentration corresponds to the solution of subsequent optimization issues using the linear dissimilarity constraint

${\displaystyle \underset{x}{max}g\left(x\right)}$ (17)${\displaystyle Subjectto:x_{\min }\le x\le x_{\max }.}$

Assume at t time instant, the beetle search at location x_t. The concentration of odor is represented as (x_t), at the existing position. By taking the second phase, the beetle measures the concentration of odor in all directions through its antennae. Consider the antennae are positioned in opposite directions, and arbitrarily produced $\overrightarrow{b}$ characterizes the direction vector of the left antennae concerning the present location of beetle x_t.

${\displaystyle x_l=x_t+\lambda \overrightarrow{b}}$ (18)${\displaystyle x_r=x_t-\lambda \overrightarrow{b}.}$

In Eq. (18), λ indicates the length of the antennae, x^l, and x^r denote the location vector of the left and right antennae correspondingly. But the vector might disturb the constraints of Eq. (17) due to arbitrarily made vector $\overrightarrow{b}$. Consequently, we determined a constraint set Ψ in the following ${\displaystyle \Psi =x|x_{\min }<x<x_{\max }}$

and projected the x_l and x_r vectors on the constraint set Ψ (19)${\displaystyle {\Psi }_{X_l}=P_{\Psi }\left(x_l\right),{\Psi }_{{\chi }_r}=P_{\Psi }\left(x_r\right)}$

Ψ denotes that the vector is anticipated on set Ψ. The P_Ψ function is named a projection function. (20)${\displaystyle P_{\Psi }\left(x\right)=max\left\{x_{min,}min\left\{x,x_{\max }\right\}\right\}.}$

The explanation of the projected function is simpler and computationally effective. The odor intensity at the predictable antennae position is considered as g(^Ψx_l) and g(^Ψx_r). By relating the values, the beetle takes the second step based on the subsequent formula, (21)${\displaystyle x_{new}=x_t+\delta sign\left(g\left(x_l\right)-g\left(x_r\right)\right)\overrightarrow{b}.}$

In Eq. (21), the signum function sign guarantees that the following phase is considered towards the direction of high odor concentration δ is the original step-size assumed by the beetle and proportionate to Euclidean distance among x_new and x_i. Afterwards, accomplishing the novel position, the beetle re-measures the concentration of odor; when there exists an enhancement and it remains at the novel position; or else, it returns to the preceding position, (22)${\displaystyle x_{t+1}=\left\{\begin{array}{cc}{x}_{ne{w}^{\prime }}\hfill & if,g\left(x_{new}\right)\ge g\left(xt\right)\hfill \\ x_{k^{\prime }}\hfill & if,g\left(x_{new}\right)<g\left(x_t\right)\hfill \end{array}\right..}$

Afterwards, accomplishing x_t+1, produce an arbitrary direction vector $\overrightarrow{b}$ and repeat the similar procedure until the attainment. Even though the abovementioned process is expressed for the maximized problem Eq. (17), it is transformed into the minimization problem by adapting the updating rule (23)${\displaystyle x_{new}=x_t-\delta sign\left(g\left(x_l\right)-g\left(x_r\right)\right)\overrightarrow{b}.}$

The process is summarized below:

1. Begin from arbitrary position x₀.

2. Produce an arbitrary direction $\overrightarrow{b}$ vector for left antennae concerning the existing location x₀ of the beetle.

3. Compute the location of left and right antennae (x^l and x^r) based on Eq. (18).

4. Estimate novel location x_t+1 based on Eqs. (21) and (22).

5. If obtained objective location x_G, stop. Or else, return to step 2.

To resolve the optimization issue of targeted and untargeted attacks. We determine a matrix X (24)${\displaystyle \overline{X}=\left[rcrgb\right].}$

By assuming the notation, the element of $\overline{X}$ becomes k × 5. This description of the novel matrix $\overline{X}$ allows for describing the objective function with one parameter. (25)${\displaystyle g\left(\overline{X}\right)=\mathbb{P}\left(f_{cnn}\left(\mathfrak{P}\left(X_{img},\left[\overline{X}\left[:,1\right],\overline{X}\left[:,2\right],X\left\{3,4,5\right\}\right]\right)\right),C_{real}\right).}$

In Eq. (25), the semicolon symbol in the matrix indexing is utilized for representing the whole column of the matrix. (26)${\displaystyle g\left(\overline{X}\right)=1-P\left(f_{cnn}\left(\mathfrak{P}\left(X_{img},\left[\overline{X}\left[:,1\right],\overline{X}\left[:,2\right],\overline{X}\left\{3,4,5\right\}\right]\right)\right)C_{target}\right).}$

Here, r and c take integer values and then employ the round function for converting floating-point values to the integer.

To derive the EBAS technique, the Adaptive β-hill climbing concept is integrated into the BAS technique (Zivkovic et al., 2021; Gowri, Surendran & Jabez, 2022). To provide the present solution X_i = (x_i,1, x_i,2, …, x_i,D),  A βHC iteratively creates an improved solution $X_i^{{}^{\prime \prime }}=\left(x_{i,1}^{{}^{\prime \prime }},x_{i,2}^{{}^{\prime \prime }},\dots ,x_{i,D}^{{}^{\prime \prime }}\right)$ on the beginning of 2 control operators such as 𝔑- and β-operators. The N-operator initial transmissions X_i to a novel neighbourhood solution $X_i^{{}^{\prime }}=\left(x_{i,1}^{{}^{\prime }},x_{i,2}^{{}^{\prime }},\dots ,x_{i,D}^{{}^{\prime }}\right)$ that is determined in Eqs. (27) and (28) as:

(27)${\displaystyle x_{i,j}^{{}^{\prime }}=x_{i,j}\pm U\left(0,1\right)\times N,j=1,2,\dots ,D}$ (28)${\displaystyle \mathfrak{N}\left(t\right)=1-\frac{t^{\frac{1}{K}}}{\mathrm{Max}ite{r}^{\frac{1}{K}}}.}$

In which U(O, 1) implies the randomly generated value within [0,1], x_ij defines the value of decision variables from the jth dimensional, t represents the present iteration, Maxiter signifies the maximal iteration count, N denotes the bandwidth distance amongst the present solution and their neighbour, D stands for the spatial dimensional, and the K parameter is a constant.

Then, the decision variable of novel solutions $X_i^{{}^{\prime \prime }}$ was allocated both in the existing solution and the existing range of β-operator.

(29)${\displaystyle x_{i,j}^{{,}^{\prime }}\leftarrow \left\{\begin{array}{cc}{x}_{i,r},\hfill & if{r}_8<\beta \hfill \\ x_{i,j}^{{}^{\prime }},\hfill & else\hfill \end{array}\right.}$ (30)${\displaystyle \beta \left(t\right)={\beta }_{\min }+\left({\beta }_{\max }-{\beta }_{\min }\right)\times \frac{t}{\mathrm{Max}iter}.}$

whereas r₈ refers to the random value between [0, 1], x_i,r stands for another arbitrary number selected in the feasible range of that certain dimensional problem, β_min and β_max stands for the minimal and maximal values of probability values β ∈ [0, 1], correspondingly. When the created solution $X_i^{{}^{\prime \prime }}$ is better than the present solution in consideration X_i, afterwards X_i is exchanged by $X_i^{{}^{\prime \prime }}$.

The EBAS approach proceeds with FF to realize maximal classifier performance (Motwakel et al., 2023; Alotaibi et al., 2021). It resolves a positive integer to characterize the best efficacy of the candidate result. In this case, the diminished classifier error rate is examined as FF (Mujahid et al., 2021; Rahman et al., 2023). A good solution is a decreased error rate and the worst solution obtains a superior error rate (Rahman et al., 2023; Mariselvam, Rajendran & Alotaibi, 2023).

${\displaystyle fitness\left(x_i\right)=ClassifierErrorRate\left(x_i\right)}$ (31)${\displaystyle =\frac{No.ofmisclassifiedsamples}{TotalNo.ofsamples}\mathrm{\ast }100.}$