Research on emotion classification technology of movie reviews based on topic attention mechanism and dual channel long short term memory

Extraction and representation of movie theme

A film review usually focuses on a particular film theme to express the feeling of watching the film. Usually, the words describing the film theme are nouns, such as special effects, plot, sound effects, etc. Each topic is an element to measure the actual meaning of the film. Its extraction in the corpus can be combined with a word clustering algorithm and the expertise of film experts. This research first uses a word clustering algorithm to cluster all nouns in the film review corpus, then uses the clustering center as the candidate subject words. Then, it combines the professional knowledge of film experts to make corrections so that the extracted subject words conform to the characteristics of the film field. The specific extraction process is shown in Fig. 1.

Afterword clustering and screening by film experts, the film theme set contains eight themes, namely, “plot,” “special effects,” “original work,” “music,” “thought,” “theme,” “acting,” and “joke.” The research paper projects each theme information of the movie into a multi-dimensional vector with continuous values. Each movie theme will have many related descriptive feature words or attribute words. This research will use the theme feature words to convert the movie theme into a vector representation form to facilitate the attention module to quickly lock the characteristics of a certain type of theme. ${\overrightarrow{V}}_{t,i}\in R^n$ represents the theme vector of the ith theme, and n represents the vector dimension. Table 1 describes the vector representation algorithm of the movie theme.

Vector representation of words

Using a word embedding algorithm to convert text information into word vectors is an important step in using deep learning to process text information. However, one of the major limitations of this algorithm is that words with similar structures but opposite emotional polarity are usually converted into similar semantic vectors, resulting in the loss of emotional information of words. To improve this limitation, this study describes a word from two aspects of semantics and emotion and inputs the emotion vector and semantic vector of the word into LSTM in a reasonable way to train a more efficient emotion classifier.

This research will use the Word2Vec tool to generate the semantic vector of words, take the movie review corpus as the training data, set the size of the co-occurrence window to 5, and set the dimension of the output vector to 150 dimensions, that is, each word will be converted into a 150-dimension semantic vector. Since most movie reviews are short texts, this study will take 100-word semantic vectors to represent a movie review. For movie reviews with less than 100 words, 150 dimensional all zero vectors will be used to supplement, to maintain the consistency of the length of the movie review sequence. So far, a movie review can be represented by 100 150-dimensional semantic vectors. A series of semantic vectors of film reviews can be expressed in Formula (1). (1)${\displaystyle T{S}_m=\left[{\overrightarrow{v}}_1,{\overrightarrow{v}}_2,..,{\overrightarrow{v}}_{100}\right].}$

The semantic vector can only represent the semantic information of words. This study uses one lot-coding to construct a word emotional vector to improve the emotional information of words. It is found that the emotional tendency of a text includes not only positive and negative emotional words but also degree adverbs, negative words, interrogative words, etc. Therefore, this study will construct a 6-dimensional emotional vector of [positive, negative, neutral, interrogative words, and degree adverbs] according to the types of words that affect the emotional tendency of the text. The position of a certain type of word will be set to 1. The rest bits are set to 0. If “bad film” belongs to negative film review emotional words, its emotional vector can be expressed as [0, 1, 0, 0, 0, 0]. Suppose a word does not belong to any dictionary in the emotional lexicon of Chinese film reviews which belongs to an unlisted word. In that case, all bits of the emotional vector is set to zero, and the emotional vector sequence of a film review can be expressed in the form of the Formulas (2)–(3).

(2)${\displaystyle e_i^n=\left\{\begin{array}{c}1,w\in \text{dictionary}\hfill \\ 0,w⁄\in \text{dictionary}\hfill \end{array}\right.}$ (3)${\displaystyle T{S}_e=\left[{\overrightarrow{e}}_1,{\overrightarrow{e}}_2,\dots ,{\overrightarrow{e}}_{100}\right].}$

Here, $e_i^n$ represents the nth position of the emotional vector of the ith word, TS_e represents the emotional vector of a film review. Like the semantic vector, the emotional vector of 100 words is also taken to represent the emotion of a film review. For film reviews with less than 100 words, a 6-dimensional zero vector will be used to supplement.

DCLSTM-TAM model construction

This research proposes a movie review emotion classification model (DCLSTM-TAM) based on topic attention mechanism and dual channel LSTM. The overall framework of this model is a coding and decoding structure. The coding part comprises the input and dual channel LSTM layers, and the decoding part comprises eight movie topic attention modules. First, based on the emotional lexicon of movie reviews constructed in this study, all words are coded into one pot, and the emotional vector of words is built. The semantic vector of words is obtained using the word2Vec tool as the input of the LSTM network. Then quantize the film theme information and construct multiple attention modules. Finally, the whole connection layer is used to classify the theme emotion of film reviews. The specific thematic attention mechanism and the construction framework of the dual channel LSTM model are shown in Fig. 2.

(1) Output of encoder

It can be seen from Fig. 2 that the encoder mainly completes the extraction of semantic and emotional features of film reviews. First, the word2Vec tool and the emotion vector representation method proposed in this study complete the word embedding vector representation of film reviews. Then, the LSTM network is used for feature extraction to obtain the hidden layer state at each time point. For a film review with a word length of w, the hidden layer matrix output by the hidden layer can be expressed as $H=\left[{\overrightarrow{h}}_1,{\overrightarrow{h}}_2,\dots ,{\overrightarrow{h}}_t,\dots ,{\overrightarrow{h}}_m\right]$, where ${\overrightarrow{h}}_t$ represents the hidden layer state of the t time step, and its dimension is the same as the output dimension of LSTM. Let ${\overrightarrow{h}}_t\in R^f$ have f dimension, then H ∈ R^f∗m. The encoder will input the hidden layer state matrix H into each movie theme attention module respectively and generate the overall representation TS₁ of the movie review. Its calculation is shown in Formula (4). (4)${\displaystyle T{S}_1=\frac{1}{n}\ast \sum _{t=1}^n{h}_t}$ (2) Topic attention module calculation

It can be seen from Fig. 1 that the decoder mainly completes emotion classification based on film review topics and is composed of multiple topic attention modules. Each topic attention module includes attention, probability, and reconstruction. The number of theme attention modules here is the same as that of the extracted film review topics, which means that one film theme corresponds to one theme attention module. When the input film review involves multiple topics, the hidden layer state matrix H obtained by the encoder will be sent to the corresponding topic attention module respectively. Different topic attention modules correspond to additional topic information, the film review topic vector. When the ${\overrightarrow{v}}_{t,i}$ The attention module model is trained to perform different attention calculations.

The function of the attention part is to enable the model to pay attention to the feature information of specific topics when extracting features from film reviews to obtain the text representation of specific topics of film reviews. First, the input hidden layer state matrix H is spliced with a specific topic vector ${\overrightarrow{v}}_{t,i}$; Then, we calculate attention and obtain the text representation ${\overrightarrow{v}}_{c,i}$ with the ith film review theme through weighted average calculation. The specific calculation method of attention is shown in Formulas (5)–(6).

(5)${\displaystyle e_{t,i}=tanh\left(W_{\alpha ,i}\left[{\overrightarrow{h}}_t\cdot {\overrightarrow{v}}_{t,i}\right]+b_{\alpha ,i}\right)}$ (6)${\displaystyle {\overrightarrow{v}}_{c,i}=\sum _{i=1}^m\frac{exp\left(e_{t,i}\right)}{\sum _{k-1}^{m}exp\left(e_{k,i}\right)}{\overrightarrow{h}}_t.}$

Here W_α,i is the weight matrix of the ith topic attention, and W_α,i ∈ R^n+s, b_α,i is the offset term of the ith topic attention.

(3) Calculation of probability part

The probability part mainly calculates the probability that the text representation ${\overrightarrow{v}}_{c,i}$ of the ith film review topic belongs to a certain category. After the text representation ${\overrightarrow{v}}_{c,i}$ The review topic is input into the full connection layer of the film, and the probability of each category of the specific film review topic text to be classified can be calculated through the softmax function. This study has set three types of emotional classification: positive, negative, and neutral. The calculation method of the emotional classification probability of the related text of the ith film review topic is shown in Formula (7). (7)${\displaystyle P_i=softmax\left(W_{q,i}{\overrightarrow{v}}_c,i+b_{q,i}\right).}$

Here, $P_i=\left[P_{i,1},P_{i,2},P_{i,3}\right]$ represents the probability that the emotional classification of the text related to the ith film review topic belongs to positive, negative, or neutral, b_q,i and W_q,j represents the weight and bias of the full connection layer, respectively.

(4) Calculation of reconstruction part

The primary function of the reconstruction part is to reconstruct the text representation ${\overrightarrow{v}}_{c,i}$ of the film review theme so that it can be a film review text representation including specific theme information, film review global information and emotional orientation information. The emotion classification probability generated by the probability calculation part is used to reconstruct ${\overrightarrow{v}}_{c,i}$. The reconstruction matrix of the specific topic i generated is $R_i=\left[R_{i,1},R_{i,2},R_{i,3}\right]$, and its reconstruction calculation is shown in Formula (8). (8)${\displaystyle R_{i,j}=p_{i,j}{\overrightarrow{v}}_{c,i},j=1,2,3.}$

DCLSTM-TAM model training

This study uses the BP algorithm to train and optimize the DCLSTM-TAM model, and the minimum related loss function is constantly adjusted to optimize the model. The minimum loss function constructed in this study consists of two-fold loss functions f₁(α) and f₂(α). The function of f₁(α) is to maximize the emotional category probability of the actual topic text in the output of each topic attention module. The function of f₂(α) is to maximize the similarity between the reconstructed representation of the topic text of the actual emotion category and the overall representation TS of the film review in the output of each topic attention module. The specific calculation method is shown in Formula (9). ${\displaystyle L\left(\alpha \right)=f_1\left(\alpha \right)+f_2\left(\alpha \right)}$ (9)${\displaystyle f_1\left(\alpha \right)=\sum _z\sum _i\sum _{j\ne c_{z,j}}max\left(0,1-p_{i,c_{z,j}}+p_{i,j}\right)}$ ${\displaystyle f_2\left(\alpha \right)=\sum _z\sum _i\sum _{j\ne c_{z,j}}max\left(0,1-T{S}_1{R}_{i,c_{z,j}}+T{S}_1{R}_{i,j}\right)}$

Where z is the zth film review sample, and c_z,j is the actual emotional category of the zth film review sample on the ith film theme.

The semantic vector of this study is initialized by word2Vec technology. The dimension of the semantic vector is set to 100 dimensions, the emotion vector is set to six dimensions, the output unit of the LSTM network is set to 200 dimensions, and the initialization of the movie theme vector and unlisted words uses a uniform distribution function to initialize randomly. A drop-out layer is added to the model to prevent overfitting during training, and its parameter is set to 0.5.

Experiments and results on the emotional thesaurus of film reviews

The influence of emotional thesaurus in the film review field on the emotional classification of film reviews

To verify the effect of the emotional lexicon in the field of film reviews built in this study on improving the emotional classification of film reviews, we will use three emotional dictionary methods to classify emotions based on the experimental data set of film reviews. Methods 1. The Chinese emotional vocabulary ontology was used as the reference dictionary for computing the emotions named M1. The Chinese emotional lexicon ontology and the emotional lexicon in film review were used as references for emotional calculation named M2. The emotional lexicon in film review was used as a reference for emotional calculation named M3. Then the emotional polarity value of emotional words in a film review is counted and accumulated. If the value is less than 0, the film review is negative; otherwise, it is positive. The experimental results of film review emotion classification based on different emotion dictionaries are shown in Fig. 3.

The above experimental results show that the emotional lexicon of film reviews constructed in this study can significantly improve the accuracy of the emotional classification. Among them, M3 classifies emotions based on the lexicon. The accuracy with M3 is 84.93% on the three test sets, which is significantly better than the M1 and M2. The emotional classification of degree and negative word dictionaries can also improve the classification accuracy to a certain extent, which shows that the emotional lexicon constructed in this study has practical and effective value.

Related experiments and results of the DCLSTM-TAM model

The influence of the DCLSTM-TAM model on the emotional classification of film reviews

To verify the rationality of the emotion vector and semantic vector input into LSTM through dual channel fusion proposed in this study and the impact of adding emotion vector on the accuracy of film review emotion classification. This research trains different LSTM networks based on the training set and test set of film reviews and classifies the emotions of film reviews. The specific comparative experimental methods are as follows:

Method 1 (LSTM): The traditional LSTM method is only a semantic vector + LSTM.

Method 2 (DCLSTM): semantic vector + emotional vector + LSTM, and the fusion method of the two vectors is to input LSTM respectively in a dual channel way, and the fusion is realized before the full connection layer is input.

Method 3 (DCLSTM-TAM): Film review emotion classification model based on topic attention mechanism and dual channel LSTM.

In this study, the accuracy rate A (accuracy) of emotion classification is taken as the experimental indicator. The accuracy changes in emotion classification of film reviews based on different LSTM networks are shown in Fig. 4.

Table 4:

Movie review & theme sentiment classification experimental data set.

Film theme	Positivity		Neutral		Negativity
	Training set	Test set	Training set	Test set	Training set	Test set
Plot	1,021	988	512	302	1121	865
Special effects	311	272	421	212	512	412
Original	221	252	102	81	212	103
Music	541	412	421	312	312	210
Thought	321	291	103	75	59	26
Theme	198	161	98	75	161	102
Acting skill	827	412	798	521	592	411
Laughing stock	211	103	421	216	312	293
Total	3,651	2,891	2,876	1,794	3,281	2,422

DOI: 10.7717/peerjcs.1295/table-4

This study compares the experimental results of the constructed DCLSTM-TAM model with the classical emotion classification model LSTM and the DCLSTM model with emotion vector. It comprehensively evaluates the model’s performance using three indicators: accuracy (P), recall (R) and F1 value. Among them, the accuracy rate P (precision) represents the proportion of correctly predicted positive samples in the actual predicted positive samples, the recall rate R (recall rate) represents the proportion of correctly predicted positive samples in the actual positive samples, and the F1 value is the evaluation index of the comprehensive accuracy rate and recall rate. This study uses these three indicators to evaluate the model’s performance, and the relevant calculation formula is shown in Formula (10).

${\displaystyle P=\frac{TP}{TP+FP}}$ ${\displaystyle R=\frac{TP}{TP+FN}}$ (10)${\displaystyle A=\frac{TP+TN}{TP+TN+FP+FN}\times 100\text{\%}}$ ${\displaystyle F_1=\frac{2\ast P\ast R}{P+R}.}$

The specific experimental results are shown in Table 5.

Table 5:

Comparison of F1 using different classification methods.

Model	Experimental indicators	Classification
		Positivity	Negativity	Neutral
LSTM	P	0.837	0.812	0.542
	R	0.881	0.831	0.19
	F1	0.858	0.821	0.530
DCLSTM	P	0.892	0.869	0.581
	R	0.925	0.835	0.561
	F1	0.908	0.852	0.571
DCLSTM-TAM	P	0.924	0.884	0.683
	R	0.931	0.908	0.701
	F1	0.927	0.896	0.692

DOI: 10.7717/peerjcs.1295/table-5

F₁ with different classification methods of the comparison results, as shown in Fig. 5. It can be seen from the results of three classification experiments that the DCLSTM model with emotion vector improves the A value of positive, negative and neutral emotion classification by 5.0%, 3.1% and 4.1%, respectively, compared with the traditional LSTM network with only semantic vector input, which indicates that the LSTM network with emotion vector can better capture emotion information; The DCLSTM-TAM model proposed in this study is applied to the classification of positive, negative and neutral emotions. In terms of 1 value, it is 6.9%, 7.5% and 16.2% higher than the traditional LSTM network which only inputs semantic vectors, respectively. It not only improves the classification efficiency of positive and negative emotional tendencies but also significantly improves the classification efficiency of film reviews with less emotional tendencies. It shows that the use of multiple film theme attention modules can fully extract the hidden features of specific film review topics so that the model has a better effect on the emotional classification of film review texts, which verifies the effectiveness of the model built in this study to a certain extent.

This study tested the effectiveness of the proposed algorithm in building an emotional dictionary. The role of the constructed emotional dictionary in a film review, in the emotional classification of film review texts, and the effect of topic attention mechanism and dual channel LSTM model.