TASCI: transformers for aspect-based sentiment analysis with contextual intent integration

Self-attention for aspect extraction

In this section, we present a technique for aspect extraction using the self-attention mechanism, a fundamental component of Transformer models. The self-attention mechanism allows the model to assign varying levels of importance to different parts of the input sequence, thereby capturing intricate contextual dependencies (Vaswani, 2017). We will detail how this mechanism can be applied to identify and classify aspects within sentences. Consider the sentence: “The phone’s battery life is impressive but the camera quality is mediocre”. We aim to identify and extract aspects such as “battery life” and “camera quality”, and perform sentiment analysis for each aspect. Self-attention helps achieve this by focusing on relevant parts of the sentence. Self-attention operates on an input sequence $\mathbf{X}=[{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n]$, where each token ${\mathbf{x}}_i$ is an embedding of the $i$-th word. For instance, if ${\mathbf{x}}_1$ represents “battery”, ${\mathbf{x}}_2$ represents “life”, and so on. Each token embedding ${\mathbf{x}}_i$ is transformed into three vectors: the query vector ${\mathbf{Q}}_i$ as shown in Eq. (1), the key vector ${\mathbf{K}}_i$ shown in Eq. (2).

(1) $${\mathbf{Q}}_i={\mathbf{W}}^Q{\mathbf{x}}_i$$ (2) $${\mathbf{K}}_i={\mathbf{W}}^K{\mathbf{x}}_i$$ (3) $${\mathbf{V}}_i={\mathbf{W}}^V{\mathbf{x}}_i.$$

Finally, the value vector ${\mathbf{V}}_i$ is given in Eq. (3). These projections are computed using learned weight matrices ${\mathbf{W}}^Q$, ${\mathbf{W}}^K$, and ${\mathbf{W}}^V$, respectively. To compute the attention score between tokens $i$ and $j$, we take the dot product of the query vector ${\mathbf{Q}}_i$ and the key vector ${\mathbf{K}}_j$, scaling the result by $\sqrt{d_k}$, where $d_k$ is the dimensionality of the key vectors, as shown in Eq. (4). Figure 1 shows how the Query, Key and Value are passed to the Attention Score module.

(4) $$\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}{\mathrm{e}}_{ij}=\frac{{\mathbf{Q}}_i\cdot {\mathbf{K}}_j^T}{\sqrt{d_k}}.$$

These scores are then normalized using the softmax function to obtain attention weights, ensuring that the weights are non-negative and sum to one, the process of normalization is shown in Eq. (5).

(5) $${\alpha }_{ij}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}{\mathrm{e}}_{ij})=\frac{\exp (\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}{\mathrm{e}}_{ij})}{{\sum }_{k=1}^n\exp (\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}{\mathrm{e}}_{ik})}.$$

The output for each token $i$ is calculated as a weighted sum of the value vectors, using the normalized attention weights as shown in Eq. (6). Figure 1 shows how attention score is calculated using Eq. (5) and how multihead attention is calculated using Eq. (6).

(6) $${\mathbf{z}}_i=\sum _{j=1}^n{\alpha }_{ij}{\mathbf{V}}_j.$$

We use multi-head attention to enhance the model’s ability to capture various relationships. Where $V_i$ is given by Eq. (3), and ${\alpha }_{ij}$ is given by Eq. (6). This involves running self-attention multiple times with different learned projections, concatenating the outputs, and applying a final linear transformation as given in Eq. (7).

(7) $${\mathbf{Z}}_i=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}({\mathbf{z}}_i^1,{\mathbf{z}}_i^2,\dots ,{\mathbf{z}}_i^h){\mathbf{W}}^O.$$where ${\mathbf{W}}^O$ is the output transformation matrix and $z_i$ is given by Eq. (6). For aspect extraction, self-attention highlights relevant tokens corresponding to specific aspects. For instance, in the sentence “The battery life of the phone is impressive but the camera quality is mediocre,” self-attention helps identify that “battery life” and “camera quality” are key aspects. After obtaining the hidden states ${\mathbf{h}}_i$ from the self-attention layers, we use a token classification head to classify each token into an aspect category. This classification is performed using a linear layer followed by a softmax activation function, as given in Eq. (8).

(8) $${\hat{y}}_i=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\mathbf{W}}^C{\mathbf{h}}_i+{\mathbf{b}}^C)$$where ${\mathbf{W}}^C$ and ${\mathbf{b}}^C$ are the weight matrix and bias for the classification layer, and ${\hat{y}}_i$ is the predicted aspect label for token $i$. Training involves minimizing the cross-entropy loss function as shown in Eq. (9), which compares predicted aspect labels with true labels ${\mathbf{y}}_i$.

(9) $$\mathcal{L}=-\sum _{i=1}^n\sum _{k=1}^K{y}_{ik}\log ({\hat{y}}_{ik})$$where K is the number of aspect categories, we effectively capture the intricate relationships between tokens in a sentence by employing self-attention, allowing the model to focus on relevant aspects with greater contextual sensitivity. This approach ensures accurate aspect extraction and sentiment analysis, leveraging the power of self-attention to understand and categorize different aspects in complex sentences. The summary of the method is given in the Algorithm 1.

Intention extraction using GRU-based Seq2seq model

In this section, we explore extracting sentence-level intention at time $t$ using a GRU-based Seq2seq model (Sutskever, 2014). This approach leverages the inherent ability of GRUs to capture dependencies over sequential data, allowing the model to consider not only the current sentence but also the contextual information provided by previous sentences (Cho et al., 2020). Let ${\mathbf{S}}_{t-1}=[{\mathbf{s}}_{t-1}^{(1)},{\mathbf{s}}_{t-1}^{(2)},\dots ,{\mathbf{s}}_{t-1}^{(m)}]$ represent the sequence of $m$ tokens from the sentence at time $t-1$, and ${\mathbf{S}}_t=[{\mathbf{s}}_t^{(1)},{\mathbf{s}}_t^{(2)},\dots ,{\mathbf{s}}_t^{(n)}]$ represent the sequence of $n$ tokens from the sentence at time $t$. Our goal is to predict the intention ${\hat{\mathbf{I}}}_t$ of sentence $t$, using both ${\mathbf{S}}_t$ and ${\mathbf{S}}_{t-1}$ as input as shown in Fig. 2.

GRUs are a type of recurrent neural network (RNN) that mitigate the vanishing gradient problem through gating mechanisms. The GRU’s hidden state at time step $i$ is computed as in Eq. (10):

(10) $${\mathbf{h}}_i=(1-{\mathbf{z}}_i)\odot {\mathbf{h}}_{i-1}+{\mathbf{z}}_i\odot {\tilde{\mathbf{h}}}_i$$where ${\mathbf{z}}_i$ is the update gate, ${\tilde{\mathbf{h}}}_i$ is the candidate hidden state, and $\odot$ denotes element-wise multiplication. The update gate ${\mathbf{z}}_i$ and reset gate ${\mathbf{r}}_i$ are defined as in Eqs. (11) and (12):

(11) $${\mathbf{z}}_i=\sigma ({\mathbf{W}}_z{\mathbf{x}}_i+{\mathbf{U}}_z{\mathbf{h}}_{i-1})$$

(12) $${\mathbf{r}}_i=\sigma ({\mathbf{W}}_r{\mathbf{x}}_i+{\mathbf{U}}_r{\mathbf{h}}_{i-1})$$where ${\mathbf{W}}_z$ and ${\mathbf{W}}_r$ are the weights of the update and reset gates associated with the input vector ${\mathbf{x}}_i$, respectively, and ${\mathbf{U}}_z$ and ${\mathbf{U}}_r$ are the weights of the update and reset gates associated with the previous hidden state ${\mathbf{h}}_{i-1}$, respectively. These weight matrices are learned during training and determine how the GRU processes the input and integrates information from the previous hidden state. Also, $\sigma$ is the sigmoid activation function, ensuring that the gate values lie between 0 and 1, allowing the GRU to balance the contributions of the current input and the past hidden state. The candidate hidden state ${\tilde{\mathbf{h}}}_i$ is computed using the reset gate as in Eq. (13):

(13) $${\tilde{\mathbf{h}}}_i=\tanh (\mathbf{W}{\mathbf{x}}_i+{\mathbf{r}}_i\odot \mathbf{U}{\mathbf{h}}_{i-1})$$where $\mathbf{W}$ is the weight matrix for the input ${\mathbf{x}}_i$, $\mathbf{U}$ is the weight matrix for the previous hidden state ${\mathbf{h}}_{i-1}$, and ${\mathbf{r}}_i$ is the reset gate controlling the influence of ${\mathbf{h}}_{i-1}$.

In the context of intention extraction, the encoder processes both ${\mathbf{S}}_{t-1}$ and ${\mathbf{S}}_t$. For the sentence at $t-1$, the GRU produces a sequence of hidden states as in Eq. (14):

(14) $${\mathbf{H}}_{t-1}=[{\mathbf{h}}_{t-1}^{(1)},{\mathbf{h}}_{t-1}^{(2)},\dots ,{\mathbf{h}}_{t-1}^{(m)}].$$

Similarly, for the sentence at $t$ is given in Eq. (15):

(15) $${\mathbf{H}}_t=[{\mathbf{h}}_t^{(1)},{\mathbf{h}}_t^{(2)},\dots ,{\mathbf{h}}_t^{(n)}].$$

The final hidden state of the encoder for both sentences serves as a summary of the sentence content and its intention is shown in Eq. (16):

(16) $${\mathbf{h}}_{t-1}^{(final)}={\mathbf{h}}_{t-1}^{(m)},{\mathbf{h}}_t^{(final)}={\mathbf{h}}_t^{(n)}.$$

To enhance the prediction of intention at $t$, we apply an attention mechanism that weighs the importance of each hidden state in ${\mathbf{H}}_{t-1}$ based on its relevance to the hidden state at $t$, as shown in Fig. 2. Equation (17) computes the attention score between the hidden states ${\mathbf{h}}_{t-1}^{(i)}$ and ${\mathbf{h}}_t^{(j)}$.

(17) $${\alpha }_{ij}=\frac{\exp ({\mathbf{h}}_t^{(j)}\cdot {\mathbf{h}}_{t-1}^{(i)})}{{\sum }_{k=1}^m\exp ({\mathbf{h}}_t^{(j)}\cdot {\mathbf{h}}_{t-1}^{(k)})}.$$

The context vector ${\mathbf{c}}_t$, which summarizes the important aspects of ${\mathbf{S}}_{t-1}$ for predicting intention at $t$, is computed as a weighted sum of the hidden states is given in Eq. (18):

(18) $${\mathbf{c}}_t=\sum _{i=1}^m{\alpha }_{ij}{\mathbf{h}}_{t-1}^{(i)}.$$where ${\alpha }_{ij}$ is given in Eq. (17). The Eq. (19) shows that the context vector ${\mathbf{c}}_t$ is concatenated with the final hidden state ${\mathbf{h}}_t^{(final)}$ to form the input to the decoder.

(19) $${\mathbf{d}}_t=\tanh ({\mathbf{W}}_d[{\mathbf{c}}_t;{\mathbf{h}}_t^{(final)}]).$$

The decoder GRU then generates the predicted intention, as given in Eq. (20):

(20) $${\hat{\mathbf{I}}}_t=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\mathbf{W}}_o{\mathbf{d}}_t+{\mathbf{b}}_o)$$where ${\mathbf{W}}_o$ and ${\mathbf{b}}_o$ are learned parameters of the output layer. To train the model, we minimize the cross-entropy loss as shown in Eq. (21), between the predicted intention ${\hat{\mathbf{I}}}_t$ and the true intention ${\mathbf{I}}_t$:

(21) $$\mathcal{L}=-\sum _{t=1}^T{\mathbf{I}}_t\log ({\hat{\mathbf{I}}}_t).$$

Algorithm 2 provides the overview of the discussion in this sub-section.

Transformer-based aspect-level sentiment classification with intent analysis

In this section, we propose a novel approach to sentiment analysis by combining aspect-level sentiment classification with intent analysis. Our goal is to determine the sentiment associated with each aspect of a given sentence while contextualizing it based on the speaker’s intent inferred from previous sentences. Consider the example sentence: “The phone’s battery life is impressive, but I was expecting a better camera.” Here, the sentiment for the aspect of “battery life” is positive, while the sentiment for “camera” is negative, influenced by the user’s prior expectation. To achieve this, we first extract aspects using a self-attention mechanism, as described in “Self-Attention for Aspect Extraction”. Each aspect is then embedded into a continuous vector space using GloVe embeddings (Pennington, Socher & Manning, 2014). The intention behind the sentence is inferred using a Transformer-based model that takes into account the context provided by previous sentences as shown in Fig. 3. Let ${\mathbf{e}}_i$ denote the GloVe embedding for token $i$. The Eq. (22) computes the embedding as follows:

(22) $${\mathbf{e}}_i={\mathbf{W}}_G{\mathbf{x}}_i$$where ${\mathbf{W}}_G$ represents the pre-trained GloVe embedding matrix, and ${\mathbf{x}}_i$ is the one-hot encoded vector of token $i$. Then in Eq. (23) each identified aspect is computed and its representation is made by aggregating the embeddings of the tokens that constitute the aspect.

(23) $${\mathbf{v}}_a=\frac{1}{|A|}\sum _{i\in A}{\mathbf{e}}_i$$where ${\mathbf{v}}_a$ is the vector representation of aspect A, and |A| is the number of tokens in the aspect. The hidden state vector for an aspect $i$, denoted by ${\mathbf{H}}_i$, is obtained after processing the aspect through the Transformer encoder, which captures the contextual relationships among tokens.

To incorporate intent, we also process the sentence context from previous statements. Let ${\mathbf{I}}_t$ represent the intent vector at time $t$, computed as in Eq. (24):

(24) $${\mathbf{I}}_t=\mathrm{I}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{E}\mathrm{x}\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}({\mathbf{S}}_{t-1},{\mathbf{S}}_t)$$where ${\mathbf{S}}_{t-1}$ and ${\mathbf{S}}_t$ are the representations of the sentences at time $t-1$ and $t$, respectively. The final sentiment classification for each aspect is then performed by combining the aspect’s hidden state ${\mathbf{H}}_i$ with the intent vector ${\mathbf{I}}_t$, in Eq. (25), using a linear classification layer followed by a softmax activation function.

(25) $$s_i=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\mathbf{W}}^S({\mathbf{H}}_i+{\mathbf{I}}_t)+{\mathbf{b}}^S)$$where ${\mathbf{W}}^S$ and ${\mathbf{b}}^S$ are the weight matrix and bias term for the sentiment classification layer, respectively. The output $s_i$ is a probability vector representing the likelihood of the aspect belonging to each sentiment category (e.g., positive, negative, neutral). Figure 3 provides a comprehensive overview of the sentiment analysis process incorporating intention prediction. It starts by displaying the Input Sentences, represented by two horizontal boxes. The first box, labeled Sentence at $t-1$, contains tokens $s_{t-1}(1),s_{t-1}(2),\dots ,s_{t-1}(m)$, illustrating the sequence of tokens at time $t-1$. This historical context is crucial for understanding how previous statements may influence the sentiment of the current sentence. The second box, Sentence at $t$, includes tokens $s_t(1),s_t(2),\dots ,s_t(n)$, indicating the current sentence’s tokens that are the focus of the sentiment and intention analysis. Next, the ‘Encoding with GRU’ section displays two vertical blocks representing the GRU encoders used to process each sentence. Each block contains smaller rectangles or circles symbolizing the hidden states $h_{t-1}(i)$ for the previous sentence and $h_t(j)$ for the current sentence. Arrows illustrate the flow of information from the tokens of each sentence to their corresponding hidden states, and then to the final hidden state that encapsulates the contextual information from both sentences. In the ‘Attention Mechanism’ section, a matrix or table between the hidden states of the sentences showcases attention scores ${\alpha }_{ij}$, which quantify the relevance of each token in one sentence concerning tokens in the other sentence. Arrows connect hidden states to this attention matrix and then to a context vector box. This matrix helps in focusing on the most pertinent parts of the previous sentence when interpreting the current sentence. The ‘Context Vector Computation’ box, labelled $c_t$, illustrates how the context vector is derived from the attention matrix. This context vector integrates information from the previous sentence, adjusting the representation of the current sentence accordingly. The ‘Decoding’ section features a GRU Decoder block. This block receives the context vector and the final hidden state $h_t(\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{l})$ as inputs, labelled ‘Concatenated Input’ $d_t$. Arrows depict how these inputs are combined in the decoder to generate the final output. Finally, the ‘Intention Prediction’ box, ${\hat{I}}_t$, shows the output of the intention prediction process. This box, with an arrow from the decoder GRU, includes a softmax operation that produces prediction scores. These scores represent the likelihood of various intents based on the combined information from the context vector and hidden states. The ‘Training Loss’ highlights the use of cross-entropy loss to evaluate and fine-tune the model’s performance, ensuring accurate sentiment and intention predictions.

Experimental setup

This section provides an in-depth description of the hardware and software infrastructure utilized for the experimental setup. The detailed specifications below highlight the computational resources and software tools used to facilitate high-performance computing tasks, including general-purpose operations as well as specialized tasks like deep learning.

Hardware configuration: The experiments were executed on a system designed using GPU. The primary hardware components are outlined below:

• Central processing unit (CPU): The computational tasks were executed on processor, the Intel Core i9-14900K. This CPU features 16 physical cores and 32 threads, enabling efficient multi-threaded processing for demanding applications. Operating at a base clock speed of 3.6 GHz, it provides robust performance for complex computations.

• Graphics processing unit (GPU): For tasks requiring accelerated computation, such as deep learning model training, the system utilized an NVIDIA GeForce RTX 4090. This GPU, equipped with 24 GB of GDDR6X memory, delivers performance for data-intensive workloads. Its advanced CUDA and Tensor Core support ensures optimal efficiency in matrix operations and neural network processing.

• Memory (RAM): The system was equipped with 128 GB of DDR5 RAM, offering extensive memory capacity to handle large datasets and computationally expensive models.

Software environment: The experiments were carried out using a software environment comprising operating systems, programming tools, frameworks, and libraries. The key elements of the software stack are detailed below:

• Operating system: The experiments were performed on a Linux-based system running Ubuntu 22.04 LTS.

• Programming language: The primary programming language used was Python version 3.10.12.

• Deep learning frameworks:

  • –

    TensorFlow 2.12.0: TensorFlow was employed for its versatile ecosystem that supports model design, training, and deployment. This version includes enhanced performance optimizations for modern GPUs.

  • –

    PyTorch 2.0.1: PyTorch was selected for its intuitive dynamic computation graph capabilities and ease of implementation, which simplified model development and experimentation.

• Libraries and Tools:

  • –

    Scikit-learn 1.2.2: This library was utilized for conducting model evaluations such as F-1 Score.

• Containerization: To ensure reproducibility and simplify dependency management, Docker 24.0.2 was used for containerization. This tool enabled the creation of isolated software environments, providing consistency across different systems and easing deployment.

Dataset

To evaluate the performance of TASCI, we utilized three benchmark datasets: SemEval (2014). These datasets are used in the state of the art for aspect-level sentiment analysis and provide diverse contexts and sentiment expressions. Each dataset was chosen to assess the model’s effectiveness across different domains.

Accuracy of sentimental analysis

In this section, we present a detailed evaluation of the proposed TASCI model against several state-of-the-art methods using three benchmark datasets: Restaurant, Laptop, and Twitter. The evaluation focuses on two key metrics, accuracy and macro-F1 score, which provide insights into the overall correctness and balanced performance across different sentiment classes. Table 3 summarizes TASCI’s performance across the three sentiment categories—positive, negative, and neutral—offering a comprehensive assessment of its effectiveness in varied contexts. Precision measures the proportion of correctly identified positive instances out of all instances classified as positive, with higher values indicating fewer false positives. Recall quantifies the model’s ability to identify all relevant instances of a class, representing the proportion of true positives that were correctly predicted. F1-score combines precision and recall into a single metric, providing a balanced measure of the model’s performance, particularly useful for imbalanced datasets.

Examining the results, TASCI demonstrates robust performance across all datasets and sentiment classes. For the Restaurant dataset, TASCI achieves a precision of 90.20, recall of 88.50, and F1-score of 89.34 for the positive sentiment, reflecting a high level of accuracy and completeness in identifying positive instances. For the negative sentiment, the model achieves a precision of 87.60, recall of 85.12, and F1-score of 86.34, with slightly lower precision compared to the positive class. The neutral sentiment exhibits a precision of 89.10, recall of 84.92, and F1-score of 86.96, showcasing strong performance in detecting neutral instances as well. Performance on the Laptop dataset, while slightly lower than the Restaurant dataset, remains effective. The model achieves a precision of 85.45, Recall of 83.20, and F1-score of 84.31 for positive sentiment. For the negative sentiment, TASCI achieves a precision of 82.75, recall of 80.40, and F1-score of 81.56, reflecting a slight reduction in performance compared to the Restaurant dataset, potentially due to the dataset’s unique characteristics or text complexity. The neutral sentiment shows a precision of 86.01, recall of 82.90, and F1-score of 84.42, indicating consistent performance across different classes. The Twitter dataset presents additional challenges due to the short and informal nature of the text, yet TASCI performs reasonably well. For the Positive sentiment, the model achieves a precision of 80.34, recall of 78.92, and F1-score of 79.62. The negative sentiment metrics are slightly lower, with a precision of 78.23, recall of 76.11, and F1-score of 77.15, highlighting the difficulties of analyzing social media text. For the neutral sentiment, TASCI achieves a precision of 79.98, recall of 78.04, and F1-score of 79.00, demonstrating the model’s adaptability even in challenging contexts. Overall, TASCI demonstrates strong and consistent performance across all three datasets, excelling in identifying sentiment classes with high precision and recall. The Restaurant dataset shows the highest metrics, followed by the Laptop dataset, with slightly lower values for the Twitter dataset due to its inherent challenges. These results underscore TASCI’s robustness and versatility in handling diverse sentiment classification tasks, while also highlighting the influence of dataset characteristics on model performance.

Table 4 provides a detailed comparison of TASCI with state-of-the-art models across three datasets Restaurant, Laptop, and Twitter. TASCI consistently achieves superior performance, surpassing all other models in both accuracy and macro-F1 scores, demonstrating its robustness and effectiveness in aspect-level sentiment classification with intent analysis. On the Restaurant dataset, TASCI achieves the highest accuracy of 89.10% and a macro-F1 score of 83.38%, outperforming the previous best model, LSEO-IT (Wu et al., 2024), which achieved 86.88% accuracy and 82.27% macro-F1. Other notable models, such as DGCN + bidirectional encoder representations from transformers (BERT) (Li et al., 2021), which recorded 87.13% accuracy and 81.86% macro-F1, and DOTI (Chen et al., 2022), which achieved 86.16% accuracy and 80.49% macro-F1, fall short compared to TASCI. These results highlight TASCI’s ability to better capture sentiment nuances in customer reviews. On the Laptop dataset, TASCI achieves an accuracy of 84.81% and a macro-F1 score of 78.63%, significantly surpassing DGCN + BERT (Li et al., 2021), which achieved 81.80% accuracy and 78.10% macro-F1, as well as DOTI (Chen et al., 2022), which recorded 81.03% accuracy and 78.10% macro-F1. Even models such as LSEO-IT, which achieved 81.41% accuracy and 77.16% macro-F1, and SASE-GCN (Zhang, Zhou & Wang, 2022), which recorded 81.01% accuracy and 77.96% macro-F1, are outperformed by TASCI. This consistent improvement underscores TASCI’s effectiveness in handling the more complex and diverse sentiment expressions typical of the Laptop dataset. On the Twitter dataset, known for its informal language and challenging text structures, TASCI achieves an accuracy of 79.08% and a macro-F1 score of 77.27%. While the margin of improvement is smaller compared to the other datasets, TASCI still outperforms existing models, such as DOTI (Chen et al., 2022), which achieved 78.11% accuracy and 77.00% macro-F1, and SASE-GCN (Zhang, Zhou & Wang, 2022), which recorded 77.40% accuracy and 76.02% macro-F1. Models like DGCN + BERT (Li et al., 2021), which achieved 77.40% accuracy and 76.02% macro-F1, also fall short, further highlighting TASCI’s adaptability to diverse text formats. Overall, TASCI sets a new benchmark across all datasets, consistently outperforming prior state-of-the-art models, including LSEO-IT, Discrete opinion tree induction (DOTI), and DGCN + BERT, by significant margins. These results underscore TASCI’s robustness, versatility, and ability to handle the challenges of aspect-level sentiment classification effectively, regardless of the domain or dataset complexity.

Abalation study

In this study, we present a detailed ablation analysis of the TASCI model to evaluate the impact of various components on its overall performance. The purpose of this analysis is to understand how each part of the model contributes to its effectiveness and to identify which elements are most crucial for achieving high performance in sentiment analysis. The results of this study are outlined in Table 5, which provides a comprehensive view of the model’s performance across different variations and datasets, including Restaurant, Laptop, and Twitter. Our primary focus was to assess how removing specific components from the TASCI model affects its accuracy and macro-F1 scores. The TASCI model integrates multiple features such as intent analysis, sentiment-specific features, and a GAT module. The first variation we examined was TASCI without the intent analysis component. In this configuration, the model’s performance showed a noticeable decline. Specifically, on the Restaurant dataset, the model achieved an accuracy of 86.75% and a macro-F1 score of 80.48%. For the Laptop dataset, the accuracy dropped to 82.90%, with a macro-F1 score of 76.32%. The Twitter dataset saw a similar decline, with an accuracy of 77.92% and a macro-F1 score of 76.12%. The reduction in performance highlights the importance of intent analysis, which is designed to capture and interpret the underlying intentions behind the text. Without this component, the model struggled to fully understand the subtleties of user sentiment, leading to lower accuracy and F1 scores. The second variation involved testing the TASCI model without the sentiment-specific features. These features are intended to enhance the model’s sensitivity to different sentiment categories. In this case, the performance metrics showed a moderate decrease. The model achieved an accuracy of 87.20% and a macro-F1 score of 81.34% on the Restaurant dataset. For the Laptop dataset, the accuracy was 83.12% with a macro-F1 score of 77.10%, and the results for the Twitter dataset were 78.45% accuracy and 76.98% macro-F1 score. While the performance reduction was less severe compared to the removal of intent analysis, it still underscores the value of sentiment-specific features in improving classification granularity.

The third variation examined the TASCI model with only the GAT module. This configuration provided insights into the contribution of the GAT component alone. The results have shown accuracy of 88.15% and a macro-F1 score of 82.50% on the Restaurant dataset. For the Laptop dataset, the model achieved an accuracy of 84.05% and a macro-F1 score of 78.45%, while on the Twitter dataset, it reached 79.15% accuracy and a macro-F1 score of 77.85%. The GAT module, which enhances the model’s ability to understand context and relationships between different entities in the text, proved to be highly effective. Although the performance with only the GAT module did not match that of the full TASCI model, it still demonstrated a substantial improvement in both accuracy and F1 scores. Finally, the baseline TASCI model, which incorporates all components intent analysis, sentiment specific features, and the GAT module achieved the highest performance across all datasets. For the Restaurant dataset, the accuracy was 89.10% with a macro-F1 score of 83.38%. On the Laptop dataset, the model achieved 84.81% accuracy and a macro-F1 score of 78.63%. For the Twitter dataset, the results were 79.08% accuracy and 77.27% macro-F1 score. These results confirm that the full TASCI model, with all components included, delivers the best performance. The integration of intent analysis, sentiment-specific features, and the GAT module provides a comprehensive approach to sentiment classification, leading to improved accuracy and more detailed classification across different sentiment categories. Overall, the ablation study highlights the significant contributions of each component within the TASCI model. The removal of intent analysis leads to a marked decline in performance, underscoring its critical role in capturing nuanced user sentiments. While sentiment-specific features also contribute to the model’s effectiveness, their absence results in a less pronounced drop in performance compared to the removal of intent analysis. The GAT module alone enhances the model’s contextual understanding and relationship modeling capabilities, although it does not reach the same level of performance as the full model. The baseline TASCI model, incorporating all components, demonstrates the most robust performance, validating the effectiveness of its comprehensive design. This study provides valuable insights into the importance of each component and the necessity of their combined application for optimal sentiment analysis results.

Results of attention mechanism

This section presents the results of the attention mechanism in TASCI, attention is used in two ways, firstly it is used to highlight aspects in the sentence. The transformer automatically determines tokens in the given sentences which can be used to exact a certain aspect. Secondly, it is used to integrate aspect-level value with intention analysis. The outcome of the attention matrix shown in Fig. 4 visualization provides a clear view of how attention is distributed across the sentence, emphasizing specific aspects. In the generated attention matrix, you will observe a concentration of higher attention scores around the words corresponding to the identified aspects, such as “battery life” and “camera quality.” This indicates that the model pays more attention to these critical phrases, reflecting their importance in the context of the sentence. The matrix shows that attention is not confined to exact aspect words but is spread over adjacent words due to the inherent nature of attention mechanisms in neural networks. This results in a more realistic distribution where attention values gradually decrease as they move away from the key aspects. The visualization allows you to see how the model allocates focus, providing insight into which parts of the sentence are considered more significant in identifying aspects. This approach helps in understanding the model’s behaviour and its emphasis on specific components of the text.