A novel attention-based deep learning model for improving sentiment classification after the case of the 2023 Kahramanmaras/Turkey earthquake on Twitter

Motivation and contributions

The motivation and contributions of the proposed study are given below:

• Original dataset and advanced preprocessing: In this study, a large dataset of tweets shared in the immediate aftermath of the February 6, 2023, Kahramanmaraş/Turkey earthquake was collected from Twitter. The raw data underwent advanced preprocessing to remove noise, apply lemmatization, and extract meaningful information. Deep learning-based NLP techniques were then used to analyze the earthquake’s psychological impact on a global scale.

• A novel hybrid model (MConv-BiLSTM-GAM): The proposed model combines convolutional neural networks (CNN), bidirectional long short-term memory (BiLSTM), and the Global Attention Mechanism (GAM) to harness their complementary capabilities for enhanced sentiment classification. FastText embeddings are utilized to convert tweets into vector representations, offering a semantic advantage by operating at the character n-gram level. This allows for better handling of misspellings, morphologically rich forms, and out-of-vocabulary (OOV) words—common features in noisy social media content. The MConv stage applies multiple 1D convolutional filters (kernel sizes 2 and 3) to extract local contextual features, while the BiLSTM layer captures temporal and bidirectional dependencies. To address potential limitations in focusing on key semantic elements, the GAM component assigns attention weights across the sequence, enhancing the model’s ability to emphasize relevant tokens and manage long-distance dependencies. This integrated design ensures a robust and context-aware sentiment classification approach, well-suited for informal and unstructured Twitter data.

• Contribution to post-disaster psychological impact analysis: This study presents an NLP-based framework for analyzing societal emotions following a disaster, utilizing social media data to assess large-scale sentiment trends. By capturing and interpreting public reactions, the proposed model provides valuable insights for crisis management and psychological support planning. Its data-driven methodology enhances decision-making processes, facilitating the development of targeted interventions to support affected communities more effectively.

The following sections in the article are organized as follows: “Related Works” presents a literature review of relevant work in the field. “The Proposed System” describes the dataset collection process, the applied preprocessing steps, and the architecture of the proposed model. “Discussions and Experimental Results” focuses on sentiment visualization, data analysis, and a comparative evaluation of the experimental results. Finally, the study is concluded in “Conclusions”.

Dataset and preprocessing

This study utilizes Twitter (Alam et al., 2021) as a data source to examine the societal impact of the February 6, 2023 Kahramanmaraş/Turkey earthquake. A total of 215,446 publicly available English tweets were collected via MAXQDA software (MAXQDA, 2020), covering the period from February 6 to April 27, 2023. To ensure ethical compliance, no personal or identifiable information was included. Relevant tweets were retrieved using common earthquake-related hashtags such as #earthquake, #turkeyearthquake, and #turkeysyriaearthquake. After removing redundant elements (e.g., special characters, links, emojis), preprocessing reduced the dataset to 81,797 clean tweets suitable for sentiment analysis. VADER was used to determine sentiment polarities: 38.91% were negative (31,824 tweets), 37.09% positive (30,336), and 24.01% neutral (19,637). This preprocessing step enhanced both the accuracy and efficiency of the sentiment classification task.

The proposed model

This study proposes a novel sentiment classification approach based on the MConv-BiLSTM-GAM model to improve performance on Twitter data. As illustrated in Fig. 2, the architecture integrates FastText embeddings with a hybrid structure combining CNN, BiLSTM, and GAM. The model consists of three main stages: MConv for local feature extraction using multiple Conv1D layers with kernel sizes 2 and 3, followed by MaxPooling, BiLSTM for capturing sequential dependencies, and GAM for focusing on semantically relevant information across the sequence. GAM enables the model to attend to distant yet important tokens by learning contextual relevance. Finally, the attention-refined output is passed through two dense layers, concluding with a softmax-based classification layer.

The proposed MConv-BiLSTM-GAM model architecture was optimized using a set of carefully selected hyperparameters for each stage. In the feature extraction stage (MConv), two 1D convolutional layers with 100 filters and kernel sizes of 2 and 3 were employed to capture diverse n-gram features, followed by ReLU activation, a max-pooling layer with a size of 2, and a dropout rate of 0.25 to prevent overfitting. In the sequence learning stage (BiLSTM), a bidirectional LSTM layer with 128 nodes and ReLU activation was used, and a dropout rate of 0.25 was used to maintain regularization. The attention mechanism (GAM) was configured with an input size of 95 to compute attention weights across the sequence. Finally, the classification stage consisted of two fully connected dense layers: the first with five units and ReLU activation, and the second with three units and a softmax activation function to output the sentiment class probabilities. These hyperparameter choices were determined through empirical tuning to effectively balance model complexity and performance.

Analyzing the sentiment distributions

To explore the psychological context of emotional tendencies, word cloud visualization was used to highlight the most frequent and sentiment-relevant terms in tweets. Figure 3 displays the word clouds corresponding to positive, negative, and neutral tweet categories. As shown in Fig. 3, non-emotive but high-frequency words such as “turkey,” “earthquake,” and “people” were excluded to better emphasize emotionally significant keywords. The word clouds for positive, negative, and neutral sentiment categories reveal distinct linguistic patterns. Despite similar proportions, positive tweets predominantly include expressions of support, solidarity, and reassurance, while negative tweets contain fear-inducing and destructive language. Such negative discourse during disasters may contribute to long-term psychological effects on affected communities. The findings illustrated in Fig. 3 offer valuable insights for decision-makers to design timely and targeted psychological, social, and financial interventions based on the collective emotional state.

Performance evaluation of the sentiment classification model

This section presents a comparative performance analysis of sentiment classification models on Twitter data related to the February 6, 2023 Kahramanmaraş earthquake. The proposed MConv-BiLSTM-GAM model is evaluated against three baseline deep learning architectures, using both FastText and GloVe embeddings to assess embedding effectiveness. Performance is measured through accuracy, precision, recall, and F1-score (Aslan & Kaya, 2018), while overfitting risk is monitored via training and testing accuracy-loss curves (Reagen et al., 2018). Additionally, a confusion matrix is used to examine misclassifications. Results demonstrate the proposed model’s superior performance, particularly in handling emotionally charged disaster-related content.

The performance of the proposed MConv-BiLSTM-GAM model was thoroughly evaluated using earthquake-related Twitter data, with detailed results provided in Table 1. When utilizing FastText embeddings, the model achieved an average accuracy of 93.32%, outperforming other deep learning models by approximately 3%. The F1-scores for positive, negative, and neutral sentiment classes were 93.48%, 93.46%, and 92.86%, respectively, demonstrating both high accuracy and classification consistency. Comparative analyses show that the CNN model performed relatively poorly, especially in the negative sentiment class, with a precision of 89.72% and a recall of 92.84%. While LSTM and BiLSTM models performed better due to their ability to capture sequential dependencies, the proposed MConv-BiLSTM-GAM model achieved the highest performance. This improvement stems from the synergy between CNN’s local feature extraction, BiLSTM’s contextual learning, and the Global Attention Mechanism’s ability to focus on semantically relevant information. In particular, the MConv layer enhances local pattern recognition through multiple kernel sizes, while BiLSTM captures bidirectional dependencies, and GAM further refines feature relevance, leading to more accurate sentiment classification.

Given that social media data shared during disasters such as earthquakes contain high emotional intensity and similar word patterns, using only CNN or BiLSTM is insufficient for precise SA. Therefore, the extracted features are processed through the GAM mechanism to prioritize the most meaningful information. Experimental results confirm the proposed model’s superior performance in sentiment classification.

Another important factor in the success of the proposed model is the use of the FastText word embedding method. To evaluate its effectiveness, the MConv-BiLSTM-GAM model was tested using datasets vectorized with both FastText and GloVe embeddings, with the results illustrated in Fig. 4. As shown, FastText led to higher performance, with validation accuracy starting at 90% and reaching 93.43% by the end of training, compared to 84% to 87.08% with GloVe. The average accuracy achieved using FastText (93.32%) is approximately 6.35% higher than that of GloVe (87.08%), indicating a notable performance advantage. FastText’s ability to capture subword and character-level information through character n-grams makes it particularly effective for handling the morphologically rich and noisy language often found in social media posts during disasters. This allows for more accurate representation of misspelled or informal words. Furthermore, analysis of the accuracy-loss curves suggests that training progresses smoothly without signs of overfitting. This is supported by the application of regularization techniques and early stopping, which help improve the model’s generalization performance.

Figure 5 presents the confusion matrices of the tested algorithms. The results indicate that the proposed model achieves the highest accuracy when used with FastText. By maintaining a more balanced and consistent performance across all sentiment classes, the proposed model minimizes misclassifications. In contrast, the CNN model demonstrates a higher error rate, particularly in the positive sentiment class, due to its limited ability to model sequential dependencies. LSTM and BiLSTM models outperform CNN by better capturing contextual relationships. However, models based on GloVe exhibit a higher misclassification rate compared to FastText. This discrepancy arises from GloVe’s limited ability to represent semantic relationships between words. Overall, the proposed model, when combined with FastText, provides high accuracy and consistency, outperforming other models in sentiment classification tasks.

To compare training durations, the training times of different word embedding and classification models were examined over 15 epochs. The CNN model had the shortest training time with both embeddings, completing in 14 min using FastText and just 2 min with GloVe. In contrast, the MConv-BiLSTM-GAM model required the longest training time—249 min with FastText and 50 min with GloVe. The BiLSTM model also showed relatively longer durations, taking 166 min with FastText and 40 min with GloVe. These findings indicate that model complexity and the choice of word embedding method significantly affect training time.

In Table 2, the classification accuracy of the proposed approach was systematically compared with existing studies on SA of earthquake-related tweets in the literature. The results indicate that the proposed model demonstrated superior classification performance, achieving higher accuracy rates than other methodologies previously introduced in this research domain. This finding suggests that integrating the MConv-BiLSTM-GAM model effectively enhances sentiment classification accuracy.

Ablation studies were conducted to assess the individual contributions of MConv, BiLSTM, and GAM within the proposed MConv-BiLSTM-GAM architecture. The full model achieved the highest accuracy (93.32%) using FastText embeddings, confirming the strength of the integrated design. Removing GAM reduced accuracy to 92.33%, highlighting the importance of attention for semantic focus. Further removals of BiLSTM and MConv led to accuracies of 91.48% and 91.18%, respectively, indicating the value of sequential modeling and local feature extraction. Models using only LSTM or BiLSTM underperformed, emphasizing the necessity of combining convolutional, recurrent, and attention mechanisms. Overall, the results validate the effectiveness of the integrated architecture for sentiment analysis in noisy social media data during disasters.