Detecting sarcasm in user-generated content integrating transformers and gated graph neural networks

Overview of the model architecture

The proposed model integrates three core components: BERT, GGNN, and a self-attention mechanism. The architecture is designed to capture both the contextual semantics of the input text and the global relationships between words to improve the performance of sarcasm detection (Qin et al., 2024).

To begin, BERT is used to generate dynamic contextual embeddings for each token in the input text. These embeddings capture fine-grained semantic and syntactic relationships by considering the bidirectional context of each word, making them particularly effective for detecting sarcasm, which often relies on subtle contrasts and ambiguities. These embeddings serve as the foundation for subsequent processing and are passed through the GGNN layer, which models the dependencies between words based on graph structures derived from syntactic or semantic relationships. The GGNN captures both local and global relationships between words by incorporating dependency graphs, which reflect grammatical structures, and affective graphs that capture emotional cues (as seen in related works such as Zhang, Zhang & Vo (2019)). This interaction between BERT and GGNN enables the model to leverage both contextual embeddings and structural relationships, allowing it to handle both explicit and implicit forms of sarcasm more effectively.

Following this, the embeddings are processed by a multi-head self-attention mechanism that assigns attention weights to different parts of the sentence, emphasizing critical sarcasm indicators, as discussed in Wu et al. (2020). The self-attention mechanism enhances the model by dynamically focusing on words or phrases that exhibit contradictory or emotionally charged cues, which are often key to detecting sarcasm. For instance, in a sentence like “Oh great, another rainy day,” self-attention would likely assign higher weights to the contrastive elements (“great” and “rainy”) that signal sarcasm. By attending to these elements across multiple attention heads, the mechanism can capture diverse perspectives on potential sarcastic cues. Additionally, the integration of self-attention with GGNN allows the model to refine its understanding of long-range dependencies by prioritizing semantically significant nodes in the graph structure. This synergy ensures that the model captures both word-level interactions and broader contextual relationships.

The overall main components are shown in Fig. 1. In summary, BERT provides rich contextual embeddings, GGNN models inter-word dependencies based on graph representations, and self-attention highlights critical sarcastic elements. The interaction between these components ensures the model effectively captures the nuanced and multi-faceted nature of sarcasm in diverse datasets.

BERT model: text embedding and contextual understanding

BERT (Devlin et al., 2019; Sun et al., 2019) is a powerful pre-trained language model that generates contextual embeddings for each token in the input text. Unlike traditional models such as Word2Vec or GloVe, BERT is based on the Transformer architecture, which uses self-attention to capture contextual relationships between tokens. This results in embeddings that vary dynamically depending on the context of the entire sentence, making BERT especially effective in capturing long-range dependencies and semantic nuances, which are crucial for detecting sarcasm (Ghosh et al., 2018; Riloff et al., 2013). The schematic diagram of each layer is shown in Fig. 2.

For this, given an input sequence $\mathbf{X}=x_1,x_2,\dots ,x_n$, where $n$ is the length of the sequence, BERT transforms it into a set of contextual embeddings $\mathbf{H}=h_1,h_2,\dots ,h_n$, with each embedding $h_i$ containing enriched contextual information for token $x_i$. The process can be formally written as:

(1) $$\mathbf{H}=\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{T}(\mathbf{X})$$where $\mathbf{H}\in R^{n\times d}$ is the output embedding matrix, and $d$ is the embedding dimension (usually 768 or 1,024 for BERT). These embeddings are then used as input for the subsequent GGNN layers, which further refine them by capturing inter-word dependencies within a graph-based framework (Li et al., 2016; Kipf & Welling, 2016).

This combination of BERT’s deep contextual embeddings and GGNN’s graph-based structure modeling allows the model to capture the complex nature of sarcasm, which often involves both implicit and explicit semantic cues. The integration of the self-attention mechanism further refines this process by allowing the model to focus on the most relevant parts of the sentence, enhancing its ability to detect sarcastic intent.

GGNN layer: information propagation in graph structures

GGNNs (Li et al., 2016; Zhang, Zhang & Vo, 2019) are an extension of GNNs that incorporate gating mechanisms similar to those in recurrent neural networks (RNNs) to control the flow of information across nodes in a graph. In the context of sarcasm detection, GGNN is employed to model word dependencies and sentiment shifts by representing the text as a graph where each node corresponds to a word, and edges represent syntactic or semantic relationships between words.

(2) $$h_i^{(t+1)}=\mathrm{G}\mathrm{R}\mathrm{U}\left(h_i^{(t)},\sum _{v_j\in \mathcal{N}(v_i)}{\mathbf{W}}_e{h}_j^{(t)}\right),$$

Let $\mathcal{G}=(\mathcal{V},\mathcal{E})$ be a directed graph where $\mathcal{V}$ is the set of nodes (words) and $\mathcal{E}$ is the set of edges (dependencies between words). Each node $v_i$ is initialized with its corresponding BERT embedding $h_i$. The GGNN updates the hidden state of each node through iterative message passing, where the hidden state of node $v_i$ at time step $t$, denoted as $h_i^{(t)}$, is updated based on the hidden states of its neighbors:

where $\mathcal{N}(v_i)$ is the set of neighboring nodes of $v_i$, ${\mathbf{W}}_e$ is a learnable weight matrix associated with the edges, and $\mathrm{G}\mathrm{R}\mathrm{U}(\cdot )$ denotes the Gated Recurrent Unit update function.

The GRU helps the model retain long-term information while preventing the vanishing gradient problem, making it well-suited for capturing long-range dependencies in sarcasm detection. After T iterations of message passing, the final hidden states $h_1^{(T)},h_2^{(T)},\dots ,h_n^{(T)}$ are used as the graph representations of the input text.

As illustrated in Fig. 3, this architecture enables the propagation of information through neighboring nodes, allowing the model to capture both local and global dependencies effectively. The diagram highlights how message passing works across different nodes, with the hidden states being updated iteratively based on neighboring nodes’ information.

Deriving affective and adjacency graphs

To effectively model sarcastic statements, we transform each sentence into a graph representation that captures both syntactic dependencies and affective information (Lou et al., 2021; Wang et al., 2023). Inspired by Li et al. (2021), we first create a dependency adjacency graph, D, which forms an $n\times n$ matrix, where $n$ is the number of words in the sentence. This graph encodes word-to-word connections based on the dependency tree T of the sentence, which captures syntactic relationships.

For example, consider the sarcastic sentence, “I truly enjoy it when folks never call me.” In this sentence, words like “call,” “folks,” and “never” establish syntactic relationships that reflect the sarcastic tone. The sentence is transformed into a dependency graph where each word is treated as a node, and the syntactic dependencies between words (as defined by the dependency tree) are represented as edges between the nodes, as shown in Fig. 4.

(6) $$D_{i,j}=1\mathrm{i}\mathrm{f}T(w_i,w_j).$$

In this equation, $D_{i,j}$ represents the syntactic dependency between words $w_i$ and $w_j$ in the sentence. The dependency graph D is undirected, meaning that $D_{i,j}=D_{j,i}=1$ for all connected word pairs, and we also assign self-loops $D_{i,i}=1$ to preserve the individual context of each word. Figure 4 illustrates an example of a dependency graph derived from this sentence.

Next, we construct an affective graph using SenticNet, which provides sentiment scores for words based on external affective common sense knowledge. For each word $w_i$ in the sentence, we retrieve its affective score $S(w_i)\in [0,1]$. If a word is not found in the knowledge base, we set its score to 0. Continuing with the example sentence “I truly enjoy it when folks never call me,” words like “enjoy” and “call” might have different sentiment scores, and the sentiment reversal with “never” further highlights sarcasm. The affective graph, as visualized in Fig. 5, shows how sentiment scores are assigned and affective distances between words are computed.

(7) $$A_{i,j}=|S(w_i)-S(w_j)|.$$

To ensure proper alignment of syntactic and affective information, we combine the dependency and affective graphs into a unified representation. Specifically, the dependency graph captures grammatical relationships between words, while the affective graph encodes emotional contrasts. By treating these graphs as complementary adjacency matrices, we allow the model to process both syntactic structures and sentiment dynamics simultaneously. This integration is achieved during the GGNN processing stage, where the dependency and affective relationships are used as inputs for iterative message passing.

The alignment process also accounts for potential noise in the data, such as slang, emojis, or abbreviations. For words not found in the SenticNet resource, we assign a default sentiment score of 0. Similarly, syntactic dependencies involving such words are retained in the dependency graph to preserve the overall sentence structure. This robust alignment ensures that both contextual meaning and sentiment information are effectively captured, even for noisy or informal text.

This formula calculates the affective distance between words $w_i$ and $w_j$, reflecting the difference in their sentiment scores. The combination of both the dependency and affective graphs enriches the representation of each sentence, allowing the model to capture both structural (syntactic) and emotional (sentiment-based) information (Ghosh et al., 2018; Castro, Cerezo & Fernandez-Martinez, 2020).

For every sentence in the dataset, we generate an affective dependency graph where words are treated as nodes and connections between words are treated as graph edges. The graph is undirected and contains self-loops, ensuring that both the contextual meaning and sentiment of each word are preserved. These graphs serve as inputs to our sarcasm detection model, enabling it to understand both the syntactic structure and sentiment shifts within the text, which are key to identifying sarcasm.

Finally, these graph-based representations are used to feed into the model, where the syntactic and affective relationships between words, as depicted in Figs. 4 and 5, help infer sarcastic probabilities for each sentence.

Model training

The entire model is trained end-to-end using binary cross-entropy loss. Given the predicted probability $\hat{y}$ and the ground truth label $y$ (where $y=1$ indicates sarcasm and $y=0$ indicates non-sarcasm), the loss is computed as:

(8) $$\mathcal{L}=-\left(y\log (\hat{y})+(1-y)\log (1-\hat{y})\right).$$

The model is optimized using the Adam optimizer (Kingma & Ba, 2014), with a learning rate of $1\times {10}^{-4}$, ${\beta }_1=0.9$, ${\beta }_2=0.999$, and $ϵ=1\times {10}^{-8}$. A dropout rate of 0.3 is applied to prevent overfitting. The model is trained for 20 epochs with early stopping if no improvement is observed after five epochs.

(9) $$\hat{y}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\mathbf{W}}_o{\mathbf{H}}^{(T)}+b_o),$$

The final graph representations ${\mathbf{H}}^{(T)}=\{h_1^{(T)},h_2^{(T)},\dots ,h_n^{(T)}\}$ are passed through a linear transformation followed by a softmax layer to produce the final sarcasm classification.

where ${\mathbf{W}}_o$ and $b_o$ are the learnable parameters of the output layer.

Loss function and optimization

In this work, sarcasm detection is treated as a binary classification problem, where the model predicts whether a given text contains sarcasm. Let $\hat{y}$ represent the predicted probability of sarcasm, and $y\in \{0,1\}$ denote the ground truth label, where $y=1$ indicates sarcasm and $y=0$ indicates non-sarcasm.

Research framework

The overall framework of this study is designed to systematically evaluate the proposed BERT-GGNN model for sarcasm detection. Two publicly available benchmark datasets, Riloff and Headlines, were used for evaluation, both containing sarcastic and non-sarcastic samples that provide a challenging and comprehensive test environment. The datasets were preprocessed to remove noise and split into training and test sets with an 80–20% ratio. The model architecture integrates BERT for generating contextual embeddings, GGNN for capturing dependencies between words, and a multi-head attention mechanism to focus on explicit and implicit cues of sarcasm. To ensure optimal performance, hyperparameters such as learning rate, batch size, dropout rate, and the number of attention heads were fine-tuned through preliminary experiments. The model’s performance was evaluated using accuracy and F1 score, providing a comprehensive assessment of both precision and recall in classification. An ablation study was conducted to assess the contributions of individual components by systematically removing or replacing key elements, such as the BERT layer, GGNN, and multi-head attention. Finally, the proposed model was compared with several baseline models, including feed-forward neural network (FNN), CNN, and BERT+GCN, demonstrating its superior performance, particularly in handling complex sarcastic expressions found in real-world datasets. This research framework provides a systematic and comprehensive approach to evaluating the model’s performance and its ability to adapt to diverse linguistic structures in sarcasm detection.

Datasets and parameter settings

In this study, two publicly available datasets were used to evaluate the proposed sarcasm detection model: the Riloff dataset and the Headlines dataset. Both datasets contain a mix of sarcastic and non-sarcastic content, providing a challenging and comprehensive benchmark for evaluating the model’s ability to distinguish between literal and sarcastic language.

The Riloff dataset was originally curated for sarcasm detection and contains a smaller number of samples compared to the Headlines dataset. The data primarily comes from social media posts, where short and concise sarcastic statements are common. This dataset is highly imbalanced, with significantly more non-sarcastic samples than sarcastic ones. To address this imbalance, oversampling techniques were applied to the sarcastic class during training, ensuring better representation in the model’s learning process. Additionally, preprocessing steps included replacing emojis with textual descriptions (e.g., T _ T converted to “sad”) and normalizing common slang expressions to standard English. These steps helped reduce noise and improve the dataset’s compatibility with the proposed model.

The Headlines dataset comprises news headlines, where sarcasm often appears in the form of ironic or exaggerated statements. The headlines were collected from various online news outlets and annotated for sarcasm. Compared to the Riloff dataset, the Headlines dataset has a more balanced distribution between sarcastic and non-sarcastic samples, offering a more varied linguistic structure. Preprocessing for this dataset included tokenization, removal of stop words, and lowercasing to ensure consistency and standardization.

To ensure a robust evaluation of the model’s generalization capabilities, the training and test data for both datasets were split such that 80% of the data was used for training, and the remaining 20% was reserved for testing. This split was performed randomly, ensuring that sarcastic and non-sarcastic examples were well-represented in both sets. The statistics for both datasets are summarized in Table 1. These preprocessing and splitting strategies ensured the data was well-prepared for training and evaluation.

The model was implemented using the PyTorch framework and trained with a range of hyperparameters optimized for the sarcasm detection task. Below, we provide a detailed breakdown of the settings used during the training process, as shown in Table 2.

To ensure optimal performance for the sarcasm detection task, various experimental parameters were carefully selected and fine-tuned through preliminary experiments. The dropout rates for both the attention probabilities and hidden layers were set at 0.1 to prevent overfitting, which helps the model generalize better by randomly omitting certain weights and neurons during training. The hidden activation function used is GELU, known for its smooth non-linearity that enhances model stability and performance compared to the traditional ReLU activation. A learning rate of $1\times {10}^{-4}$ was selected to ensure the model’s convergence in a stable and efficient manner, allowing small updates at each training step without overshooting the loss function minimum. The hidden size of 768 and intermediate size of 3,072 allow the model to capture complex patterns in the input, while the multi-head attention mechanism with 12 heads ensures that the model attends to different aspects of the text simultaneously (Vaswani, Shazeer & Parmar, 2017). A batch size of 32 strikes a balance between model performance and efficient use of computational resources, and 50 epochs were set to give the model ample opportunities to learn from the data, with early stopping applied to avoid overfitting. The Adam optimizer was chosen due to its efficiency in handling sparse gradients, using ${\beta }_1=0.9$, ${\beta }_2=0.999$, and $ϵ={10}^{-8}$ to ensure adaptive learning rates during training (Kingma & Ba, 2014). These settings were benchmarked in alignment with recent studies in sentiment and sarcasm detection, ensuring the selection of state-of-the-art configurations (Devlin et al., 2019).

Experimental results

We compared the performance of the proposed BERT+GGNN model against the baseline FNN (You, Gao & Katayama, 2014), CNN, and BERT+GCN models on two datasets: the News Headlines and Riloff datasets. Both models were evaluated using accuracy and F1-score metrics. The results across various epochs are presented in Table 3. In addition, the two main comparison model training process indicators were visualized, as shown in Figs. 6 and 7.

As illustrated in Figs. 6 and 7, the BERT+GGNN model consistently outperforms the BERT+GCN model in terms of both accuracy and F1 scores. This improvement is particularly evident on the Riloff dataset, where sarcasm detection presents greater challenges. Sarcastic expressions in the Riloff dataset often require a deeper understanding of external knowledge and implicit contextual cues, making them harder to detect. While the BERT+GGNN model performs well across both datasets, its performance is slightly lower on the Riloff dataset due to the need to interpret these more complex and nuanced sarcastic expressions. In contrast, sarcasm in the Headlines dataset tends to be more direct, allowing the model to more easily capture the necessary patterns. Therefore, the accuracy difference can be attributed to the increased complexity of the Riloff dataset, where sarcasm relies more on subtle, context-driven cues, posing a greater challenge even for advanced models like BERT+GGNN.

Table 4 provides a comparison of the performance of large-scale models (GPT-3.5-turbo, GPT-4o-mini, Qwen2.5, and the proposed model) across three datasets: Headlines, Riloff, and Reddit. The proposed model demonstrates competitive performance, showcasing its ability to handle both structured and unstructured sarcasm detection tasks effectively. On the Headlines dataset, the proposed model achieves the highest accuracy (0.908), precision (0.904), recall (0.905), F1-score (0.904), and AUC (0.9053). This superior performance is attributed to the structured nature of the Headlines dataset, where sarcasm is often explicitly expressed. By leveraging GGNN and attention mechanisms, the model effectively captures both direct patterns and contextual cues, surpassing GPT-3.5-turbo, GPT-4o-mini, and Qwen2.5.

Table 4 provides a comparison of the performance of large-scale models (GPT-3.5-turbo (Anand et al., 2023), GPT-4o-mini (Hurst et al., 2024), Qwen2.5 (Yang et al., 2024)) across three datasets: Headlines, Riloff, and Reddit. The proposed model demonstrates competitive performance, showcasing its ability to handle both structured and unstructured sarcasm detection tasks effectively. On the Headlines dataset, the proposed model achieves the highest accuracy (0.908), precision (0.904), recall (0.905), F1-score (0.904), and AUC (0.9053). This superior performance is attributed to the structured nature of the Headlines dataset, where sarcasm is often explicitly expressed. By leveraging GGNN and attention mechanisms, the model effectively captures both direct patterns and contextual cues, surpassing GPT-3.5-turbo, GPT-4o-mini, and Qwen2.5. To ensure a fair comparison of sarcasm detection capabilities, all models—GPT-3.5-turbo, GPT-4o-mini, and Qwen2.5—employed the same prompt: “Determine whether the following sentence contains sarcasm. Output 1 if sarcastic, and 0 if not.” This standardized prompt allowed for a consistent evaluation of their respective performances across the different datasets, isolating the models’ inherent abilities to detect sarcasm.

On the Riloff dataset, the proposed model achieves an accuracy of 0.864 and an F1-score of 0.785, demonstrating competitive performance. However, its recall (0.753) is lower compared to GPT-4o-mini (0.981), suggesting that the model occasionally struggles to identify subtle sarcastic expressions in unstructured social media posts. This limitation may stem from the noisy nature of the dataset or the lack of external contextual information. Nonetheless, the model’s overall performance, including its AUC (0.753), indicates robustness in handling informal and challenging datasets.

For the Reddit dataset, the proposed model achieves strong results, including an accuracy of 0.832, precision of 0.801, recall of 0.821, F1-score of 0.820, and AUC of 0.801. Although Qwen2.5 achieves a slightly higher AUC (0.818), the proposed model excels in precision and F1-score, highlighting its ability to effectively capture sarcasm in conversational and informal contexts. The GGNN component facilitates the capture of relationships between words and sentences, while the attention mechanism identifies sarcasm-related cues such as contradictions and emotional shifts. These results underscore the model’s robustness and adaptability compared to GPT-3.5-turbo, GPT-4o-mini, and Qwen2.5.

Cross-dataset validation analysis

To evaluate the generalizability of the proposed model, we conducted cross-dataset validation experiments using three sarcasm datasets: Headlines, Riloff, and Reddit. These datasets represent different linguistic structures and sarcasm characteristics, which significantly influence the model’s performance when trained on one dataset and tested on another. The detailed results are presented in Table 5.

Cross-dataset validation results reveal the significant impact of dataset characteristics—such as linguistic structure, content, and sarcasm expression—on model performance. When the model was trained on the Headlines dataset and tested on the Riloff dataset, it achieved moderate performance (accuracy: 0.701, F1-score: 0.687) due to structural differences between the two datasets. The Headlines dataset features well-structured, concise news headlines with explicit sarcasm markers, whereas the Riloff dataset consists of informal social media posts with noise such as slang and fragmented sentences, challenging the model’s generalization. Similarly, training on Headlines and testing on Reddit resulted in better performance (accuracy: 0.786, F1-score: 0.772), as Reddit posts, while informal, exhibit more structured tones and longer contextual information, partially aligning with news headlines.

Conversely, training on Riloff and testing on Headlines achieved lower performance (accuracy: 0.683, F1-score: 0.667) as the model adapted to noisy, informal data struggled with structured and concise sarcasm in headlines. In contrast, training on Riloff and testing on Reddit yielded the highest cross-dataset performance (accuracy: 0.802, F1-score: 0.792), likely due to their similar informal and conversational tones, reducing the distribution gap.

The lowest performance (accuracy: 0.605, F1-score: 0.587) was observed when training on Reddit and testing on Headlines, highlighting the difficulty of generalizing from informal, user-generated content to structured news headlines. Similarly, moderate results were obtained when training on Reddit and testing on Riloff (accuracy: 0.722, F1-score: 0.707), as the higher noise levels and shorter texts in Riloff posed additional challenges.

These findings emphasize the importance of aligning linguistic and contextual characteristics between training and testing datasets to improve sarcasm detection. Future work could explore domain adaptation techniques, such as transfer learning or adversarial adaptation, to bridge the distribution gap. Additionally, incorporating multimodal information, such as emojis and metadata, or pretraining on large, diverse datasets across domains, could enhance the model’s robustness and generalizability in handling varied sarcasm expressions.

Error analysis

Despite the strong performance of the BERT+GGNN model, a comprehensive error analysis reveals areas where the model encounters difficulties. These insights provide valuable guidance for addressing the inherent challenges of sarcasm detection, particularly in cases where subtle linguistic nuances or contextual dependencies play a significant role.

To better understand the model’s behavior, we analyzed its performance across different categories of sarcastic and non-sarcastic content. Table 6 presents the performance metrics for each category, highlighting the challenges associated with implicit and context-dependent sarcasm. The model excels in detecting explicit sarcasm and non-sarcastic content but struggles with more nuanced forms of sarcasm, where external knowledge or deeper contextual understanding is required.

A detailed examination of failure cases reveals three primary issues: over-reliance on trigger words, challenges with ambiguous sentences, and difficulty handling long or complex structures. Table 7 illustrates these failure types with specific examples. For instance, the model tends to misclassify sentences like “Advancing the world’s women,” over-interpreting the word “advancing” as sarcastic. Similarly, the lack of external context often leads to incorrect predictions for sentences relying on implicit or juxtaposed meanings.

Furthermore, attention visualization reveals additional challenges with the model’s decision-making process. In several misclassified cases, the model disproportionately focuses on irrelevant tokens, such as conjunctions or determiners, while neglecting critical words conveying sarcasm. For instance, in the sentence “Christian Bale visits Sikh temple victims,” the model fails to prioritize words that signal sarcasm, instead focusing on less relevant elements of the sentence structure. Similarly, scattered attention patterns in longer sentences often result in the model missing key sarcastic cues, especially when these cues are dispersed across multiple clauses.

Ablation study

To evaluate the contributions of different components in our model, we conducted an ablation study. In this study, we removed or replaced key components, such as the BERT layer, GGNN, and SVM (Wang & Hu, 2005), and observed the performance impact on both the Headlines and Riloff datasets. Additionally, we experimented with different numbers of multi-head attention mechanisms to see their effect on the model’s performance. The results are shown in Table 8.

The ablation study results highlight the importance of both the BERT and GGNN components in achieving optimal performance. As shown in the table, removing or replacing these components significantly reduces the model’s accuracy and F1 score. For instance, replacing GGNN with CNN reduces the accuracy on the Headlines dataset from 0.9200 to 0.6880, and the F1 score drops from 0.9151 to 0.6739. Similarly, on the Riloff dataset, the performance drops from 0.8311 to 0.6959 in terms of accuracy when using CNN instead of GGNN. The BERT layer also plays a crucial role in understanding the contextual relationships in the text. Removing BERT and using only GGNN results in a significant drop in performance, especially on the Riloff dataset, where the F1 score drops from 0.7727 to 0.5815.

The BERT layer also plays a crucial role in understanding the contextual relationships in the text. Removing BERT and using only GGNN results in a significant drop in performance, especially on the Riloff dataset, where the F1 score drops from 0.7727 to 0.5815. This drop highlights BERT’s effectiveness in providing high-quality contextual embeddings that capture word-level semantics and their interactions within the broader context. Without BERT, the model struggles to interpret nuanced sarcastic cues that depend on subtle contrasts or contextual shifts, particularly prevalent in social media posts.

According to Table 8, introducing multi-head attention significantly improves the model’s performance compared to not using it (e.g., the ‘BERT+GGNN’ model). For instance, on the Riloff dataset, adding four attention heads (as in ‘BERT+GGNN+MulAttention’ (4)) improves the F1 score to 0.8659. In the Headlines dataset, the F1 score is also comparable to the model without multi-head attention, showing that attention improves the precision of predictions.

In this study, the selection of the number of attention heads in the multi-head attention mechanism played a significant role in the model’s performance. The results demonstrated that using eight attention heads provided the best balance between model complexity and performance. For instance, with four heads (e.g., BERT+GGNN+MulAttention (4)), the F1 score on the Riloff dataset increased from 0.7727 (without attention) to 0.8659, indicating that the introduction of attention allowed the model to better capture essential features in the text. However, increasing the number of heads from 4 to 8 further improved performance, especially on the Headlines dataset, achieving a maximum F1 score of 0.9154, which suggests that the model benefited from the ability to attend to more nuanced relationships within the data.

Nevertheless, when the number of heads was increased to 16 (e.g., BERT+GGNN+MulAttention (16)), performance began to drop slightly. This decline in performance may be attributed to overfitting or the introduction of redundant information, highlighting that while more heads provide more detailed information, too many heads can negatively impact model generalization. This finding aligns with prior research, which suggests that a moderate number of attention heads typically leads to optimal performance (Vaswani, Shazeer & Parmar, 2017).

Performance across structured and unstructured data

This research highlights the significant advancements achieved through the BERT+GGNN+Attention model, particularly in terms of F1 score. The model’s robust performance in handling both structured and unstructured data exemplifies its versatility. On the structured and relatively balanced Headlines dataset, the model outperformed baseline models in both accuracy and F1 score. The structured nature of this dataset, where sarcasm indicators in headlines are more explicit, allowed the model to effectively leverage its capability to focus on key sarcasm-signifying phrases (Vaswani, Shazeer & Parmar, 2017). In contrast, the Riloff dataset, characterized by unstructured, noisy, and imbalanced social media posts with a predominance of non-sarcastic samples, posed a greater challenge. Nevertheless, the BERT+GGNN+Attention model outperformed the BERT+GCN model (Yao, Mao & Luo, 2019), further demonstrating its robustness in tackling informal and ambiguous content. Social media posts often contain fragmented language, slang, and emoticons, complicating sarcasm detection. The model’s ability to handle this complexity suggests that combining BERT’s rich contextual embeddings (Devlin et al., 2018) with GGNN’s capacity to model relationships between non-sequential words enhances its interpretation of sarcasm in informal, short-text settings. Moreover, the attention mechanism further strengthens this capacity by allowing the model to assign varying importance to different words or phrases (Vaswani, Shazeer & Parmar, 2017). Sarcasm frequently relies on specific keywords or phrases that contrast with the literal meaning of the surrounding text. By leveraging multi-head attention, the model can focus on these contrastive elements, enhancing its ability to detect sarcasm in both structured and unstructured contexts.

Challenges identified through error analysis

Despite the strong performance of the BERT+GGNN model, a comprehensive error analysis reveals areas where the model encounters difficulties. These insights provide valuable guidance for addressing the inherent challenges of sarcasm detection, particularly in cases where subtle linguistic nuances or contextual dependencies play a significant role.

To better understand the model’s behavior, we analyzed its performance across different categories of sarcastic and non-sarcastic content. The model excels in detecting explicit sarcasm and non-sarcastic content but struggles with more nuanced forms of sarcasm, such as implicit or context-dependent sarcasm. Common failure cases include over-reliance on trigger words, difficulty handling ambiguous sentences, and challenges in understanding long or complex structures. For instance, the model sometimes misclassifies sentences due to misplaced attention, focusing on irrelevant tokens or neglecting critical words that signal sarcasm. These challenges underscore the need for improving the model’s contextual understanding and refining its attention mechanisms. Integrating external knowledge or employing hierarchical attention structures could help mitigate these issues in future iterations.

Advancements over existing models

The proposed model demonstrates significant advancements over previous approaches to sarcasm detection, addressing limitations in earlier methodologies. Traditional models, such as support vector machines (SVM) and CNNs (Yao, Mao & Luo, 2019), relied on hand-engineered features or shallow neural architectures, which restricted their ability to capture the complex nuances of sarcasm. While effective for structured datasets, these methods often struggled with unstructured data, particularly when sarcasm was implicit.

Transformer-based models like BERT (Devlin et al., 2018) marked a turning point by capturing deep contextual representations, but they struggled to model long-range dependencies and non-sequential relationships in unstructured texts. CSDGCN (Li, Chen & Tang, 2022), which combines BERT with GCNs, introduced graph-based representations and achieved an F1-score of 81.67% on the Riloff dataset. However, its reliance on static graph structures and sequential dependencies limited its adaptability to informal, fragmented texts found in social media.

The proposed BERT+GGNN+Attention model fundamentally differs by introducing GGNNs and self-attention mechanisms. GGNNs capture dynamic, non-sequential relationships, enabling the model to handle dispersed sarcasm cues in unstructured texts. The multi-head attention module further enhances this by focusing on sarcasm-indicative elements, regardless of their position or context. These innovations allow the model to effectively bridge the gap between structured and unstructured data. Quantitatively, the model achieves an F1-score of 86.59% on the Riloff dataset, outperforming CSDGCN (F1 = 81.67%), demonstrating superior balance between precision and recall.

Compared to other state-of-the-art approaches, such as hierarchical graph neural network (HGNN) (Xu, Zhao & Liu, 2023), which uses hierarchical structures to capture sentence- and document-level sarcastic cues, the proposed model is more adaptable to short, informal texts. HGNN achieves an F1-score of 83.10% on the Riloff dataset but relies heavily on hierarchical representation, limiting its flexibility. In contrast, the proposed model’s dynamic architecture excels in detecting both local and global sarcastic patterns without relying on rigid structures.

In summary, the BERT+GGNN+Attention model effectively addresses the limitations of prior approaches by combining contextual embeddings, dynamic graph modeling, and attention mechanisms. These advancements enable it to outperform existing models in both structured and unstructured sarcasm detection tasks, highlighting its potential for real-world applications.

Experimental design and insights

The experimental design of this study was well-structured to test the proposed model’s generalization across different text types. The inclusion of both the Riloff and Headlines datasets allowed for comprehensive evaluation across structured and unstructured data, ensuring that the model was not overfitted to one specific data type. This dual-dataset approach enhances the generalizability of the results, providing a more realistic assessment of the model’s potential for real-world applications.

Key hyperparameter choices, such as a learning rate of $1\times {10}^{-4}$, batch size of 32, and dropout rate of 0.3 (Ruder, 2017), were carefully calibrated to ensure stable convergence during training. The use of early stopping (Goodfellow, Bengio & Courville, 2016) further prevented overfitting, particularly on the smaller Riloff dataset. These practices are in line with best practices in deep learning and contributed significantly to the model’s stable and strong performance.

A notable finding is that the BERT+GGNN model excelled on the Riloff dataset, despite its imbalanced and informal nature. This dataset mirrors real-world social media environments, where sarcasm detection is often more difficult due to fragmented linguistic structure and noise from abbreviations, misspellings, and emojis. The model’s success on this dataset underscores its robustness and its potential for practical applications in sentiment analysis, content moderation, and other natural language processing tasks on social media platforms.