Fast2Vec, a modified model of FastText that enhances semantic analysis in topic evolution

Topic model for topic evolution

Topic models have been widely applied across various disciplines to analyze diverse text types, ranging from short texts to extensive documents. Initially defined by Blei, Griffiths & Jordan (2010), topic modeling is a statistical method that identifies latent semantic structures and underlying topics within large datasets (Churchill & Singh, 2022). Further, topic modeling is an unsupervised mathematical model that processes document collections to generate topics that accurately and coherently represent their content. Researchers have employed topic models in various domains, including the analysis of climate change literature (Sleeman et al., 2017), precision medicine through genomic and diabetes literature (Ni Ki et al., 2022), and strategy discussions related to global issues like the war in Ukraine (Maathuis & Kerkhof, 2023). Additionally, topic modeling has been instrumental in detecting fraud (Sánchez & Urquiza, 2024) and mapping research trends (Lou & Meng, 2023; Hu et al., 2020; Chen et al., 2020).

LDA, one of the most widely used topic modeling methods, conceptualizes topics as probability distributions over words and documents as distributions over topics (Blei, Ng & Jordan, 2003). LDA identifies thematic structures by analyzing word co-occurrence patterns, making it applicable to various natural language processing tasks (Churchill & Singh, 2022). However, LDA faces limitations in addressing temporal dynamics. To overcome this, Blei & Lafferty (2006) introduced the DTM, which incorporates time-based elements to track topic evolution. DTM has been utilized to analyze scientific entities, identify research trends, and categorize them into mainstream, short-term, and long-term directions (Lou & Meng, 2023). Applications of DTM extend to geographical planning and policy (Yao & Wang, 2020; Jiang et al., 2022) and exploring online communities engaged in abortion debates (Pleasants et al., 2023). Despite these advances, both LDA and DTM struggle with capturing semantic regularity and contextual shifts, as Kai et al. (2019) and Geeganage, Xu & Li (2021) have noted. This limitation underscores the importance of semantic analysis in understanding topic evolution, particularly in tracking changes in contextual meaning over time (Hu et al., 2017; Li et al., 2019; Chen et al., 2017).

Semantic analysis in topic evolution

Recent advancements in computational semantic technologies have driven the development of word-embedding techniques, enabling more nuanced analysis of meaning changes at the word level. Word2Vec, for instance, facilitates the exploration of semantic regularity in documents by analyzing word relationships based on contextual co-occurrence frequencies (Gao et al., 2022). Researchers have leveraged Word2Vec to uncover semantic patterns, such as Hamilton, Leskovec & Jurafsky (2018), who demonstrated how historical changes could be captured by comparing word embeddings. Similarly, Geeganage, Xu & Li (2021) utilized semantic patterns to interpret textual meaning, while Aboelela, Gad & Ismail (2021) applied semantic-based opinion-mining techniques to support decision-making.

Despite its utility, Word2Vec exhibits significant limitations, particularly in handling OOV words due to its reliance on fixed vocabularies and lack of morphological processing (Choi & Lee, 2020). To address these challenges, Bojanowski et al. (2017) proposed FastText, which extends Word2Vec by incorporating subword information through n-gram character representations, enabling improved handling of OOV words and morphological variation. FastText has been shown to enhance the understanding of word prefixes and suffixes (Athiwaratkun, Wilson & Anandkumar, 2018). It also excels in syntactic analysis, particularly in language with complex morphologies (Selva Birunda & Kanniga Devi, 2021). Asudani, Nagwani & Singh (2023) further highlighted that FastText, developed by the Facebook AI Research (FAIR) lab effectively generates vectors for unknown words based on their morphology. Furthermore, Zhang, Gao & Yuan Fang (2020) demonstrated the applicability of FastText in extending LDA to shorter sentences, improving semantic regularity in text analysis.

Most of the latest approaches in representation learning are embodied in BERT, which provides context-solid embeddings that distinguish both directional meanings (George & Sumathy, 2023). BERT can successfully be employed as a practical feature extraction technique, and a more recent work, BERT EmHash, employed BERT within the EmHash framework for hashtag recommendation tasks, demonstrating its effectiveness. However, this performance is optimal when executing the necessary classification set of tasks instead of the exploratory or topic distribution set of functions. Such encoding by BERT may introduce distortions such as excessive representations of hashtags or over-representation of frequently used terms, which would undermine the necessary semantic coherence in topic modeling and study of the evolution of the topic (Kaviani & Rahmani, 2020). It is widely recognized that BERT performs much better in tasks that do not require substantial semantic knowledge; however, its use in the context of topic dynamics remains modest.

Fast2Vec and the LDA integration model

Previous research highlighted the importance of combining topic models with word embedding techniques. Niu & Dai (2015) introduced Topic2Vec, which integrates LDA and Word2Vec to represent topics in a semantic vector space. Zhang, Gao & Yuan Fang (2020) also combined LDA with similar methods to enhance topic coherence by utilizing external corpus information using cosine distance to measure text-topic similarities in the semantic space. This approach has shown superior performance in topic categorization compared to other methods.

The Fast2Vec model integrates the FastText and Word2Vec algorithms, merging their strengths in word representation. FastText captures sub-word information, while Word2Vec generates vectors that capture semantic relationships. Fast2Vec is further integrated with LDA to improve topic analysis by identifying hidden document structures. The Fast2Vec algorithm consists of three main stages: FastText training, Word2Vec training, and merging the models. FastText training involves initializing subwords and words, calculating their weights, and using the Continuous Bag of Words (CBoW) model with negative sampling to predict target words, followed by weight updates via gradient calculations. Word2Vec training follows the same process but focuses solely on words without subwords. The final stage merges the vector matrices from FastText and Word2Vec.

Integrating Fast2Vec with LDA provides several advantages, such as leveraging sub-word information and word context, leading to more accurate word representations and improved performance in identifying hidden topics. Fast2Vec also captures rich linguistic variations, mainly through morphological analysis, making it especially valuable for languages with complex morphological structures.

Using the SINTA text corpus for experiments showed that the Fast2Vec model with LDA finds more meaningful and cohesive topics than the Word2Vec or FastText vectors alone. This study shows that the Fast2Vec model can offer researchers new insights into topic development and enhance the integration of topic models with word embeddings for analyzing topic evolution. This study aims to bridge the research gap in semantic topic analysis by capturing variations in semantic distribution over time and contributing to understanding topic evolution in scientific literature.

Dataset

The dataset for this study was extracted from scientific articles published in reputable journals in SINTA between 2009 and 2023. Each document contains key metadata, including the abstract (in English), the author’s name, title, and publication year. A rigorous preprocessing workflow was implemented to prepare the data for topic analysis. Initially, the dataset was comprised of 89,345 abstracts, but after applying filters to ensure quality and relevance, only 54,689 abstracts were retained. The selected articles’ abstracts, titles, publication years, and keywords were extracted to build the corpus. The abstracts selected were those written in English. The proposed combination models were used to capture the evolution of dynamic topics, identify the semantic concepts in documents, and then analyze the semantic distribution of topics and track changes in their characteristics over time. Table 1 shows a sample of the dataset.

Preprocessing

One of the main challenges in preparing the dataset was the variation in journal templates across SINTA, which required significant effort to standardize the data. These variations led to inconsistencies in how metadata such as the author, title, and abstract were presented, necessitating additional steps to ensure uniformity. For instance, mixed or unnecessary information in the author column was cleaned to retain only the essential data. The preprocessing ensured that the final dataset was structured, consistent, and free from duplicates, enabling accurate and meaningful topic analysis. This thorough approach highlights the critical importance of data cleaning and standardization when working with diverse sources to ensure the reliability of subsequent studies.

Key preprocessing steps included removing entries with missing values in critical columns such as the abstract, title, year, or author information. Only abstracts written in English were retained to maintain consistency. Each abstract was further processed by tokenizing the text into individual words, applying lemmatization to normalize words to their base forms (e.g., “running” to “run”), and removing irrelevant elements such as stopwords, punctuation, numbers, and special characters. Words appearing fewer than five times were excluded, resulting in a clean and structured dataset suitable for reliable topic extraction.

Fast2Vec model

This section explains that Fast2vec, a modification of FastText, enhances semantic analysis in the evolution of the topic. This model was used to extract abstractions from scientific articles published in reputable journals in SINTA. The main stages of the study included topic detection, semantic analysis, and analysis of evolutionary development.

The graphical model proceeded through several steps. First, abstracts, titles, publication years, and keywords from the selected articles were extracted to build the corpus. The abstracts selected were those written in English. Second, DTM was used to capture the evolution of dynamic topics. Third, the Fast2Vec model, a combination of FastText and Word2vec, was employed to identify the semantic concepts in documents from 2009 to 2023. This helped analyze the semantic distribution of topics and track changes in their characteristics over time.

Word embedding was crucial in this study, as DTM alone cannot capture semantic shifts in topic evolution (Gao et al., 2022). To address this, the cosine distance between word vectors was computed to detect changes in the semantic distribution and dynamics of evolving topics. Figure 1 shows the general model diagram describing a five-stage framework for analyzing topic evolution using semantic and clustering approaches.

First, preprocessing involved activities such as document extraction, relevant data selection, and the application of preprocessing methods to get the text data ready. The next step, topic detection, included document classification, preparation of document-topic distribution vectors, examination of the changes in topic intensity over time, and preparation of topic movement graphs. In the third phase, the semantic analysis of the framework employed Fast2Vec for searching modified word vectors, UMAP for dimensionality reduction (McInnes, Healy & Melville, 2018), and AP clustering for constructing semantic spaces. The fourth stage, analysis of evolving dynamics, provided the estimates of the similarity of single words, topic change, topic entropy, and rate of change to identify important developments through time. The last step was the output phase, which included different illustrations, including but not limited to graphs and animations, to address the comprehensiveness of the changes in the topics concerned. This model integrates modern semantic methods and methods of clustering as a means of adequate consideration of the dynamics of topics.

Topic detection

DTM is an extension of LDA, which allows it to track topic changes over time by partitioning documents based on time slices. DTM assumes that the topic at time t-1 is the starting point for the topic at time t before changing. The model uses the Gaussian noise component and applies a spatial model that connects the natural parameters of each topic through a state space model, which describes the natural distribution of the topic ${\beta }_{k,w}^t$ at time t. The distribution of the document topic is generated using a normal distribution with an average of α.

This study divides the period from 2009 to 2023 into 15-time intervals. The perplexity index is used as an evaluation metric to determine the optimal number of topics, according to research by Chen et al. (2017). The perplexity metric typically identifies an optimal number of topics ranging between 80 and 100. However, this number was too large for this study, which could lead to overfitting, which occurs when a model tends to capture too many random variations in the data instead of significant patterns. In addition, having too large of a number of topics can make it challenging to interpret the analysis’s results.

Before experimenting, prior discussion helped the research team determine the appropriate number of topics. First, the corpus was summarized using an LDA mode, with topic numbers between 5 and 20. The second stage involved applying DTM to optimize parameter selection, estimating around fifteen topics. However, further reassessment by domain experts was conducted to refine these results.

As shown in Table 2, the perplexity and coherence scores were analyzed for different numbers of topics, revealing that eight topics provided the best balance between coherence and model performance (perplexity: 3,030.99, coherence: 0.75), compared to other topic numbers such as five topics (perplexity: 3,250.12, coherence: 0.42) or 15 topics (perplexity: 3,120.56, coherence: 0.60).

By combining quantitative evaluation (perplexity and coherence scores) with qualitative assessment from domain experts, we determined that eight topics provided the most interpretable and meaningful summarization of the corpus.

Building on these findings, Table 3 presents the document-topic distribution per period as generated by DTM. Each document is associated with a probabilistic distribution across eight topics, illustrating how thematic structure evolves over time. The probability distribution ${\theta }_{d,k}^t$ represents the strength of a document’s association with a given topic k at time t.

By analyzing these distributions, we can observe shifts in dominant topics and how their prevalence changes across different time periods. This analysis is essential for identifying emerging research trends and tracking the evolution of semantic themes within the dataset.

Determining the number of topics

We compared the Word2Vec and FastText models to evaluate the model’s performance by combining it with LDA. We used cosine distance to measure and explain similarities between documents and topics in semantic space. The results showed that FastText can handle OOV well but is less optimal in semantic analysis compared to Word2Vec. This can be seen after we eliminated OOV cases for a more balanced result. Based on these findings, we propose a new model that modifies FastText by adding weights to improve semantic representation, which we call Fast2Vec.

Suppose a word sequence. $w_1^n$ consists of n words, represented as $w_1,w_2,\dots ,w_n$. The probability of observing this sequence $,P(w_1^n)$ is computed as:

(1) $$P\left(w_1^n\right)={\prod }_{t=1}^{n}P\left(w_1^{t-1}\right)$$where $w_1^n=\left\{w_1,w_2,\dots .,w_n\right\}$, $w_t$ is the t-th word in the sequence, $w_1^{t-1}$ is the context consisting of all preceding words $\left\{w_1,w_2,\dots .,w_{t-1}\right\}$, and $P\left(w_1^{t-1}\right)$ denotes the conditional probability of $w_t$ given the context.

Topic intensity and distribution probability

Table 4 presents the topic word probability distribution and illustrates the relationship between topics and words at different periods. Each row corresponds to a specific topic (e.g., Topic 1 to Topic 8), while each column represents words within the corpus (e.g., Word 1 to Word n). The probability ${\beta }_{k,w}^t$ indicates the likelihood of a topic k being associated with the word w at time t. The probability of distribution of all words for a given topic, as well as all topics for a given document, sums to one, are expressed in Eq. (1). This table provides valuable insights into the lexical composition of topics and their evolution over time to understand how certain words define or contribute to each topic across temporal dimensions.

Table 5 displays the topic intensity values, which are used to quantify the popularity of each topic over time. A document-topic probability distribution is employed to identify trend patterns, while topic probability distributions facilitate semantic analysis. The intensity of a topic is calculated as the sum of the normalized probability distribution of topic intensity, represented by ${\sum }_{d=1}^{d=M}{\theta }_{d,k}^t$ where M is the total number of documents, as shown in Eq. (2):

(2) $${\theta }_{tk}^{\ast }={\displaystyle \frac{{\sum }_{d=1}^{d=M}{\theta }_{d,k}^t}{M}.}$$

This formula computes the aggregate probability distribution of topic k across all documents at time t, normalized by dividing it by the M total number of documents. The values in Table 5 represent the resulting topic intensity for various clusters (e.g., Cluster 1 to Cluster 10) and topics (e.g., Topic 1 to Topic 8), providing insights into the temporal trends and variations in topic popularity over several years.

Building a semantic analysis model

The semantic analysis model aims to group words with similar semantic concepts in a two-dimensional space. Before the model training stage, a word embedding model was developed using FastText and Word2vec. The Fast2Vec algorithm produces a more comprehensive representation of vectors.

Suppose a document corpus consists of $D=\left\{d_1,d_2,\dots ,d_N\right\}$ N documents. The process begins with the initialization of the parameters $\alpha ,\beta ,{\gamma }_{word},{\gamma }_{subword},d,e,w$. Where α and β are the merging weights for Word2Vec and FastText. The word and subword merging weights for the FastText model are ${\gamma }_{word}+{\gamma }_{subword}=1$, with the embedding dimension d, the number of iterations e, and the window size w.

Analysis of evolving dynamics

The topic-semantic probability distribution, represented by ${\theta }_{k,c}^t$k emerges from the relationship between topic-word distributions and semantic space. Fluctuations in this distribution indicate how topics evolve, with cosine distance between word vectors preserving conceptual differences (Kim, Kim & Cho, 2017). Semantic evolution is further analyzed using linear regression, where time is the independent variable and topic-semantic probabilities are the dependent variable. Positive coefficients suggest topic concentration in specific semantic ideas, while negative coefficients reflect dispersion across concepts—entropy measures topic concentration, with lower entropy indicating more focused topics. Entropy is calculated using $H(x)$, as described in Eq. (14).

(14) $$H\left(x\right)=-\sum _{i=1}^{n}p\left({\theta }_i\right)lo{g}_{2}p\left({\theta }_i\right).$$

Integrating UMAP for dimensionality reduction, AP for clustering, and Fast2Vec for enhanced word representation enables more coherent dynamic topic analysis. The approach reveals how topics evolved in semantic spaces, offering insights into their concentration and dispersion over time. It also contributes to a better understanding of topic evolution in scientific research.

As the analysis progresses, the model emphasizes the evolving dynamics of topics and semantics over time. By matching topic words to their corresponding semantic clusters, the model derives topic-semantic probability distributions for different periods. This enables the discovery of evolving dynamics using line charts, which capture changes in topic intensity and semantic relationships year by year. Statistical measures such as entropy and linear coefficient calculations are used to quantify these transformations further. These analyses uncover regular patterns in topic evolution and highlight significant shifts, offering a comprehensive view of how topics and their meanings evolve over time.

Visualization dynamic pattern

By resorting to detailed reporting, it becomes easier to better comprehend the intrinsic role each topic has in academic and practical matters. Each emerging topic is presented alongside river graphs, line charts, bar charts, and stacked area plots, which link to a more extensive discussion of prints and topic semantic change over time. Moreover, these reports give a better understanding of each topic’s emergence, transformation, and interaction with others. All this visualization is synthesized into visual representations. Alongside the preprocessing, topic detection, and semantic analysis, all blend in excellently, giving insights into the modification of semantics and topics contained within large-scale resolution.

Performance measurement

Perplexity was used as an evaluation metric in this study. The cross-validation technique divided the dataset into three equal parts, and the model was trained and evaluated three times using different fold combinations. In each iteration, one-fold serves as the test data, while the other two are used for training. This approach evaluates how well the model can represent the distribution of words in the test data based on the resulting topic, as Wallach et al. (2009) suggested. The perplexity score reflects the model’s uncertainty in predicting new data, with a lower score indicating better performance in capturing word distribution patterns. This score has also been consistent with other cluster quality metrics (Tang et al., 2014).

The Fast2Vec model integrates FastText and Word2Vec, enabling it to handle OOV issues effectively. As a result, it produces superior semantic representations. Comparative tests using four target words (“student,” “network,” “computer,” and “scrum”) demonstrate the closer similarity between Word2Vec and FastText, as reflected in Table 7 through cosine similarity calculations. The table provides detailed comparisons of semantic associations generated by Word2Vec, FastText, and Fast2Vec, highlighting Fast2Vec’s ability to maintain contextual relevance across diverse word categories.

Figure 2 shows the bar chart that illustrates a comparative analysis of cosine similarity scores for the semantic representation of selected words using three embedding models: Word2Vec, FastText, and Fast2Vec. Each bar corresponds to the similarity vector for a specific word, enabling an evaluation of the model’s performance in capturing semantic relationships. Across the dataset, Fast2Vec consistently outperformed Word2Vec and FastText regarding similarity scores, demonstrating its robustness in representing semantic nuances.

Evaluation on benchmark word similarity datasets

To evaluate the semantic representation capability of Fast2Vec, the model was tested on twelve standardized word similarity benchmark datasets: EN-MC-30, EN-MEN-TR-3k, EN-MTurk-287, EN-MTurk-771, EN-RG-65, EN-RW-Stanford, EN-Simlex-999, EN-Verb-143, EN-WS-353-ALL, EN-WS-353-REL, EN-WS-353-SIM, and EN-YP-130. These datasets capture a range of semantic characteristics, including similarity vs relatedness, syntactic variation, and differences in word frequency. Evaluation was conducted using Spearman’s rank correlation and Pearson’s correlation, which used to assess the relationship between model-generated cosine similarity scores and human-annotated similarity ratings (Finkelstein et al., 2001; Mukaka, 2012; Zhang & Wallace, 2015).

Fast2Vec achieved the highest Spearman correlation in seven out of 12 datasets and the highest Pearson correlation in six out of 12. It performed particularly well in datasets requiring fine-grained semantic understanding (e.g., EN-SimLex-999, EN-WS-353-SIM) and those involving rare or morphologically complex words (e.g., EN-RW-Stanford). The results are presented in Table 8.

These findings suggest that Fast2Vec produces more consistent and generalizable semantic representations than both Word2Vec and FastText. Its superior performance on linguistically complex datasets demonstrated its robustness in contexts where traditional models tend to underperform. Bu combining the contextual sensitivity of Word2Vec with the subword modeling strengths of FastText, Fast2Vec emerges as a more reliable embedding model for capturing nuanced semantic relationships across diverse linguistic settings.

Topic discovery and distribution

After defining eight topics with DTM, topic probability distributions were computed across the dataset, and representative terms were identified through expert validation. These topics cover key areas such as food availability, performance analysis, health research, and economic stability, offering insight into the dataset’s thematic diversity.

Initially, Topic 1 focuses on agriculture, natural resources, and rural development. Words such as ‘farmer,’ ‘water,’ ‘fiber, and ‘food’ emphasize farming and natural products. In addition, terms like ‘score,’ percentage, and ‘value’ indicate that measurement and evaluation play a role in this topic. This combination points toward discussions involving agricultural productivity, resource management, or rural economic assessments.

Similarly, Topic 2 emphasizes governance, institutional performance, and societal development. Terms like ‘government,’ power,’ and ‘analysis’ suggest a focus on administrative functions and decision-making processes. Moreover, words like ‘design,’ performance,’ and ‘character’ introduce the evaluation of organizational efficiency or institutional frameworks. As a result, this topic likely explores themes of public administration, governance, or strategic planning.

Topic 3 focuses on public health, surveys, and social research. The presence of terms such as ‘health,’ respondents,’ sampling,’ and ‘control’ implies a connection to empirical studies, data collection, or public health initiatives. Additionally, the recurrence of the word “number” highlights the quantitative aspect of this topic. Consequently, this cluster may represent health-related surveys, population studies, or broader social research.

Topic 4 revolves around economic systems, financial institutions, and social structures. Words like ‘finance,’ ‘bank,’ ‘company,’ and ‘local’ indicate a strong focus on economic development and financial operations. Furthermore, terms such as ‘community,’ institution,’ and ‘social’ suggest the interplay between economic growth and societal dynamics. Thus, this topic addresses local governance, economic policies, and community development.

Despite their differences, there are notable overlaps among the topics. For instance, government’ and ‘power’ appear in multiple columns, underscoring the interdisciplinary nature of governance and leadership. This repetition suggests that while the topics diverge in focus, they may collectively contribute to a broader narrative about societal progress, economic growth, and public welfare.

Visualization and topic trend analysis

Figure 3 visualizes the evolution of topic intensity from 2009 to 2023, highlighting fluctuations and trends in the prominence of each topic over time. Topic intensity is calculated based on the normalized distribution of document-topic associations for each year, reflecting the collective attention given to a particular topic within the dataset. These visualizations reveal notable differences in how heavily certain topics were emphasized during specific periods. For example, Topic 1 consistently dominated throughout the years and experience a sharp increase in prominence in 2014, indicating its heightened significance. In contrast, Topic 2 and Topic 3 exhibited moderate and shifting levels of focus, reflecting realignments in research agendas.

Topics 6 and 7 showed lower intensities, suggesting they were niche areas or of less popularity. These patterns emphasize the dynamic growth of some topics and decline of others, illustrating the evolving nature of research interests. This comprehensive analysis underscores the importance of employing effective analytical techniques, such as the proposed Fast2Vec integration, to account for the structural changes and temporal dynamics of topics over time.

Figure 4 illustrates changes in topic intensity from 2009 to 2023. Topic 1 (food availability) experienced a sharp peak in 2013–2014, reaching an intensity of 0.43, likely driven by global attention to food crises or agricultural challenges. Afterward, its intensity declined but stabilized between 0.2 and 0.3, remaining relevant throughout 2023. Topic 2 (performance analysis) showed a stable trend throughout the period, with consistent intensity levels between 0.2 and 0.3, reflecting sustained interest. Topic 3 (health and epidemiology) remained steady, with minor fluctuations and a slight increase approaching 2023, likely tied to rising interest in global health issues.

Topic 4 demonstrated relatively stable intensity between 0.1 and 0.25 without significant spikes. Topic 5 saw a notable rise at the start of the period but dropped sharply around 2015, suggesting temporary attention to forest ecosystem management. Topic 6 showed moderate fluctuations, with intensity ranging between 0.1 and 0.2, indicating a steady focus on research methodologies. Topic 7 (health and biomedical research) maintained low intensity over time, with a slight rise in 2018. Finally, Topic 8 (banking business) exhibited a steady increase, peaking in 2016, followed by a decline until 2023.

Overall, Topic 1 stands out with a significant peak in 2013–2014, while Topic 2 and Topic 4 displayed strong stability. Topic 8 showed an early rise but experienced a notable decline after its peak in 2016.

Evolution of dynamics in semantic space

Figure 5 shows the clusters representing various semantic categories connected to each topic. The evolution of the eight topics over time demonstrates distinct patterns of dominance and fluctuations across clusters. In Cluster 1, the dominant topic was Topic 1 (food availability), which peaked around 2014 and 2022, reflecting heightened discussions on local agricultural products and water availability, likely in response to global or regional agrarian challenges. Topic 2 (performance analysis) went through significant changes (especially in Cluster 4) and peaked in 2015 and 2018. This suggests that performance evaluations were more important during improved governance and operation. For Topic 3 (health and epidemiology), Cluster 1 consistently dominated, peaking in 2021, possibly due to the global focus on public health issues during the COVID-19 pandemic.

Topic 4 (economic stability) showed relative stability in Cluster 4, with peaks in 2018 and 2023, highlighting ongoing attention to financial and governance-related topics. Meanwhile, Topic 5 (environmental management) saw a decline in Cluster 1’s dominance, replaced by a significant spike in Cluster 8 in 2023, indicating a shift toward algorithmic and technical approaches to environmental research. Topic 6 (methodological research) demonstrated a change in Cluster 1, peaking in 2023, suggesting the growing relevance of statistical methods like correlation and sampling. In Topic 7 (biomedical research), Cluster 1 maintained dominance, with Cluster 10 peaking in 2018, reflecting research on public health and cholesterol-related studies. Finally, Topic 8 (Islamic banking) showed a decline in Cluster 1 over time, with a sharp increase in Cluster 4 around 2018, highlighting the periodic focus on Islamic banking management.

Overall, the analysis underscores how global, regional, and disciplinary dynamics influence the evolution of research priorities across topics.

Analysis of evolving dynamic topic patterns

Entropy measures the dispersion or concentration of a topic across different clusters. A topic spreads across multiple clusters, indicating a wide variation or diffusion in its semantic focus when its entropy value is closer to 1.3. Conversely, a lower entropy value (closer to −0.1) suggests a more concentrated focus within a limited number of clusters. Figure 6 indicates peaks and valleys in entropy for the eight topics over time. These variations reflect how each topic’s relevance and focus changed, either by becoming more diffuse (spread across clusters) or concentrated (focused on fewer semantic areas).

Topic 1: Entropy fluctuates, showing moments of both concentration and diffusion. Around 2014, entropy peaks, indicating a more varied discussion across several semantic clusters, due to diverse research interests during that period. Topic 2 (performance analysis): There was a noticeable peak in entropy around 2018, indicating that discussions on performance analysis were spread across multiple clusters, suggesting varied approaches or subtopics gaining attention simultaneously. Topic 4 (economic stability): This topic showed several spikes in entropy, particularly in recent years (2020–2023), indicating that economic discussions became more fragmented, reflecting the broadening scope of financial stability to cover different aspects like governance, finance, and societal impact.

Topic 7: Entropy remains low, suggesting that interest in this topic stayed more focused over time, concentrating on specific subtopics, such as biomedical interventions or health metrics, with less diffusion across other semantic clusters.

Diffusion vs. stability: trend insights. Topics such as Topic 1 and Topic 2 exhibited high entropy at different points, indicating periods of exploration across multiple dimensions or subtopics. On the other hand, Topic 7 maintained a low entropy trend, suggesting that the topic stayed consistently focused on a core set of ideas or clusters. Significant shifts: Sudden jumps or dips in entropy, such as the peaks seen for Topic 6 and Topic 5, indicate periods where these topics either expanded in scope or became more specialized. These shifts are crucial for understanding how research trends fluctuate over time.

The overall entropy distribution suggests a mix of diffusion, where topics spread into new semantic areas, and stabilization, where specific topics retain focus over time. The graph provides a visual illustration of the dynamic interplay between these modes, helping to track how topics evolve and shift in relevance across years.

Both Figs. 5 and 6 illustrate the semantic evolution of eight topics over time (trends), highlighting the dynamic shifts in their probability distributions. Each topic exhibits unique patterns of fluctuation, revealing periods of heightened focus and diversification. Peaks in the graph signify moments of expansion where research broadened to include various subtopics, aligning with increased entropy values. Conversely, lower-intensity periods indicate a concentrated focus on specific thematic areas, reflected in reduced entropy values.

These dynamic transitions underscore the varying nature of topic evolution, showing how some topics, such as Topics 1 and 5, maintain consistent prominence, while others, like Topics 3 and 7, exhibit intermittent surges. Clustering within each topic provides additional insights into the interrelation and differentiation of subtopics over time. This evolving semantic landscape demonstrates the complexity and fluidity of research focus, offering valuable perspectives for understanding the shifts in academic priorities and thematic structures over the analyzed period. These results show how important it is to use useful models, like Fast2Vec, to understand these complex changes and give accurate explanations of how topic structures are changing.

Semantic evolution and topic dynamics

The categorization of diffusion, stability, moderate fluctuation, and shift topics accurately reflects the dynamic nature of research development, as shown in Fig. 7. Topics like food availability and performance analysis demonstrate the broadening of the research scope over time, as indicated by high entropy values. These diffusion topics suggest increasing interdisciplinarity and incorporating new research dimensions, driven by global challenges such as food security and performance evaluation in governance and education.

In contrast, stable topics such as health and biomedical research exhibit concentrated focus, due to continuous interest in specific subtopics, particularly during major health crises. Low entropy value for these topics suggests a lack of significant semantic shifts over time.

Shifts and fluctuations in research focus

Moderate fluctuation topics, such as forest ecosystem management, show periodic intensity spikes, often in response to external events or new technologies. Though dynamic, these topics maintain an overall steady trajectory. Shifts topic like economic stability demonstrates significant changes across clusters over time, often triggered by socio-political and economic events, as reflected in increasing entropy in later years.

Fast2Vec model and semantic clustering

By combining subword information from FastText with contextual sensitivity from Word2Vec, Fast2Vec effectively capture nuances semantic variation within topic clusters. The use of UMAP and AP further supports the temporal interpretation of topic structures. These results confirm Fast2Vec’s utility for semantic-based topic modeling, particularly in handling OOV terms.

Evaluation of semantic similarity

The findings presented in Table 8 provide strong empirical support for the semantic reliability of the Fast2Vec model. Fast2Vec consistently performed well across twelve word similarity datasets, covering diverse linguistic features such as similarity vs. relatedness, Part-of-Speech (POS) categories, and term frequency. It achieved the highest Spearman correlation in seven datasets and the highest Pearson correlation in six, indicating strong alignment with human-annotated similarity judgments.

Fast2Vec outperformed both Word2Vec and FastText on several challenging benchmarks. These include EN-Simlex-999, EN-RW-Stanford, and EN-Verb-143. Notably, Word2Vec yielded a negative Spearman correlation (−0.0168) on the EN-Verb-143 dataset, underscoring its difficulty in modeling syntactic variation, an area where Fast2Vec demonstrated clear robustness. While FastText showed strengths in capturing subword-level patterns, it occasionally lacked contextual coherence. conversely, Word2vec remained effective on lexical stable datasets, such as EN-MEN-TR-3k and EN-YP-130, but struggled with infrequent or morphologically rich terms.

Overall, the results underscore Fast2Vec’s ability to integrate subword modeling and contextual semantic representation. This integration produce robust and consistent outcome in word similarity tasks. These findings position Fast2Vec as a promising hybrid embedding model capable of capturing nuanced semantic relationship across diverse linguistic settings.

Implication of entropy trends

Entropy, as a measure of topic dispersion or concentration, provided more profound insights into how topics evolved in semantic space. Topics with high entropy values are indicated to be widespread across semantic clusters, reflecting a diffusion of ideas or the emergence of new subtopics. On the other hand, low entropy values highlighted concentrated areas of research where topics remained focused on specific semantic themes. Particularly in topics like performance analysis, the peaks in entropy indicate periods of simultaneous pursuit of multiple research directions. This spread may reflect increased interdisciplinary collaboration or diversification of methods and approaches within the topic.

Broader research implications

The topic categorization reveals that diffusion topics reflect the need for adaptable research paradigms, while stable topics offer essential continuity. Shift and diffusion topics highlight responsiveness to external stimuli, guiding future prioritization. The combines semantic evaluation and topic dynamics analysis affirm Fast2Vec’s role as a powerful model for exploring and interpreting topic evolution over time and contributes both to the refinement of topic modeling frameworks and the advancement of hybrid word embedding techniques for semantic analysis.

These findings align with prior research, such as Zhang et al. (2019), which proposed a generative model to detect dynamic topical communities by integrating topic evolution with author networks. While their approach emphasized structural and probabilistic modeling, the present study offers a complementary perspective by focusing on semantic representation using hybrid embeddings and entropy-based topic categorization. This integrative strategy enhances interpretability and validates topic evolution with quantitative semantic similarity benchmarks aligned to human judgments.