Development of hybrid approach for named entity recognition in Uzbek language text

About Uzbek language

The Uzbek language is the official language of the Republic of Uzbekistan, and by its nature belongs to the Turkic language family, where most languages are agglutinative (Mengliev, Barakhnin & Abdurakhmonova, 2021). The property of agglutinativity itself implies the formation of word forms by concatenating affixes (endings) to the root of the word (Sharipov et al., 2022, 2023). Thus, we can get a word in another case or tense (past, present, etc.). For example, consider the word uy (house):

$$Uy+lar+imiz,$$here, we added -lar (plural ending) and -imiz (possessive ending), after which we got the word uylarimiz (our houses) (Madatov, Bekchanov & Vici, 2023).

Moreover, compared to other related Turkic languages, such as Kazakh or Turkish, the agglutinative nature of Uzbek has become so prominent that the addition of affixes can even result in a word with a completely different meaning (Gilmullin et al., 2024). For instance, starting with the word oq (white):

$$Oq+lash$$adding the affix -lash (a verbalizing suffix) produces the verb oqlash (to whiten), while adding the affix -landi results in oqlandi (justified). If we instead add the affix -ladi, we get the word:

$$oq+ladi$$which can be translated as justified, in the sense of fulfilling. This is commonly used in phrases such as: umidimni oqladi (fulfilled my hope).

Furthermore, it is important to specifically highlight the dialectal forms of the Uzbek language (Raxmatova & Kuzibayeva, 2021). While dialects primarily exist in spoken language, their influence can also be observed in written texts (Turaeva, 2015). For example, imagine a dialogue between three users in a social messenger. Each of these users predominantly communicates in their “own” Uzbek dialect, specifically the Karluk (standard Uzbek), Kipchak, and Oghuz dialects (Mengliev et al., 2024a).

In practice, it is quite common to encounter situations where each of the aforementioned users employs their dialect during a conversation. For clarity, a table (Table 1) is provided below, showcasing words in different Uzbek dialects along with their translations. The Kipchak dialect predominantly dominates in the Autonomous Republic of Karakalpakstan, while the Oghuz dialect is actively used in the Khorezm region (a neigh-boring area to Karakalpakstan and Turkmenistan) (Mengliev et al., 2023a).

Rule-based algorithms

Letter translation algorithm

The preprocessing stage begins by checking each character in the text to determine whether it is Cyrillic. If so, the algorithm replaces the Cyrillic character with its Latin equivalent. The pseudocode and flowchart of this algorithm are shown in Fig. 5.

Dialect word replacement algorithm

Once the text has been transliterated, it is processed by an algorithm that searches for and replaces dialect words with their Karluk (standard Uzbek) equivalents. Initially, the algorithm uses a dictionary-based approach, replacing successfully identified dialect words with their equivalents from the dictionary. If the algorithm cannot find an equivalent, it performs a morphological analysis to extract the root of the word and repeats the replacement steps. If a match is found, the word is replaced. Otherwise, the morphological analysis is repeated (up to three iterations in total). If the third iteration is unsuccessful, the algorithm skips the word. The pseudocode and flowchart for this algorithm are shown in Fig. 6.

Dialect normalization is applied strictly before tokenization and Begin, Inside, Outside, End, Single (BIOES) labeling. Replacements operate on whitespace-delimited tokens using a longest-match strategy with stem-aware look-ups; after substitution we re-tokenize and recompute character offsets, so BIOES boundaries are aligned to the post-normalized token sequence (no insertion/deletion of whitespace is allowed to avoid span drift). In addition, The source of dialectical words in the dataset was a book, which was written by F. Abdullaev in 1965, published by the A.S. Pushkin Institute of Language and Literature of the Academy of Sciences of the Uzbek SSR. The dataset was compiled with the participation of linguistic experts and native speakers of this dialect. The dataset contains 2,249 words and terms, as well as information on the geographical location of each word. For example, a dialect word can be used either in all regions of the Khorezm region, or in certain ones. It should be noted that the Oghuz dialect of the Uzbek language (the words and terms of which are contained in the dataset) dominates not only in the Khorezm region (with more than 2 million speakers) of Uzbekistan, but also in the Tashkhauz region of Turkmenistan, where the number of speakers is more than 0.5 million people. The dataset is available at Mengliev (2025a).

Morphology analysis

The morphological algorithm involves the use of the “Lievenshtein” algorithm, which allows you to identify the difference between letters in a word. The parameter for this algorithm is set to 1, that is, if one letter in a word is written incorrectly, it is automatically replaced with the correct equivalent. Words are checked using a dictionary approach, using a dialect word database, where if successfully detected, the word is replaced with a Karluk (standard Uzbek) version. Otherwise, the word is subject to stemmatization, where the root of the word is identified by cutting off affixes. The root of the word is looked up in the dictionary of stems and compared with the dictionaries of dialect words, where if successfully detected, the dialect word is replaced, otherwise the stemming process is repeated. In total, the stemming process works up to three iterations, if after the third attempt the word is not found in the dialect word dictionaries, it is simply skipped. The pseudocode and flowchart for this algorithm are shown in Fig. 7.

Postprocessing algorithm

To improve the model results, a rule-oriented algorithm was developed, which is activated in post-processing as soon as it receives the result from the neural network. For example, let’s consider one of the examples when the models predict not exactly what was expected, or rather, a confusing choice like “viloyati hokimiyati (B-ORG E-ORG)” and “viloyati (E-LOC) hokimiyati (S-ORG)” was eliminated. Although, both of these examples are existed in training dataset (so, they are correctly predicted), there is a particular case in each situation. Sometimes, we need to show the level of governor office, while in other case we should separate these entities. To solve such problems, rules were implemented where a branch of choice priorities was specified, allowing the algorithm to mark entities in the most correct way. For correct understanding of words above, there are translations: viloyati—region, hokimiyati—governor office.

After implementing rule-based modules for transliteration, dialect normalization and morphological analysis, we evaluated several neural architectures to capture contextual dependencies. The following subsections detail the CNN–BiLSTM, mBERT and SpaCy models.

Neural networks models

It should be noted that all models were trained with our custom dataset (Mengliev, 2025b). Speaking about the used dataset, it can be noted that it contains more than 3,000 marked sentences, and the total number of words is 34,911. In the process of studying the text annotation schemes for the task of recognizing named entities, various options were investigated, including BIO, BIOS, BILOU and others. However, the authors decided to choose the BIOES annotation scheme, where it is possible to clearly define the boundaries of the beginning and end of named entities (Sang & Veenstra, 1999). Meanwhile, speaking about the sources from which the texts were taken to form the dataset, it can be noted that the original source was legal documents that are grammatically correct and have almost no typos in words. Although, other sources were also used, for example, fragments of articles from news blogs and similar services. It should be noted that according to the Uzbek copyright law (Articles 37, 41, 42), individual words and short phrases are not protected by copyright. Moreover, it should be noted that data were collected as previously described in Mengliev et al. (2025).

During the work, the data for training and evaluating the models was divided into three parts, but different proportions were used for each architecture: for mBERT—70% training set, 15% validation and 15% test; for CNN+BiLSTM—80% training and 20% test; for SpaCy—70% training and 30% validation. This decision was due to several factors.

• (1)

  Our corpus as it was mentioned above, contains over 3,000 sentences and 34,911 tokens. When working with a low-resource language, it is important to keep a sufficient volume of the training set so that the model can learn the representation of words and contexts. Therefore, allocating too much to test and validation would be suboptimal.

• (2)

  For mBERT and SpaCy, a classic triad scheme (train/validation/test) was used. This allowed us to control overfitting using validation testing and select the optimal number of epochs. For CNN+BiLSTM, we used a two-part split (train/test), since the model was trained jointly with post-processing and subsequent validation on the test, and the internal regularization effect was provided by dropout and early stopping.

• (3)

  Many studies of NER in low-resource languages use an 80/20 or 70/30 scheme, which ensures comparability of results. Our goal was not to test different splitting schemes, but to compare approaches; therefore, we kept the standard proportions characteristic of specific architectures and justified them with practical considerations.

• (4)

  Early stopping was used for all models, which avoided overfitting even in the absence of separate validation for CNN+BiLSTM. Metric values were observed during training, and the retained models were selected at those epochs where the validation metrics reached the optimum.

Given the above facts, the adopted data split proportions reflect a balance between the need for a sufficient training sample size and the practical limitations of each architecture. Figures 8–10 clearly show the eras to which the model weights were rolled back using dotted lines.

SpaCy

The SpaCy model used standard tok2vec and NER components. The data was converted to .spacy format and split 70:30. Training was conducted with the Adam optimizer (lr = 0.001) and early stopping. The best results were obtained in the 4th epoch: F1 = 76.56, precision = 77.93%, recall = 75.23. The final model (model-best) demonstrated stable quality of entity recognition on test data (Figs. 10, 11).

Analysis of model errors

Post-processing

To enhance text analysis results, a post-processing algorithm was implemented. After the initial output of a language model, the algorithm applies rule-based corrections to improve entity labeling accuracy. As described earlier, the rules determine the most appropriate variant of a named entity for each token.

Figures 13–16 present the updated metrics after post-processing for overall performance and by entity type across mBERT, CNN+BiLSTM, and SpaCy. For clarity and comparison, Table 6 summarizes accuracy, recall, and F1-scores. Among prior works on Uzbek (Mengliev et al., 2023a), only one applied AI-based NER, while others relied on traditional approaches, making direct comparison less meaningful.

Table 6:

Comparative results of our models with existed ones.

Language model type	Precision	Recall	F1-score
BiLSTM+CRF	86.8%	79.2%	82.8%
w/o Data translation	84.6%	80.0%	82.2%
BartNER	92.3%	90.1%	91.2%
w/o Data translation	90.4%	88.0%	89.2%
W2NER	92.4%	90.2%	91.3%
w/o Data translation	92.3%	90.0%	91.1%
UIE	91.2%	87.1%	89.1%
w/o Data translation	87.2%	83.4%	85.2%
UZNER	92.8%	90.6%	91.7%
w/o Data translation	91.9%	90.3%	91.1%
Our solutions (without post-processing)
mBERT	91.8%	92.3%	92.0%
CNN+BiLSTM	93.9%	88.0%	90.8%
SpaCy	77.9%	75.2%	76.5%
Our solutions (with post-processing)
mBERT	95.0%	95.5%	95.2%
CNN+BiLSTM	96.0%	92.0%	94.0%
SpaCy	83.0%	80.0%	81.4%

DOI: 10.7717/peerj-cs.3489/table-6

Our experiments show that without post-processing, results were comparable to existing tools. However, after applying the algorithm, performance improved markedly: F1 reached 94% for CNN+BiLSTM and 95% for mBERT.