Optimizing lexical fitness assessment in L2 Chinese reading texts

Corpus construction

Center for Language Education and Cooperation (CLEC) developed the HSK (Hànyǔ Shuǐpíng Kǎoshì), a standardized test designed to assess the proficiency of non-native Chinese speakers. Over time, the HSK has become a crucial gateway for foreign students to study in China, a requirement for scholarship applications, and an important instrument for assessing instruction in schools. In an increasing number of countries, government agencies and multinational corporations rely on the HSK as a key determinant in hiring decisions, salary adjustments, and employee promotions. The six levels of the HSK exam offer a comprehensive assessment of language competence, catering to learners ranging from novice to expert. Among these, the reading comprehension section spans all levels and primarily evaluates students’ reading skill.

To construct the dataset, we compiled a total of 1,332 HSK reading texts, including those from the HSK Standard Tutorial Levels 1–6, compiled with permission from CLEC/Confucius Institute Headquarters, as well as texts from the New HSK Levels 1-6, also provided by CLEC/Confucius Institute Headquarters (Table 1). The example sections of the reading texts, along with the corresponding responses, were excluded, and the word choice and sentence ordering tasks were manually processed. Sentences were ordered with commas for clauses and periods for endings. To facilitate processing by computational software, the cleaned texts were divided, saved as some txt files, and each entry was assigned a unique identifier. For the subsequent analyses, the dataset was randomly split into training and test set in a 7:3 ratio. The training set was used to train the model, while the test set served to evaluate the accuracy and performance of the classification model.

Rule-based lexical features extraction

As previously discussed, lexical fitness is closely associated with lexical features. Therefore, in the rule-based lexical feature extraction process, we aim to extract features as exhaustively as possible. Consequently, 73 comprehensive and systematically defined lexical features were constructed, encompassing three dimensions: lexical form, lexical meaning, and lexical syntax. Table 2 presents the three primary features and twelve supplementary features that we developed, totaling 73 lexical fitness features, which encompass all vocabulary levels and provide a more comprehensive evaluation of the lexical fitness of the reading texts. Among these, a period, an exclamation point, and a question mark finish the sentence, while a comma, a stop number, and a semicolon finish the clause. Uber Type-Token Ratio (UTTR) and Type-Token Ratio (TTR) are measurement tools used to assess lexical diversity, which refers to the richness of vocabulary employed by language users. This includes the use of a variety of different words, such as synonyms, superlatives, and other related terms, while minimizing repetition of specific words (Jarvis, 2002).

Semantic transparency in Chinese refers to the extent to which the overall meaning of synthetic words and phrases can be inferred from the meanings of their constituent morphemes (Gao, 2015). A word is considered to have high semantic transparency if its morphemic meaning facilitates the understanding of its overall meaning. Conversely, a word exhibits low semantic transparency if its meaning cannot be easily deduced from its constituent morphemes. To measure semantic transparency, we utilize a pre-trained Word2Vec model, which converts both words and their constituent morphemes into vectors. The cosine similarity between the vectors of the morphemes and the corresponding word is then calculated to assess the degree of semantic transparency. Lexical confusion within the reading texts is evaluated through the presence of synonyms, homophone-homograph, and polysemous words. The Modern Chinese Dictionary (7th edition) is consulted to identify polysemous words, which are defined as words with more than three meanings. The proportion of polysemous words is calculated based on their occurrence in the texts. The top 3,000 words from the Modern Chinese Corpus’ word frequency list are considered high-frequency words, and the percentage of such words in each reading text is computed. Ha, Nguyen & Stoeckel (2024) defined the rate of commonly used words, and this measure is applied to gauge the prominence of high-frequency vocabulary within the reading materials. The seriousness or complexity of the reading texts is assessed through the inclusion of written words and idiomatic expressions. To determine the proportion of written words, we refer to $⟨⟨$Expressions of Written Chinese $⟩⟩$ by Chinese scholar Shengli (2006). Since true mastery of vocabulary—encompassing its meaning, usage, and its relationships with other words—is crucial for comprehension and application, vocabulary in the New Chinese Proficiency Test Outline (the Outline) is categorized within the lexical meaning dimension. Different words contribute to the overall meaning of a sentence to varying degrees. Certain words, such as content words (e.g., nouns and verbs), often carry a greater share of the semantic load and are essential for conveying the core meaning of the sentence. In contrast, other words, such as adverbs, adjectives, conjunctions, and function words, typically serve to modify, connect, or complement the main meaning, but their contribution to the overall semantic structure is comparatively smaller. By quantifying lexical contribution, it becomes possible to identify which words are central to the sentence’s meaning and which serve a supplementary or relational function. To compute the contribution of lexical meaning, the Word2Vec model initially transforms all words in a sentence into vector representations. These sentence vectors are then processed using the BERT deep learning model. Subsequently, the cosine similarity between the two types of vectors is calculated and averaged. Finally, to assess the commonness of subject words, the top three topic words for each reading text are determined using TF-IDF. These subject words are then compared to the top 1,000 most common words, and the ratio of commonality is calculated.

Vocabulary in Chinese can be categorized into simple, derived, and compound words based on the structure of morphemes that form the words. Simple words consist of morphemes that do not carry independent meanings. Derived words are typically classified into prefix words (e.g., lăo xiăo ā) and suffix words (e.g., zi er xìng). Chinese affixes are non-word elements that have fixed positions within words, such as the xìng a noun labeling similar to the English suffixes like “-ion” or “-ment” Compound words, on the other hand, are made up of multiple morphemes that carry meaning. These compound words are further divided into categories such as “joint-type,” “modifying-type,” and “subject-verb” combinations. They are generally considered more complex than simple words and derived words (Wang & Yuan, 2018). A higher frequency of content words (those with actual meaning) typically corresponds to a greater number of concepts, which can increase the complexity and reduce the comprehensibility of the text (Sung et al., 2015b). Although function words may lack independent meaning, they play an essential role in maintaining the coherence and structure of the reading text. Consequently, the rates of function words and content words are reported as a single attribute to assess the lexical fitness. The Chinese Proficiency Vocabulary and Chinese Character Leveling Outline, which classifies vocabulary based on frequency, with expert manual intervention as a supplement, forms the basis for determining the A-, B-, C-, and D- words rates. This classification system is grounded in a robust scientific framework and is often employed to evaluate “lexical complexity” in Chinese text assessments (Wu, 2016). Therefore, we include A-, B-, C-, and D- words rates into the category of the lexical syntax of the text’lexical fitness. In addition to the vocabulary-based measures, the study incorporates the average number of semantic roles assigned to words within a sentence. This metric helps assess the influence of grammatical meaning on syntactic complexity (Du, Wang & Wang, 2022). To achieve this, BERT is used to perform semantic role labeling (SRL) in Chinese texts. Furthermore, discourse articulators—words that link, direct, and organize discourse—play a crucial role in maintaining the logical flow between sentences and paragraphs, contributing to the cohesion and comprehensibility of the text. To avoid any misunderstanding, it is necessary to explain the terms “central_sent_num” and “central_sent_ratio”: the sum of the cohesion coefficients for all word pairs between two sentences is referred to as the sentence-level lexical cohesion coefficient, denoted as SentCo. Sentences with an average cohesion coefficient exceeding 80 when compared to other sentences are designated as central sentences. Furthermore, the number and ratio of central sentences are subsequently calculated. For more detailed information on this methodology, readers are referred to the works of Peng, Hu & Wu (2023).

Comparative analysis of classification models

Prior research has extensively explored machine learning approaches for text readability assessment (Heilman, Collins-Thompson & Eskenazi, 2008; Schwarm & Ostendorf, 2005). In comparative studies, Kate et al. (2010) evaluated multiple classification algorithms—including bagged decision trees, decision trees, linear regression, SVM regression and gaussian process regression—for text classification tasks. Their results indicated that bagged decision trees achieved optimal performance with a Pearson correlation coefficient of 0.8173. However, other studies have demonstrated that SVM yield superior results in readability evaluation (Sung et al., 2013, 2015a), suggesting model performance may be task-dependent.

Given that existing studies primarily focus on general text classification tasks with varying model performance across different contexts, we conducted a systematic comparison of multiple established machine learning classifiers to identify the optimal model for our specific research needs. Through this empirical evaluation, we aim to determine the most suitable classification approach for our task. We primarily employed six traditional machine learning classification models for training: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (KNN), logistic regression (LR), classification and regression tree (CART), and random forest (RF). We used the TF-IDF approach to lexical feature extraction in order to provide a contrast to the rule-based feature extraction approach. TF-IDF measures the importance of a word within a collection of documents (corpus) by combining term frequency (TF) and inverse document frequency (IDF). This approach highlights words that occur frequently in a specific document while diminishing the weight of words that are common across all documents, thus emphasizing more relevant and distinctive terms.

Given that the 73 fitness features are measured on different scales, we normalized them to ensure that no single feature dominates the model training process due to its broader range of values. The effectiveness of the model was assessed through a five-fold cross-validation during the training phase. The results, along with the model’s strengths and limitations, were analyzed using the F1-score, which is a balanced measure of precision and recall. Given the rising complexity of text levels, the model predicts HSK 1 text as HSK2 more correctly than HSK6. To more accurately differentiate between various text levels, we also incorporated near accuracy, which evaluates how closely the predicted levels align with the actual text level for the three primary features. This additional indicator provided a more nuanced assessment of the model’s performance in classifying texts across different HSK levels.

A higher F1-score indicates that the model achieves a good balance between precision and recall, two crucial metrics for evaluating classification performance. In comparison to the performance of the 73-feature system developed in this study (F1-score = 0.649–0.879), Table 3 shows that the F1-score for the features based on TF-IDF across the six classification models ranges from 0.41 to 0.67. This demonstrates that the features we constructed offer a better ability to distinguish the lexical adaptation of HSK reading texts and are more effective in performing the task of automatically assessing lexical fitness. Among them, the F1-score of RF reaches 0.879, with the best training results and the strongest evaluation ability.

Next, we conduct comparative experiments between the MHA neural network classifier and the RF model to identify the architecture with superior classification performance for further model refinement.

During the training process, we utilized the pre-trained Transformer model BERT for fine-tuning to reduce training time. The experimental result in Table 4 reveals that the precision, recall, and F1-score of the MHA model were lower than those of the RF classifier, further demonstrating that machine classification model based on rule-based lexical feature extraction performs more effectively. As a result, the RF classification model was ultimately selected as the automatic evaluation model for lexical fitness.

Impact of lexical features on evaluation performance

Table 5 presents the results of an additional analysis focused on the performance of the three primary features in evaluating text levels using the best-performing RF model. With an F1-score of 0.773, lexical meaning demonstrates the strongest evaluation capacity, only 0.106 lower than that of the 73-feature system. Lexical syntax ranks second, also achieving an F1-score of 0.773, but with a slightly smaller difference of 0.11 compared to lexical meaning. The lexical form feature, among the three primary features, exhibits the weakest performance, with an F1-score decrease of 0.067 compared to lexical meaning, the highest-performing feature. Additionally, it shows an F1-score decline of 0.173 when compared to the 73-feature system. Additionally, the near accuracies of the 73-feature system were 0.195, 0.122, and 0.13 higher than those of lexical form, lexical meaning, and lexical syntax, respectively. This suggests that most of the classification errors for the three primary features occurred within adjacent levels, further supporting the notion that the model’s ability to distinguish between adjacent levels of text complexity is relatively strong.

Next, we examine the assessment performance of the twelve supplementary features rooted in three primary features. Table 6 reveals that the strongest assessment ability is exhibited by the lexical meaning of words themselves (word-word), with an F1-score of 0.769. This is followed by the grammatical meaning of the words under the lexical level (word-word) with an F1-score of 0.689, and the lexical meaning of words in relation to the chapter (word-chapter) (F1-score = 0.194). These results suggest that the overall semantics of a Chinese chapter cannot be solely determined by the comprehension of individual subject words, highlighting the importance of context and syntactic structure in understanding the meaning of a text. A comparison of the F1-score rankings for the twelve supplementary features reveals that the word-specific (word-word) features generally exhibit superior assessment performance. This suggests that the lexical properties of Chinese, including lexical form, meaning, and syntax, are primarily determined by vocabulary itself, rather than by other linguistic elements. In contrast to other language types, such as analytic or isolating languages, in agglutinative languages, the meaning of each grammatical category is typically represented by a distinct morpheme that attaches to the root word. This distinction underscores the unique nature of Chinese vocabulary, where lexical form and meaning are largely independent of the grammatical markers that modify them.

Evaluation performance of three primary features at different levels of reading text

To assess the performance of each feature’s capacity for evaluating different reading text levels, we computed the corresponding F1-scores for the lexical form, lexical meaning, lexical syntax, and 73-feature system, respectively, and plotted them as line graphs to observe the assessment performance of each feature on different levels of reading text. Figure 1 demonstrates that the evaluation ability based on 73-feature system consistently outperforms the performance of the three primary features across all reading text levels. Among the various text levels, the HSK2 level shows the highest assessment performance, with an F1-score of 0.936, while the HSK5 level exhibits the weakest performance, with an F1-score of 0.765. Notably, there is little variation in the F1-scores for the lexical form, lexical meaning, lexical syntax, and 73-feature system between HSK1 and HSK2 texts. However, from HSK2 to HSK4, the performance disparities become more pronounced. At the next levels of text, they all seem to agree on a “downward-ascending” trend of performance, with a narrowing of the differences in assessment performance for the advanced reading texts.

Among three primary features, lexical syntax and lexical meaning are generally stronger than lexical form assessment performance for HSK2 to HSK4 texts, but lexical syntax is dominant in HSK5 to HSK6. For HSK1 texts, lexical form exhibits the strongest evaluation capacity, with an F1-score of 0.842. However, as the reading text level increases, the ability of word form to assess the reading text from HSK1 to HSK5 decreases, with its F1-score dropping to 0.643 at HSK5. This decline is reversed at the final stage (HSK6), where lexical form show improvement. In contrast, when evaluating HSK2 texts, lexical meaning provides the best performance, with an F1-score of 0.9. However, the assessment ability of lexical meaning fluctuates more than that of the other features. Notably, during the HSK4 to HSK6, its F1-score drops significantly, from a higher position than that of lexical form and lexical syntax to the lowest point, reaching an F1-score of 0.563. It is not until the final stage of reading text at an advanced level that assessment performance in lexical meaning picks up and is nearly on par with performance in lexical form. For the lexical syntax assessment, performance remains stable from HSK1 to HSK4, with HSK6 texts (F1-score = 0.825) being the best assessment object and HSK5 texts (F1-score = 0.682) the weakest. Lexical syntax assessment performance, however, shows improvement in the final two levels (HSK5 and HSK6), surpassing the performance of lexical form and lexical meaning in these stage.

Optimal model after features screening

To enhance the accuracy and efficiency of the model for automatic vocabulary fitness assessment, we employed a feature extraction algorithm known as mutual information (MI). This algorithm helps eliminate redundant, noisy, or unnecessary features from the original data, thereby improving the performance of the model and making it more suitable for evaluating HSK reading test texts. MI measures the correlation between two variables, indicating the extent to which one variable provides information about another. In this study, MI is used to quantify the dependency of features on goal variables, such as categorical labels (text levels in this case). Initially, we calculated the MI values between the 73 features and their corresponding categorization labels. Table 7 lists the features above the mean of mutual information (MI > 0.27), and a total of 31 features were retained. The retained features were organized under the categories of lexical form, lexical meaning, and lexical syntax. Specifically, five features were retained under lexical form, six under lexical meaning, and twenty under lexical syntax, reflecting the most informative and relevant aspects for the task of automatic vocabulary fitness assessment.

A line graph illustrating the contribution of 31 features to the classification model’s evaluation accuracy is shown in Fig. 2. In the graph, the vertical axis represents the model’s correctness, while the horizontal axis depicts the sequence of features progressively added to the model, ordered by diminishing MI values. As observed in Fig. 2, the model’s accuracy reaches a peak of 0.887 when the 22nd feature is included. After this point, the accuracy fluctuates in a wavy pattern as additional features are incorporated, but it never surpasses the initial value of 0.887. This indicates that the optimal set of features for automatic lexical fitness assessment is limited to the first 22 features, inclusive.

The selected features are categorized as follows:

Lexical form: total number of words in chapter, average number of words in a clause, and average number of strokes in vocabulary.

Lexical meaning: elementary words, intermediate words, advanced words, written word rate, semantic transparency, and homophonic and homomorphic rate.

Lexical syntax: A-words rate, B-words rate, D-words rate, Average number of semantic roles, central_sent_ratio, central_sent_num, local_lexical_cohesion, PN_ratio, global_lexical_coheison, pron_density, pronouns ratio, C_NPS, local_noun_cohesion.

These 22 features form the most informative subset for improving the model’s accuracy in assessing the lexical fitness of reading texts in the HSK evaluation system.

Assessment ability analysis of lexical form

Although lexical form exhibits a lower assessment capacity compared to the other three primary features, their F1-score is only 0.067 below that of the best-performing feature, lexical meaning. This suggests that lexical form can explain lexical fitness to some extent. Among the features under the lexical form, the correlation coefficient between the total number of words relative to the chapter and the level of the reading text amounts to 0.485, making it the best indicator for assessing lexical fit at the lexical form level. To further investigate, we randomly selected texts from HSK1 and HSK6 based on the superior performance of lexical form in assessing these two levels. Our analysis reveals that HSK1 texts typically consist of 2–4 short sentences, mostly composed of simple, everyday exchanges, such as “Thank you” and “You’re welcome.” In contrast, HSK6 texts have far more total words than HSK1 texts, up to hundreds or thousands of words, in a variety of genres, with full narrative or argumentative plots. This discrepancy highlights the shift in focus within the HSK from primarily assessing communicative competence in basic, everyday scenarios at lower levels to evaluating the ability to engage in logical reasoning, information retrieval, and more sophisticated discourse at higher levels. The significant difference in word count across these levels further underscores the sensitivity of text length as an indicator of proficiency, with longer texts signaling greater complexity and depth, which aligns with higher-level linguistic and cognitive abilities required at more advanced stages.

The average number of words in a clause, a new evaluation index introduced in this study, emerges as the second most significant assessment factor under the category of lexical form. This metric is notably more sensitive to text level than the average number of words in a sentence. As previously mentioned, primary reading texts predominantly consist of communicative phrases, where short sentences like “Thanks, I’ll see you tomorrow.” or “Okay, I’ll be home in a minute.” are commonly used for conveying everyday information. These brief exchanges reflect the functional, communicative nature of lower-level reading texts. However, as the reading text level increases, there is a noticeable shift toward more complex expressions. In higher-level texts, clauses are typically expanded to accommodate more logical intent, integrating additional semantic roles and discourse markers. This results in a gradual increase in the number of words within the clauses. It is important to note that sentence meaning is primarily conveyed through clauses, with the semantic richness of individual sentences being less pronounced than that of the clauses within them.

The average number of strokes in vocabulary, a key word shape variable in the best-performing classification model, is another important feature in evaluating lexical fitness. Strokes are a fundamental aspect of the Chinese language due to its logographic nature, where each character is composed of strokes that contribute to its visual complexity. Zhang (2008) posited that reading speed and accuracy are directly influenced by the number of strokes in a character; a higher stroke count typically leads to longer reading times and an increased error rate. Moreover, the average number of strokes per word reflects the cognitive load involved in visual word processing. For learners whose first language employs an alphabetic writing system, such as English or French, these visual complexities pose an additional challenge. In the context of reading proficiency, one- and two-character words dominate lower-level texts, while more complex, multi-character words, particularly idiomatic expressions, become more frequent in higher-level texts. As a result, the average number of strokes per word increases with the text level, contributing to more intricate visual processing. This greater visual complexity also correlates with longer word identification times, particularly for words with more characters, making this feature a key indicator of text difficulty (Fang, 1994; Chen & Su, 2010).

Assessment ability analysis of lexical meaning

Lexical meaning ranks first in terms of assessment ability across the three primary features, with the best assessment performance based on the features of the lexical meaning itself (word-word) (F1-score = 0.769). This dimension encompasses several key features, including synonym rate, homophone-homograph rate, polysemy rate, commonly used words rate, written words rate, idioms rate, and the classification of words by New Chinese Proficiency Test Outline (elementary, intermediate, and advanced words). Notably, five of the nine features—specifically the elementary, intermediate and advanced word rate, written words rate, homophone-homograph rate—show MI values greater than 0.27. These features are identified as the most significant contributors to the model’s optimal performance. In selecting appropriate words for the Chinese proficiency test, it is evident that the emphasis is placed on the meanings of words themselves, with particular attention paid to the leveled vocabulary outlined in the Outline. As Feng, Wang & Huang (2008) argued, the use of a rich written vocabulary effectively enhances the formality of the text, distinguishing it from spoken language. The Chinese language is particularly rich in homophones and polysemous words—terms that have multiple meanings but are pronounced the same or similarly. For example, the word huā can signify both the verb to spend and the noun flower. The prevalence of homophones and polysemous words in advanced reading texts significantly increases the complexity of meaning interpretation. This semantic ambiguity poses challenges for second language learners, who must rely on context to disambiguate the meanings of such words.

Semantic transparency is a crucial aspect of the lexical features integrated into the ideal model for assessing lexical fitness in Chinese reading texts. Laufer (1997) identified the concept of “deceptive transparency,” wherein learners mistakenly infer the meaning of an expression based on the individual meanings of the morphemes it comprises. This misinterpretation, according to Laufer, complicates the process of learning second language vocabulary. The findings of this study further underscore the role of semantic opacity, particularly at higher reading levels. As the reading text level increases, semantic opacity climbs along with it, thus exacerbating the issue of “deceptive transparency.” In Chinese, where most vocabulary is composed of two morphemes, the degree of semantic transparency can vary significantly. Semantic transparency is high when the meaning of a word is directly inferred from its constituent morphemes. In contrast, it is low when a word’s meaning is altered through derivation, metaphor, or other forms of figurative speech, making it harder for learners to deduce its meaning from the morphemes alone. For Chinese, a language in which two or more morphemes combine to form a word, it is essential to comprehend not only the semantics of individual morphemes (assuming they have inherent meaning) but also their combined, extended, and contextual meanings. This approach is crucial for accurately reflecting the learner’s vocabulary acquisition.

Assessment ability analysis of lexical syntax

The lexical syntax feature also demonstrates strong performance in assessing lexical fitness, with only a slight decrease of 0.004 in the F1-score compared to that for lexical meaning. Specifically, the F1-score for words relative to their syntactic features (word-word) reaches 0.689. The features included in the optimal model’s assessment metrics are Pronoun rate, A-words rate, B-words rate and D-word rate. Regarding pronouns, some scholars suggested that an overuse of pronouns can hinder reader comprehension and lead to confusion (Graesser, McNamara & Kulikowich, 2011). However, in the context of Chinese, the presence of additional pronouns can enhance the cohesion of a conversation, thereby improving comprehension. The increasing frequency of pronouns in texts as the reading level rises reflects a richer linguistic structure, contributing to a more coherent discourse. This suggests that the use of pronouns, in moderation, is beneficial for maintaining textual cohesion and comprehensibility, especially as the complexity of the reading material increases. Furthermore, comprehensibility is influenced by various factors beyond pronoun usage, such as pragmatic meaning and unmarked conjunction, which are particularly prevalent in Chinese. These factors work together to shape the overall clarity and accessibility of a text. The A-words, B-words, and D-words are categorized based on frequency, with commonly used words distributed across different levels of text. As the length of the text increases, the frequency of these commonly used words also increase accordingly. Infrequently used words rarely appear in elementary texts, mostly in advanced reading texts.

The grammatical features of words relative to sentences (word-sentence)—average semantic roles—are closely related to reading text level (MI = 0.5), and this difference is largely determined by text form. The primary purpose of elementary reading texts is to facilitate basic communication in diverse contexts, often adopting a conversational style. These texts are typically focused on simple exchanges and straightforward information. In contrast, advanced reading texts challenge students to comprehend more complex elements, such as the story’s initiator, the driving force behind the plot, and intricate details. The higher-level texts serve as a more comprehensive assessment of students’ reading proficiency, testing their ability to navigate nuanced meanings and more sophisticated discourse structures.

A range of syntactic features related to the chapter (word-chapter) are incorporated into the evaluation metrics of the optimal model, including lexical articulation metrics such as central_sent_ratio, central_sent_num, local_lexical_cohesion, local_noun_cohesion, and global_lexical_cohesion, as well as grammatical articulation indicators such as PN_ratio, pron_density, and C_NPS. These articulation features are vital for ensuring that the text is logically structured, allowing readers to follow the progression of ideas smoothly. A greater number and proportion of center sentences mean a tighter articulation factor between words from each center sentence. Global_lexical_cohesion accounts for the coherence between any two sentences across the entire text, while local_lexical_cohesion and local_noun_cohesion specifically focus on the relationship between adjacent clauses. In this study, grammatical articulation indicators correspond to the pronoun-noun ratio, pronoun density, and the number of conjunctions per sentence.It is well-established that pronouns and conjunctions are among the most common and effective articulators, enhancing the coherence of the text (Sanders, Spooren & Noordman, 1992; Crossley et al., 2007). Both grammatical and lexical articulation features have proven to be valuable tools for evaluating lexical fitness, emphasizing that the level of articulation between sentences should correspond to the reading text level.

Evaluating the efficacy of lexical feature extraction for text analysis

To highlight the advantages of the vocabulary-based features developed in this study, we compared them to existing research on Chinese language feature extraction. Table 8 displays the pertinent feature values of 1,332 HSK reading texts based on the vocabulary characteristics of Jiang et al. (2020) and Du, Wang & Wang (2022), using the same corpus as the experimental object. Our comparison reveals that the vocabulary features constructed in this study are the most effective in assessing the lexical fitness of the texts. Specifically, the features we developed outperform those of Jiang et al. (2020) by an F1-score margin of 0.23, and exceed those of Du, Wang & Wang (2022) by 0.068. To address this latter difference, we introduced new variables, such as articulation words and the average number of words per clause. These findings suggest that the features we propose offer a more accurate means of evaluating the lexical fitness of reading texts across different proficiency levels.